2024-04-22T23:28:40.453 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T23:28:40.454 DEBUG:teuthology.run:Teuthology command: teuthology --name adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236 --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose -- /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/orig.config.yaml 2024-04-22T23:28:40.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T23:28:40.589 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236 branch: wip-adk-testing-2024-04-22-1618 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: adking@redhat.com first_in_suite: false job_id: '7669236' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-adk-testing-2024-04-22-1618 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-adk-testing-2024-04-22-1618 sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6551 sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 sleep_before_teardown: 0 subset: 1/16 suite: orch:cephadm suite_branch: wip-adk-testing-2024-04-22-1618 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESVGLMkNtRRe7tETX/sK+TUY0nJ8AYlK9F1xxqjyxN75ulpFLBtUjlxBtE2ciBRMxb5W1aN3R5fVL9RrqD7pWQ= smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCmB9spPn3CIDgj/WILKVNRGeaMpgJey6jQF8fstR2T8K+07kmqyozznVXOu6KkIm6Jw91VigVeqSmLM5K/qMp4= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_22:45:33 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-22T23:28:40.589 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa; will attempt to use it 2024-04-22T23:28:40.590 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks 2024-04-22T23:28:40.590 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T23:28:40.592 INFO:teuthology.task.internal:Checking packages... 2024-04-22T23:28:40.613 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '43be020184947e53516056c9931e1ac5bdbbb1a5' 2024-04-22T23:28:40.613 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:28:40.613 INFO:teuthology.packaging:ref: None 2024-04-22T23:28:40.613 INFO:teuthology.packaging:tag: None 2024-04-22T23:28:40.613 INFO:teuthology.packaging:branch: wip-adk-testing-2024-04-22-1618 2024-04-22T23:28:40.613 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:28:40.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-adk-testing-2024-04-22-1618 2024-04-22T23:28:40.749 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3252.g43be0201 2024-04-22T23:28:40.750 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T23:28:40.757 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T23:28:40.757 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T23:28:40.765 INFO:teuthology.task.internal:Saving configuration 2024-04-22T23:28:40.784 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T23:28:40.789 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T23:28:40.815 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:23:57.527950', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESVGLMkNtRRe7tETX/sK+TUY0nJ8AYlK9F1xxqjyxN75ulpFLBtUjlxBtE2ciBRMxb5W1aN3R5fVL9RrqD7pWQ='} 2024-04-22T23:28:40.835 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:23:57.529779', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCmB9spPn3CIDgj/WILKVNRGeaMpgJey6jQF8fstR2T8K+07kmqyozznVXOu6KkIm6Jw91VigVeqSmLM5K/qMp4='} 2024-04-22T23:28:40.835 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T23:28:40.841 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-22T23:28:40.841 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-22T23:28:40.841 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T23:28:40.914 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33c2cff160>, signals=[15]) 2024-04-22T23:28:40.914 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T23:28:40.921 INFO:teuthology.task.internal:Opening connections... 2024-04-22T23:28:40.921 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:28:40.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:28:40.999 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T23:28:41.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:28:41.073 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T23:28:41.080 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-04-22T23:28:41.098 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-04-22T23:28:41.098 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-22T23:28:41.155 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-22T23:28:41.155 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T23:28:41.156 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T23:28:41.157 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-22T23:28:41.177 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-22T23:28:41.194 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-22T23:28:41.195 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-22T23:28:41.252 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-22T23:28:41.252 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T23:28:41.253 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T23:28:41.253 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-22T23:28:41.276 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T23:28:41.285 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T23:28:41.292 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T23:28:41.292 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-04-22T23:28:41.294 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-22T23:28:41.311 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T23:28:41.317 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T23:28:41.318 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-04-22T23:28:41.350 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-22T23:28:41.381 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T23:28:41.435 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T23:28:41.449 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-22T23:28:41.449 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-22T23:28:41.449 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T23:28:41.449 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T23:28:41.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:28:41.465 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-04-22T23:28:41.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:28:41.466 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-04-22T23:28:41.521 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-437.el9.x86_64 2024-04-22T23:28:41.521 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-437.el9.x86_64 2024-04-22T23:28:41.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-04-22T23:28:41.524 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-437.el9.x86_64 2024-04-22T23:28:41.524 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-437.el9.x86_64 2024-04-22T23:28:41.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-04-22T23:28:43.963 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-22T23:28:47.103 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.0 MB 00:04 2024-04-22T23:28:47.780 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 19 MB 00:02 2024-04-22T23:28:48.879 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-22T23:28:54.238 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-22T23:28:56.114 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-22T23:28:58.514 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-04-22T23:28:59.422 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 793 kB/s | 6.2 MB 00:07 2024-04-22T23:29:00.889 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-22T23:29:02.517 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-22T23:29:03.681 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-04-22T23:29:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T23:29:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T23:29:05.700 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:29:05.702 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:29:05.702 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:29:05.804 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2024-04-22T23:29:06.326 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 11:29:03 PM UTC. 2024-04-22T23:29:06.449 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:29:06.465 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:29:06.466 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-22T23:29:06.466 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-22T23:29:06.466 INFO:teuthology.orchestra.run.smithi031.stderr:Operation aborted. 2024-04-22T23:29:06.521 INFO:teuthology.orchestra.run.smithi031.stdout:Is this ok [y/N]: 2024-04-22T23:29:06.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2024-04-22T23:29:07.033 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 11:29:03 PM UTC. 2024-04-22T23:29:07.153 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:29:07.154 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:29:07.154 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:29:07.154 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:29:07.154 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-22T23:29:07.155 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:29:07.599 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-22T23:29:07.599 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:29:07.599 INFO:teuthology.orchestra.run.smithi031.stdout:Total 574 kB/s | 255 kB 00:00 2024-04-22T23:29:07.621 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:29:07.674 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-22T23:29:07.822 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:29:07.822 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:29:07.828 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:29:07.828 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:29:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:29:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:29:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:29:08.666 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:29:08.667 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:29:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:29:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:29:09.368 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalled: 2024-04-22T23:29:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T23:29:09.368 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:29:09.368 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:29:09.536 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T23:29:09.623 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T23:29:09.624 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T23:29:09.624 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T23:29:09.624 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-22T23:29:09.625 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T23:29:09.630 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T23:29:09.630 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T23:29:09.658 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-04-22T23:29:09.679 DEBUG:teuthology.parallel:result is None 2024-04-22T23:29:09.719 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:29:09.721 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:29:09.721 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:29:09.820 DEBUG:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2024-04-22T23:29:10.343 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 11:29:07 PM UTC. 2024-04-22T23:29:10.467 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:29:10.478 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:29:10.478 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 255 k 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-04-22T23:29:10.479 INFO:teuthology.orchestra.run.smithi044.stderr:Operation aborted. 2024-04-22T23:29:10.550 INFO:teuthology.orchestra.run.smithi044.stdout:Is this ok [y/N]: 2024-04-22T23:29:10.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2024-04-22T23:29:11.067 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 11:29:07 PM UTC. 2024-04-22T23:29:11.188 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:29:11.188 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 255 k 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-04-22T23:29:11.189 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:29:11.493 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-22T23:29:11.493 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:29:11.493 INFO:teuthology.orchestra.run.smithi044.stdout:Total 839 kB/s | 255 kB 00:00 2024-04-22T23:29:11.512 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:29:11.701 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:29:11.701 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:29:11.706 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:29:11.707 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:29:11.861 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:29:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:29:11.952 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:29:12.627 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:29:12.628 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:29:13.356 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:29:13.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:29:13.356 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalled: 2024-04-22T23:29:13.356 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T23:29:13.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:29:13.356 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:29:13.531 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T23:29:13.578 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T23:29:13.578 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T23:29:13.579 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T23:29:13.579 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-22T23:29:13.579 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T23:29:13.611 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-04-22T23:29:13.647 DEBUG:teuthology.parallel:result is None 2024-04-22T23:29:13.648 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T23:29:13.655 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T23:29:13.656 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T23:29:13.658 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T23:29:13.703 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T23:29:13.709 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T23:29:13.715 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T23:29:13.715 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T23:29:13.717 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T23:29:13.773 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T23:29:13.779 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T23:29:13.779 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T23:29:13.786 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T23:29:13.820 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:29:13.831 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:29:13.861 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:29:13.871 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:29:13.872 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T23:29:13.880 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T23:29:13.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T23:29:13.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T23:29:13.908 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T23:29:13.917 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T23:29:13.917 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T23:29:13.951 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T23:29:13.966 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:29:14.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:29:14.101 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:29:14.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:29:14.230 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:29:14.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T23:29:14.298 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:29:14.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:29:14.393 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:29:14.477 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:29:14.516 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:29:14.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T23:29:14.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-04-22T23:29:14.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-22T23:29:14.618 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T23:29:14.660 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T23:29:15.002 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T23:29:15.010 INFO:teuthology.task.internal:Starting timer... 2024-04-22T23:29:15.010 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T23:29:15.077 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T23:29:15.090 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-22T23:29:15.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-04-22T23:29:15.168 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-04-22T23:29:15.170 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-22T23:29:15.245 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-22T23:29:15.246 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T23:29:15.247 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-04-22T23:29:15.288 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-04-22T23:29:15.288 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-22T23:29:15.317 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-22T23:29:15.317 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi044.front.sepia.ceph.com': 'permissive'} 2024-04-22T23:29:15.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T23:29:15.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:29:15.355 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T23:29:15.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:29:15.383 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T23:29:15.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-04-22T23:29:15.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-22T23:29:15.448 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T23:29:15.457 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T23:29:15.473 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T23:29:15.474 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi044.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T23:32:53.226 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi044.front.sepia.ceph.com')] 2024-04-22T23:32:53.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-22T23:32:53.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:32:53.313 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-22T23:32:53.400 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-22T23:32:53.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-22T23:32:53.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:32:53.522 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-22T23:32:53.607 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-22T23:32:53.608 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T23:32:53.618 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T23:32:53.619 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T23:32:53.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T23:32:53.622 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T23:32:53.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T23:32:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T23:32:53.669 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T23:32:53.680 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T23:32:53.695 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-04-22T23:32:53.695 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T23:32:53.706 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-04-22T23:32:53.720 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-22T23:32:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T23:32:53.732 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-22T23:32:53.734 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T23:32:53.745 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T23:32:53.759 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T23:32:53.793 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-22T23:32:53.796 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T23:32:53.796 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-22T23:32:53.796 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.796 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.796 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.796 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.817 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-22T23:32:53.821 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T23:32:53.822 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-22T23:32:53.822 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.822 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.822 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.822 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:32:53.823 INFO:teuthology.run_tasks:Running task install... 2024-04-22T23:32:53.832 DEBUG:teuthology.task.install:project ceph 2024-04-22T23:32:53.832 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5'}} 2024-04-22T23:32:53.832 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5'} 2024-04-22T23:32:53.832 INFO:teuthology.task.install:Using flavor: default 2024-04-22T23:32:53.839 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T23:32:53.839 INFO:teuthology.task.install:extra packages: [] 2024-04-22T23:32:53.839 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-22T23:32:53.840 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-22T23:32:53.840 INFO:teuthology.packaging:ref: None 2024-04-22T23:32:53.840 INFO:teuthology.packaging:tag: v18.2.1 2024-04-22T23:32:53.840 INFO:teuthology.packaging:branch: None 2024-04-22T23:32:53.840 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:54.254 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.1^{} -> None 2024-04-22T23:32:54.255 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-22T23:32:56.549 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-22T23:32:56.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-22T23:32:56.553 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-22T23:32:56.553 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-22T23:32:56.553 INFO:teuthology.packaging:ref: None 2024-04-22T23:32:56.553 INFO:teuthology.packaging:tag: v18.2.1 2024-04-22T23:32:56.553 INFO:teuthology.packaging:branch: None 2024-04-22T23:32:56.553 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:56.554 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-22T23:32:56.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-22T23:32:56.785 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-04-22T23:32:56.785 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-22T23:32:56.789 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-04-22T23:32:56.789 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-22T23:32:56.835 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T23:32:56.835 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:32:56.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T23:32:56.847 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T23:32:56.847 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:32:56.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T23:32:56.866 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-22T23:32:56.866 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-22T23:32:56.866 INFO:teuthology.packaging:ref: None 2024-04-22T23:32:56.866 INFO:teuthology.packaging:tag: v18.2.1 2024-04-22T23:32:56.866 INFO:teuthology.packaging:branch: None 2024-04-22T23:32:56.866 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:56.867 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T23:32:56.878 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-22T23:32:56.878 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-22T23:32:56.878 INFO:teuthology.packaging:ref: None 2024-04-22T23:32:56.878 INFO:teuthology.packaging:tag: v18.2.1 2024-04-22T23:32:56.879 INFO:teuthology.packaging:branch: None 2024-04-22T23:32:56.879 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:56.879 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T23:32:56.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T23:32:56.949 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T23:32:57.018 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T23:32:57.028 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T23:32:57.085 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-04-22T23:32:57.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-04-22T23:32:57.095 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-22T23:32:57.097 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-22T23:32:57.328 INFO:teuthology.orchestra.run.smithi031.stdout:48 files removed 2024-04-22T23:32:57.337 INFO:teuthology.orchestra.run.smithi044.stdout:48 files removed 2024-04-22T23:32:57.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-04-22T23:32:57.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw 2024-04-22T23:32:57.774 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 517 kB/s | 76 kB 00:00 2024-04-22T23:32:57.793 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 486 kB/s | 76 kB 00:00 2024-04-22T23:32:57.967 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 64 kB/s | 9.4 kB 00:00 2024-04-22T23:32:57.985 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 64 kB/s | 9.4 kB 00:00 2024-04-22T23:32:58.169 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 15 kB/s | 2.2 kB 00:00 2024-04-22T23:32:58.267 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 8.2 kB/s | 2.2 kB 00:00 2024-04-22T23:32:59.172 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-22T23:33:00.094 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-22T23:33:02.112 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-22T23:33:03.387 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 19 MB 00:02 2024-04-22T23:33:07.296 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-22T23:33:07.723 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.2 MB 00:00 2024-04-22T23:33:08.780 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-22T23:33:09.197 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-22T23:33:10.292 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-22T23:33:14.714 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 21 MB 00:05 2024-04-22T23:33:15.690 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-22T23:33:17.876 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-22T23:33:17.879 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T23:33:17.880 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout:Install 28 Packages 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-04-22T23:33:17.881 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:17.882 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 53 M 2024-04-22T23:33:17.882 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:33:18.533 INFO:teuthology.orchestra.run.smithi044.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 29 MB/s | 5.2 MB 00:00 2024-04-22T23:33:18.558 INFO:teuthology.orchestra.run.smithi044.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 967 kB/s | 24 kB 00:00 2024-04-22T23:33:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-04-22T23:33:18.625 INFO:teuthology.orchestra.run.smithi044.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-22T23:33:18.726 INFO:teuthology.orchestra.run.smithi044.stdout:(5/30): librgw2-18.2.1-0.el9.x86_64.rpm 45 MB/s | 4.5 MB 00:00 2024-04-22T23:33:18.751 INFO:teuthology.orchestra.run.smithi044.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-22T23:33:18.776 INFO:teuthology.orchestra.run.smithi044.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-04-22T23:33:18.801 INFO:teuthology.orchestra.run.smithi044.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-04-22T23:33:18.826 INFO:teuthology.orchestra.run.smithi044.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 13 MB/s | 321 kB 00:00 2024-04-22T23:33:18.919 INFO:teuthology.orchestra.run.smithi044.stdout:(10/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 14 MB/s | 7.7 MB 00:00 2024-04-22T23:33:18.944 INFO:teuthology.orchestra.run.smithi044.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 2.5 MB/s | 297 kB 00:00 2024-04-22T23:33:18.969 INFO:teuthology.orchestra.run.smithi044.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-04-22T23:33:19.178 INFO:teuthology.orchestra.run.smithi044.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-04-22T23:33:19.836 INFO:teuthology.orchestra.run.smithi044.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-04-22T23:33:19.880 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-22T23:33:19.887 INFO:teuthology.orchestra.run.smithi044.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 113 kB/s | 107 kB 00:00 2024-04-22T23:33:19.987 INFO:teuthology.orchestra.run.smithi044.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 157 kB/s | 160 kB 00:01 2024-04-22T23:33:20.654 INFO:teuthology.orchestra.run.smithi044.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 63 kB/s | 42 kB 00:00 2024-04-22T23:33:20.854 INFO:teuthology.orchestra.run.smithi044.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 302 kB/s | 292 kB 00:00 2024-04-22T23:33:20.882 INFO:teuthology.orchestra.run.smithi044.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 492 kB/s | 111 kB 00:00 2024-04-22T23:33:21.171 INFO:teuthology.orchestra.run.smithi044.stdout:(20/30): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-22T23:33:21.196 INFO:teuthology.orchestra.run.smithi044.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 904 kB/s | 309 kB 00:00 2024-04-22T23:33:21.238 INFO:teuthology.orchestra.run.smithi044.stdout:(22/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-04-22T23:33:21.264 INFO:teuthology.orchestra.run.smithi044.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 723 kB/s | 49 kB 00:00 2024-04-22T23:33:21.290 INFO:teuthology.orchestra.run.smithi044.stdout:(24/30): librdkafka-1.6.1-102.el9.x86_64.rpm 456 kB/s | 662 kB 00:01 2024-04-22T23:33:21.314 INFO:teuthology.orchestra.run.smithi044.stdout:(25/30): libunwind-1.6.2-1.el9.x86_64.rpm 890 kB/s | 67 kB 00:00 2024-04-22T23:33:21.398 INFO:teuthology.orchestra.run.smithi044.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-04-22T23:33:21.449 INFO:teuthology.orchestra.run.smithi044.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-22T23:33:21.524 INFO:teuthology.orchestra.run.smithi044.stdout:(28/30): librados2-18.2.1-0.el9.x86_64.rpm 26 MB/s | 3.3 MB 00:00 2024-04-22T23:33:21.583 INFO:teuthology.orchestra.run.smithi044.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 23 MB/s | 3.0 MB 00:00 2024-04-22T23:33:21.608 INFO:teuthology.orchestra.run.smithi044.stdout:(30/30): re2-20211101-3.el9.x86_64.rpm 603 kB/s | 192 kB 00:00 2024-04-22T23:33:21.610 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:33:21.611 INFO:teuthology.orchestra.run.smithi044.stdout:Total 14 MB/s | 53 MB 00:03 2024-04-22T23:33:21.687 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:33:21.728 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:33:21.728 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:33:22.140 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:33:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:22.142 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-22T23:33:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T23:33:22.143 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-04-22T23:33:22.144 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout:Install 28 Packages 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:22.145 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 53 M 2024-04-22T23:33:22.146 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:33:22.174 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:33:22.175 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:33:22.548 INFO:teuthology.orchestra.run.smithi031.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 27 MB/s | 5.2 MB 00:00 2024-04-22T23:33:22.573 INFO:teuthology.orchestra.run.smithi031.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 972 kB/s | 24 kB 00:00 2024-04-22T23:33:22.607 INFO:teuthology.orchestra.run.smithi031.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 19 MB/s | 658 kB 00:00 2024-04-22T23:33:22.640 INFO:teuthology.orchestra.run.smithi031.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-22T23:33:22.741 INFO:teuthology.orchestra.run.smithi031.stdout:(5/30): librgw2-18.2.1-0.el9.x86_64.rpm 45 MB/s | 4.5 MB 00:00 2024-04-22T23:33:22.766 INFO:teuthology.orchestra.run.smithi031.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-22T23:33:22.791 INFO:teuthology.orchestra.run.smithi031.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-04-22T23:33:22.817 INFO:teuthology.orchestra.run.smithi031.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-04-22T23:33:22.850 INFO:teuthology.orchestra.run.smithi031.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 9.5 MB/s | 321 kB 00:00 2024-04-22T23:33:22.950 INFO:teuthology.orchestra.run.smithi031.stdout:(10/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 13 MB/s | 7.7 MB 00:00 2024-04-22T23:33:22.975 INFO:teuthology.orchestra.run.smithi031.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 2.3 MB/s | 297 kB 00:00 2024-04-22T23:33:23.001 INFO:teuthology.orchestra.run.smithi031.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-04-22T23:33:23.218 INFO:teuthology.orchestra.run.smithi031.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-04-22T23:33:23.584 INFO:teuthology.orchestra.run.smithi031.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 176 kB/s | 107 kB 00:00 2024-04-22T23:33:23.610 INFO:teuthology.orchestra.run.smithi031.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 115 kB/s | 45 kB 00:00 2024-04-22T23:33:23.618 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:33:23.702 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-22T23:33:23.702 INFO:teuthology.orchestra.run.smithi031.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 229 kB/s | 160 kB 00:00 2024-04-22T23:33:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-22T23:33:23.788 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-22T23:33:23.835 INFO:teuthology.orchestra.run.smithi031.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 317 kB/s | 42 kB 00:00 2024-04-22T23:33:24.152 INFO:teuthology.orchestra.run.smithi031.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-04-22T23:33:24.162 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-22T23:33:24.219 INFO:teuthology.orchestra.run.smithi031.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 289 kB/s | 111 kB 00:00 2024-04-22T23:33:24.220 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-22T23:33:24.313 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-22T23:33:24.395 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-04-22T23:33:24.427 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-22T23:33:24.486 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-22T23:33:24.524 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-22T23:33:24.569 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-04-22T23:33:24.620 INFO:teuthology.orchestra.run.smithi031.stdout:(20/30): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-22T23:33:24.635 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-22T23:33:24.645 INFO:teuthology.orchestra.run.smithi031.stdout:(21/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 282 kB/s | 292 kB 00:01 2024-04-22T23:33:24.659 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-22T23:33:24.708 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-22T23:33:24.795 INFO:teuthology.orchestra.run.smithi031.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 481 kB/s | 309 kB 00:00 2024-04-22T23:33:24.818 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-22T23:33:24.845 INFO:teuthology.orchestra.run.smithi031.stdout:(23/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 115 kB/s | 26 kB 00:00 2024-04-22T23:33:24.850 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-22T23:33:24.890 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-22T23:33:24.904 INFO:teuthology.orchestra.run.smithi031.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 841 kB/s | 67 kB 00:00 2024-04-22T23:33:24.923 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-22T23:33:24.946 INFO:teuthology.orchestra.run.smithi031.stdout:(25/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-04-22T23:33:24.950 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-22T23:33:24.987 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-22T23:33:24.996 INFO:teuthology.orchestra.run.smithi031.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-22T23:33:25.019 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-22T23:33:25.046 INFO:teuthology.orchestra.run.smithi031.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-22T23:33:25.063 INFO:teuthology.orchestra.run.smithi031.stdout:(28/30): liboath-2.6.7-2.el9.x86_64.rpm 116 kB/s | 49 kB 00:00 2024-04-22T23:33:25.130 INFO:teuthology.orchestra.run.smithi031.stdout:(29/30): librados2-18.2.1-0.el9.x86_64.rpm 24 MB/s | 3.3 MB 00:00 2024-04-22T23:33:25.189 INFO:teuthology.orchestra.run.smithi031.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-04-22T23:33:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:33:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Total 17 MB/s | 53 MB 00:03 2024-04-22T23:33:25.268 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:33:25.309 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:33:25.309 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:33:25.333 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-22T23:33:25.373 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-22T23:33:25.465 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-22T23:33:25.495 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-04-22T23:33:25.531 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-22T23:33:25.754 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:33:25.754 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:33:25.953 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-22T23:33:25.984 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-22T23:33:26.056 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-22T23:33:26.136 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-04-22T23:33:26.237 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-04-22T23:33:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:33:27.273 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-22T23:33:27.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-22T23:33:27.360 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-22T23:33:27.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-22T23:33:27.801 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-22T23:33:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-04-22T23:33:27.975 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-04-22T23:33:28.003 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-22T23:33:28.065 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-04-22T23:33:28.094 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-22T23:33:28.137 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-04-22T23:33:28.204 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-22T23:33:28.229 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-22T23:33:28.278 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-04-22T23:33:28.389 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-22T23:33:28.395 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-22T23:33:28.425 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-22T23:33:28.427 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-04-22T23:33:28.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-22T23:33:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-22T23:33:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-04-22T23:33:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-22T23:33:28.580 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-22T23:33:28.894 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-22T23:33:28.935 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-22T23:33:28.993 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-22T23:33:29.029 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-22T23:33:29.132 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-22T23:33:29.132 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T23:33:29.132 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T23:33:29.132 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:29.167 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-04-22T23:33:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-22T23:33:29.198 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-04-22T23:33:29.234 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-22T23:33:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-04-22T23:33:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-22T23:33:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-04-22T23:33:29.820 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-04-22T23:33:29.864 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-04-22T23:33:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-22T23:33:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-22T23:33:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-04-22T23:33:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-22T23:33:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-04-22T23:33:32.764 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T23:33:32.764 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T23:33:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-22T23:33:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:39.938 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-22T23:33:39.963 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-22T23:33:39.963 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:33:39.964 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T23:33:39.964 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:33:39.964 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:33:39.964 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:39.965 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-22T23:33:40.049 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-22T23:33:40.049 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-22T23:33:41.179 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-22T23:33:41.180 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-22T23:33:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-22T23:33:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-22T23:33:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-04-22T23:33:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-22T23:33:41.181 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-04-22T23:33:41.946 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-22T23:33:41.946 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:41.946 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-22T23:33:41.946 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.946 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:33:41.946 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.947 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:41.948 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:33:42.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-test 2024-04-22T23:33:42.735 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-04-22T23:33:42.735 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-04-22T23:33:42.736 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:42.774 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:27 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:33:42.897 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:33:42.897 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:42.898 INFO:teuthology.orchestra.run.smithi044.stdout:Install 4 Packages 2024-04-22T23:33:42.899 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:42.899 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 40 M 2024-04-22T23:33:42.899 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 166 M 2024-04-22T23:33:42.899 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:33:43.165 INFO:teuthology.orchestra.run.smithi044.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 165 kB 00:00 2024-04-22T23:33:43.359 INFO:teuthology.orchestra.run.smithi044.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 380 kB/s | 64 kB 00:00 2024-04-22T23:33:43.401 INFO:teuthology.orchestra.run.smithi044.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 817 kB/s | 305 kB 00:00 2024-04-22T23:33:43.512 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-22T23:33:43.541 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-04-22T23:33:43.541 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:33:43.541 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T23:33:43.541 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:33:43.541 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:33:43.541 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:43.543 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-22T23:33:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-22T23:33:43.623 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-22T23:33:43.828 INFO:teuthology.orchestra.run.smithi044.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 50 MB/s | 40 MB 00:00 2024-04-22T23:33:43.831 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:33:43.832 INFO:teuthology.orchestra.run.smithi044.stdout:Total 43 MB/s | 40 MB 00:00 2024-04-22T23:33:43.859 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:33:43.882 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:33:43.882 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:33:44.213 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:33:44.213 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:33:44.789 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-04-22T23:33:44.834 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-04-22T23:33:44.835 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-22T23:33:44.836 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-04-22T23:33:44.863 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-22T23:33:44.961 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-22T23:33:44.988 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:45.722 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:33:45.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-04-22T23:33:46.519 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:27 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:33:46.643 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:33:46.643 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-04-22T23:33:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:46.645 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 40 M 2024-04-22T23:33:46.645 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 166 M 2024-04-22T23:33:46.645 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:33:46.900 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 165 kB 00:00 2024-04-22T23:33:47.006 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 601 kB/s | 64 kB 00:00 2024-04-22T23:33:47.057 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-22T23:33:47.550 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 52 MB/s | 40 MB 00:00 2024-04-22T23:33:47.553 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:33:47.553 INFO:teuthology.orchestra.run.smithi031.stdout:Total 44 MB/s | 40 MB 00:00 2024-04-22T23:33:47.580 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:33:47.602 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:33:47.602 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:33:47.938 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:33:47.939 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:33:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:33:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-22T23:33:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-22T23:33:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-04-22T23:33:48.916 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-04-22T23:33:48.996 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-04-22T23:33:50.919 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-04-22T23:33:50.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-04-22T23:33:50.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-04-22T23:33:50.919 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:51.196 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:33:51.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph 2024-04-22T23:33:51.991 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:36 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:33:52.118 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:33:52.120 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:52.120 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T23:33:52.120 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:52.120 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T23:33:52.121 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T23:33:52.122 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout:Install 33 Packages 2024-04-22T23:33:52.123 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:33:52.124 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 30 M 2024-04-22T23:33:52.124 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 110 M 2024-04-22T23:33:52.124 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:33:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-04-22T23:33:52.684 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-04-22T23:33:52.721 INFO:teuthology.orchestra.run.smithi044.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 59 kB/s | 6.4 kB 00:00 2024-04-22T23:33:52.771 INFO:teuthology.orchestra.run.smithi044.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.1 MB/s | 1.4 MB 00:00 2024-04-22T23:33:52.830 INFO:teuthology.orchestra.run.smithi044.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.7 MB/s | 2.1 MB 00:00 2024-04-22T23:33:52.854 INFO:teuthology.orchestra.run.smithi044.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.6 MB/s | 242 kB 00:00 2024-04-22T23:33:52.930 INFO:teuthology.orchestra.run.smithi044.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-22T23:33:53.155 INFO:teuthology.orchestra.run.smithi044.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 46 MB/s | 18 MB 00:00 2024-04-22T23:33:53.956 INFO:teuthology.orchestra.run.smithi044.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 230 kB/s | 253 kB 00:01 2024-04-22T23:33:53.972 INFO:teuthology.orchestra.run.smithi044.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 130 kB/s | 106 kB 00:00 2024-04-22T23:33:54.725 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-04-22T23:33:54.725 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-04-22T23:33:54.725 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-04-22T23:33:54.725 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-22T23:33:54.748 INFO:teuthology.orchestra.run.smithi044.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 171 kB/s | 135 kB 00:00 2024-04-22T23:33:54.773 INFO:teuthology.orchestra.run.smithi044.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 156 kB/s | 125 kB 00:00 2024-04-22T23:33:54.932 INFO:teuthology.orchestra.run.smithi044.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-22T23:33:54.978 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-22T23:33:54.978 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:54.978 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:33:54.978 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-04-22T23:33:54.978 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T23:33:54.979 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:54.979 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:33:54.982 INFO:teuthology.orchestra.run.smithi044.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 696 kB/s | 35 kB 00:00 2024-04-22T23:33:55.066 INFO:teuthology.orchestra.run.smithi044.stdout:(13/33): python3-pytz-2021.1-5.el9.noarch.rpm 931 kB/s | 51 kB 00:00 2024-04-22T23:33:55.108 INFO:teuthology.orchestra.run.smithi044.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-22T23:33:55.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-04-22T23:33:55.249 INFO:teuthology.orchestra.run.smithi044.stdout:(15/33): python3-cryptography-36.0.1-4.el9.x86_ 541 kB/s | 1.2 MB 00:02 2024-04-22T23:33:55.500 INFO:teuthology.orchestra.run.smithi044.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 111 kB/s | 43 kB 00:00 2024-04-22T23:33:55.667 INFO:teuthology.orchestra.run.smithi044.stdout:(17/33): python3-urllib3-1.26.5-5.el9.noarch.rp 234 kB/s | 215 kB 00:00 2024-04-22T23:33:55.742 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:36 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:33:55.808 INFO:teuthology.orchestra.run.smithi044.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 307 kB/s | 172 kB 00:00 2024-04-22T23:33:55.851 INFO:teuthology.orchestra.run.smithi044.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-04-22T23:33:55.868 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-04-22T23:33:55.871 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T23:33:55.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:33:55.873 INFO:teuthology.orchestra.run.smithi031.stdout:Install 33 Packages 2024-04-22T23:33:55.874 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:33:55.874 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 30 M 2024-04-22T23:33:55.874 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 110 M 2024-04-22T23:33:55.874 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:33:55.909 INFO:teuthology.orchestra.run.smithi044.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 176 kB/s | 18 kB 00:00 2024-04-22T23:33:55.951 INFO:teuthology.orchestra.run.smithi044.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 234 kB/s | 23 kB 00:00 2024-04-22T23:33:55.976 INFO:teuthology.orchestra.run.smithi044.stdout:(22/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 35 kB/s | 11 kB 00:00 2024-04-22T23:33:56.010 INFO:teuthology.orchestra.run.smithi044.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-04-22T23:33:56.052 INFO:teuthology.orchestra.run.smithi044.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-04-22T23:33:56.127 INFO:teuthology.orchestra.run.smithi044.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 675 kB/s | 79 kB 00:00 2024-04-22T23:33:56.169 INFO:teuthology.orchestra.run.smithi044.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 241 kB/s | 46 kB 00:00 2024-04-22T23:33:56.194 INFO:teuthology.orchestra.run.smithi044.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-22T23:33:56.228 INFO:teuthology.orchestra.run.smithi044.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-04-22T23:33:56.295 INFO:teuthology.orchestra.run.smithi044.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 357 kB/s | 36 kB 00:00 2024-04-22T23:33:56.362 INFO:teuthology.orchestra.run.smithi044.stdout:(30/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 467 kB/s | 90 kB 00:00 2024-04-22T23:33:56.395 INFO:teuthology.orchestra.run.smithi044.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-22T23:33:56.450 INFO:teuthology.orchestra.run.smithi031.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 63 kB/s | 6.4 kB 00:00 2024-04-22T23:33:56.454 INFO:teuthology.orchestra.run.smithi044.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.6 MB/s | 427 kB 00:00 2024-04-22T23:33:56.479 INFO:teuthology.orchestra.run.smithi044.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 171 kB/s | 20 kB 00:00 2024-04-22T23:33:56.485 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:33:56.485 INFO:teuthology.orchestra.run.smithi044.stdout:Total 6.9 MB/s | 30 MB 00:04 2024-04-22T23:33:56.500 INFO:teuthology.orchestra.run.smithi031.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 9.5 MB/s | 1.4 MB 00:00 2024-04-22T23:33:56.550 INFO:teuthology.orchestra.run.smithi031.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-04-22T23:33:56.576 INFO:teuthology.orchestra.run.smithi031.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-04-22T23:33:56.607 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:33:56.649 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:33:56.649 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:33:56.667 INFO:teuthology.orchestra.run.smithi031.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-22T23:33:56.953 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:33:56.954 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:33:56.976 INFO:teuthology.orchestra.run.smithi031.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 37 MB/s | 18 MB 00:00 2024-04-22T23:33:57.026 INFO:teuthology.orchestra.run.smithi031.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 562 kB/s | 253 kB 00:00 2024-04-22T23:33:57.076 INFO:teuthology.orchestra.run.smithi031.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-04-22T23:33:57.101 INFO:teuthology.orchestra.run.smithi031.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-04-22T23:33:57.127 INFO:teuthology.orchestra.run.smithi031.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-04-22T23:33:57.160 INFO:teuthology.orchestra.run.smithi031.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 2.5 MB/s | 1.2 MB 00:00 2024-04-22T23:33:57.186 INFO:teuthology.orchestra.run.smithi031.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-04-22T23:33:57.286 INFO:teuthology.orchestra.run.smithi031.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 277 kB/s | 35 kB 00:00 2024-04-22T23:33:57.336 INFO:teuthology.orchestra.run.smithi031.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 340 kB/s | 51 kB 00:00 2024-04-22T23:33:57.361 INFO:teuthology.orchestra.run.smithi031.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-04-22T23:33:57.387 INFO:teuthology.orchestra.run.smithi031.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 663 kB/s | 172 kB 00:00 2024-04-22T23:33:57.479 INFO:teuthology.orchestra.run.smithi031.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 305 kB/s | 43 kB 00:00 2024-04-22T23:33:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 643 kB/s | 172 kB 00:00 2024-04-22T23:33:57.654 INFO:teuthology.orchestra.run.smithi031.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 61 kB/s | 11 kB 00:00 2024-04-22T23:33:57.679 INFO:teuthology.orchestra.run.smithi031.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-04-22T23:33:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 234 kB/s | 18 kB 00:00 2024-04-22T23:33:57.730 INFO:teuthology.orchestra.run.smithi031.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-04-22T23:33:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-04-22T23:33:57.780 INFO:teuthology.orchestra.run.smithi031.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-04-22T23:33:57.814 INFO:teuthology.orchestra.run.smithi031.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 553 kB/s | 46 kB 00:00 2024-04-22T23:33:57.839 INFO:teuthology.orchestra.run.smithi031.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 943 kB/s | 79 kB 00:00 2024-04-22T23:33:57.864 INFO:teuthology.orchestra.run.smithi031.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-04-22T23:33:57.890 INFO:teuthology.orchestra.run.smithi031.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-22T23:33:57.907 INFO:teuthology.orchestra.run.smithi031.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-04-22T23:33:57.932 INFO:teuthology.orchestra.run.smithi031.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 533 kB/s | 36 kB 00:00 2024-04-22T23:33:57.957 INFO:teuthology.orchestra.run.smithi031.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-22T23:33:57.983 INFO:teuthology.orchestra.run.smithi031.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-04-22T23:33:58.008 INFO:teuthology.orchestra.run.smithi031.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-04-22T23:33:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:33:58.009 INFO:teuthology.orchestra.run.smithi031.stdout:Total 14 MB/s | 30 MB 00:02 2024-04-22T23:33:58.130 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:33:58.155 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:33:58.171 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:33:58.171 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:33:58.215 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-22T23:33:58.257 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-22T23:33:58.289 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-22T23:33:58.388 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-22T23:33:58.473 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-22T23:33:58.474 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:33:58.474 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:33:58.527 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-22T23:33:58.561 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-22T23:33:58.590 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-22T23:33:58.625 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-22T23:33:58.658 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-22T23:33:58.699 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-22T23:33:58.743 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-22T23:33:58.776 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-22T23:33:58.806 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-22T23:33:58.884 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-22T23:33:59.038 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-22T23:33:59.119 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-22T23:33:59.169 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-22T23:33:59.206 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-22T23:33:59.257 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-22T23:33:59.321 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-22T23:33:59.522 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-22T23:33:59.577 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-22T23:33:59.651 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-22T23:33:59.707 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:33:59.750 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-22T23:33:59.784 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-22T23:33:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-22T23:33:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-22T23:33:59.909 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-22T23:33:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-22T23:34:00.016 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-22T23:34:00.069 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-04-22T23:34:00.091 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-22T23:34:00.093 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-22T23:34:00.120 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-22T23:34:00.120 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:00.120 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T23:34:00.120 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:34:00.120 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:34:00.120 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:00.146 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-22T23:34:00.180 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-22T23:34:00.210 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-22T23:34:00.245 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-22T23:34:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-22T23:34:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-22T23:34:00.354 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-22T23:34:00.387 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-22T23:34:00.426 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-22T23:34:00.504 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-22T23:34:00.774 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-22T23:34:00.854 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-22T23:34:00.931 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-22T23:34:00.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-22T23:34:01.027 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-22T23:34:01.091 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-22T23:34:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-22T23:34:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-22T23:34:01.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-22T23:34:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-22T23:34:01.570 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-22T23:34:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-22T23:34:01.728 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-22T23:34:01.764 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-22T23:34:01.764 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:01.764 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T23:34:01.764 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:34:01.764 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:34:01.764 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-04-22T23:34:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-22T23:34:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-04-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:02.262 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-22T23:34:02.287 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-22T23:34:02.287 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:02.287 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T23:34:02.288 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:34:02.288 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:34:02.288 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:02.504 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-22T23:34:02.529 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-22T23:34:02.529 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:02.529 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T23:34:02.529 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:34:02.529 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:34:02.529 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:02.594 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-22T23:34:03.505 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-22T23:34:03.533 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-04-22T23:34:03.533 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:03.534 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T23:34:03.534 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:34:03.534 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:34:03.534 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:03.998 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-22T23:34:04.022 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-04-22T23:34:04.022 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:04.022 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T23:34:04.023 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:34:04.023 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:34:04.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-22T23:34:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-04-22T23:34:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T23:34:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:34:04.256 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:34:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:04.268 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-22T23:34:04.268 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-04-22T23:34:04.268 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-22T23:34:04.269 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-22T23:34:04.270 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-22T23:34:04.271 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-22T23:34:04.321 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-22T23:34:05.116 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-22T23:34:05.116 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T23:34:05.117 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T23:34:05.118 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:05.119 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:05.460 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-base 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-22T23:34:05.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-22T23:34:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-22T23:34:05.933 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-22T23:34:05.933 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-22T23:34:05.934 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-22T23:34:05.934 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-22T23:34:05.934 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-22T23:34:05.934 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-22T23:34:05.934 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-22T23:34:05.934 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-22T23:34:06.003 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:51 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:06.085 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:06.130 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:06.131 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:06.131 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:06.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install cephadm 2024-04-22T23:34:06.731 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:51 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-04-22T23:34:06.802 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T23:34:06.803 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T23:34:06.804 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:06.805 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:06.856 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:06.857 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:06.858 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:06.858 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:06.858 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:06.858 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 221 k 2024-04-22T23:34:06.858 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 213 k 2024-04-22T23:34:06.858 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:06.920 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.5 MB/s | 221 kB 00:00 2024-04-22T23:34:06.921 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:06.921 INFO:teuthology.orchestra.run.smithi044.stdout:Total 3.4 MB/s | 221 kB 00:00 2024-04-22T23:34:06.921 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:06.927 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:06.927 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:06.930 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:06.930 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:06.998 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:07.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-04-22T23:34:07.350 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:07.443 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:07.701 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:48 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:07.754 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:07.782 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:07.826 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:07.827 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:07.827 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:07.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-04-22T23:34:08.021 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:08.022 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-04-22T23:34:08.022 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:08.022 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:08.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-immutable-object-cache 2024-04-22T23:34:08.428 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:49 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:08.553 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:08.554 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 221 k 2024-04-22T23:34:08.555 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 213 k 2024-04-22T23:34:08.555 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:08.622 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.2 MB/s | 221 kB 00:00 2024-04-22T23:34:08.623 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:08.623 INFO:teuthology.orchestra.run.smithi031.stdout:Total 3.1 MB/s | 221 kB 00:00 2024-04-22T23:34:08.623 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:08.628 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:08.628 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:08.631 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:08.632 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:08.701 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:08.761 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:53 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:08.885 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:08.885 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repo Size 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 142 k 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 395 k 2024-04-22T23:34:08.886 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:08.952 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.1 MB/s | 142 kB 00:00 2024-04-22T23:34:08.952 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:08.952 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.1 MB/s | 142 kB 00:00 2024-04-22T23:34:08.952 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:08.960 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:08.960 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:09.020 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:09.020 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:09.152 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:09.181 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:09.226 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:09.262 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:09.287 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:09.287 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:09.288 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T23:34:09.288 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:09.630 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-04-22T23:34:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:09.898 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-04-22T23:34:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:09.898 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:10.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-04-22T23:34:10.160 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:10.161 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:10.161 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:10.161 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:10.371 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr 2024-04-22T23:34:10.643 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:51 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:10.767 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:10.769 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 142 k 2024-04-22T23:34:10.769 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 395 k 2024-04-22T23:34:10.769 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:10.849 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.7 MB/s | 142 kB 00:00 2024-04-22T23:34:10.849 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:10.850 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.7 MB/s | 142 kB 00:00 2024-04-22T23:34:10.850 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:10.858 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:10.858 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:10.909 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:55 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:10.918 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:10.919 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:10.991 INFO:teuthology.orchestra.run.smithi044.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:11.034 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:11.035 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:11.035 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:11.077 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:11.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-dashboard 2024-04-22T23:34:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:11.196 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:11.196 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T23:34:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:11.634 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:56 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:11.759 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-04-22T23:34:11.760 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout:Install 7 Packages 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 2.1 M 2024-04-22T23:34:11.761 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 8.5 M 2024-04-22T23:34:11.762 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:12.044 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:12.044 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:12.223 INFO:teuthology.orchestra.run.smithi044.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 207 kB/s | 23 kB 00:00 2024-04-22T23:34:12.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-04-22T23:34:12.249 INFO:teuthology.orchestra.run.smithi044.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 107 kB/s | 15 kB 00:00 2024-04-22T23:34:12.299 INFO:teuthology.orchestra.run.smithi044.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 9.4 MB/s | 1.8 MB 00:00 2024-04-22T23:34:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 45 kB/s | 9.0 kB 00:00 2024-04-22T23:34:12.483 INFO:teuthology.orchestra.run.smithi044.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 175 kB/s | 41 kB 00:00 2024-04-22T23:34:12.508 INFO:teuthology.orchestra.run.smithi044.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 197 kB/s | 31 kB 00:00 2024-04-22T23:34:12.550 INFO:teuthology.orchestra.run.smithi044.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-04-22T23:34:12.554 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:12.554 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.6 MB/s | 2.1 MB 00:00 2024-04-22T23:34:12.554 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:12.563 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:12.563 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:12.619 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:12.620 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:12.773 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:53 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:12.853 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:12.897 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:12.933 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:12.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-04-22T23:34:12.984 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-22T23:34:13.019 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-22T23:34:13.066 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-22T23:34:13.097 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-22T23:34:13.126 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-04-22T23:34:13.374 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-04-22T23:34:13.403 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-22T23:34:13.498 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:54 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:13.623 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:34:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout:Install 7 Packages 2024-04-22T23:34:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:13.626 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.1 M 2024-04-22T23:34:13.626 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 8.5 M 2024-04-22T23:34:13.626 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:13.882 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-22T23:34:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-04-22T23:34:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-04-22T23:34:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-04-22T23:34:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-22T23:34:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-22T23:34:13.883 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-22T23:34:13.922 INFO:teuthology.orchestra.run.smithi031.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 173 kB/s | 23 kB 00:00 2024-04-22T23:34:13.947 INFO:teuthology.orchestra.run.smithi031.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 92 kB/s | 15 kB 00:00 2024-04-22T23:34:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 8.4 MB/s | 1.8 MB 00:00 2024-04-22T23:34:14.040 INFO:teuthology.orchestra.run.smithi031.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 77 kB/s | 9.0 kB 00:00 2024-04-22T23:34:14.073 INFO:teuthology.orchestra.run.smithi031.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 326 kB/s | 41 kB 00:00 2024-04-22T23:34:14.098 INFO:teuthology.orchestra.run.smithi031.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 307 kB/s | 31 kB 00:00 2024-04-22T23:34:14.141 INFO:teuthology.orchestra.run.smithi031.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-04-22T23:34:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:Total 4.0 MB/s | 2.1 MB 00:00 2024-04-22T23:34:14.143 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-22T23:34:14.146 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:14.146 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T23:34:14.147 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T23:34:14.148 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:14.148 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:14.151 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:14.205 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:14.206 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:14.321 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-22T23:34:14.503 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:14.570 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-22T23:34:14.602 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-22T23:34:14.649 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-22T23:34:14.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-22T23:34:14.716 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-04-22T23:34:14.849 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:59 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-04-22T23:34:14.974 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:14.975 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:14.975 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T23:34:14.975 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:14.975 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T23:34:14.976 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout:Install 12 Packages 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 38 M 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 209 M 2024-04-22T23:34:14.977 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-22T23:34:15.407 INFO:teuthology.orchestra.run.smithi044.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 41 MB/s | 7.4 MB 00:00 2024-04-22T23:34:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-04-22T23:34:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-04-22T23:34:15.483 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-04-22T23:34:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-04-22T23:34:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-22T23:34:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-22T23:34:15.484 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-22T23:34:15.549 INFO:teuthology.orchestra.run.smithi044.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-04-22T23:34:15.590 INFO:teuthology.orchestra.run.smithi044.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 535 kB/s | 195 kB 00:00 2024-04-22T23:34:15.641 INFO:teuthology.orchestra.run.smithi044.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-22T23:34:15.707 INFO:teuthology.orchestra.run.smithi044.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-04-22T23:34:15.733 INFO:teuthology.orchestra.run.smithi044.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 384 kB/s | 35 kB 00:00 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-04-22T23:34:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-22T23:34:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-22T23:34:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T23:34:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T23:34:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:15.742 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:15.791 INFO:teuthology.orchestra.run.smithi044.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-04-22T23:34:15.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-22T23:34:15.933 INFO:teuthology.orchestra.run.smithi044.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-22T23:34:15.959 INFO:teuthology.orchestra.run.smithi044.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-22T23:34:15.992 INFO:teuthology.orchestra.run.smithi044.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.2 MB/s | 250 kB 00:00 2024-04-22T23:34:16.449 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:57 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:16.575 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:34:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:16.577 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-04-22T23:34:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:16.578 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-04-22T23:34:16.578 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 209 M 2024-04-22T23:34:16.578 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:16.726 INFO:teuthology.orchestra.run.smithi044.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.7 MB/s | 4.7 MB 00:01 2024-04-22T23:34:17.159 INFO:teuthology.orchestra.run.smithi044.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-22T23:34:17.162 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:17.162 INFO:teuthology.orchestra.run.smithi044.stdout:Total 17 MB/s | 38 MB 00:02 2024-04-22T23:34:17.305 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 45 MB/s | 7.4 MB 00:00 2024-04-22T23:34:17.422 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-04-22T23:34:17.480 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): libquadmath-11.4.1-3.el9.x86_64.rpm 575 kB/s | 195 kB 00:00 2024-04-22T23:34:17.521 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:17.549 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:17.549 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:17.555 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-22T23:34:17.581 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 182 kB/s | 18 kB 00:00 2024-04-22T23:34:17.664 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-04-22T23:34:17.773 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-22T23:34:17.981 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:17.981 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:18.173 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.0 MB/s | 3.0 MB 00:00 2024-04-22T23:34:18.315 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-04-22T23:34:18.770 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:18.773 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.0 MB/s | 4.7 MB 00:01 2024-04-22T23:34:18.823 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-22T23:34:18.867 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:34:18.940 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-04-22T23:34:18.971 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:34:19.009 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-22T23:34:19.197 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-22T23:34:19.233 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-22T23:34:19.313 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-22T23:34:20.008 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-22T23:34:20.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-22T23:34:21.436 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:34:22.540 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-22T23:34:22.566 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-22T23:34:22.832 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.6 MB/s | 16 MB 00:04 2024-04-22T23:34:22.832 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:22.833 INFO:teuthology.orchestra.run.smithi031.stdout:Total 6.1 MB/s | 38 MB 00:06 2024-04-22T23:34:23.198 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:23.226 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:23.226 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:23.657 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:23.657 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-22T23:34:24.547 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:34:24.660 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:34:24.698 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-22T23:34:24.875 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-22T23:34:24.905 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-22T23:34:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-22T23:34:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-22T23:34:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-04-22T23:34:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:34:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:34:25.253 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:34:25.254 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-22T23:34:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T23:34:25.796 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:25.797 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-22T23:34:26.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-rook 2024-04-22T23:34:26.655 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:11 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:26.782 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T23:34:26.783 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:26.784 INFO:teuthology.orchestra.run.smithi044.stdout:Install 13 Packages 2024-04-22T23:34:26.785 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:26.785 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 2.2 M 2024-04-22T23:34:26.785 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 26 M 2024-04-22T23:34:26.785 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:34:27.140 INFO:teuthology.orchestra.run.smithi044.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 814 kB/s | 50 kB 00:00 2024-04-22T23:34:27.166 INFO:teuthology.orchestra.run.smithi044.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 295 kB/s | 26 kB 00:00 2024-04-22T23:34:27.191 INFO:teuthology.orchestra.run.smithi044.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 171 kB/s | 19 kB 00:00 2024-04-22T23:34:27.308 INFO:teuthology.orchestra.run.smithi044.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-22T23:34:27.350 INFO:teuthology.orchestra.run.smithi044.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.0 MB/s | 222 kB 00:00 2024-04-22T23:34:27.375 INFO:teuthology.orchestra.run.smithi044.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-04-22T23:34:27.401 INFO:teuthology.orchestra.run.smithi044.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 581 kB/s | 54 kB 00:00 2024-04-22T23:34:27.659 INFO:teuthology.orchestra.run.smithi044.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 53 kB/s | 15 kB 00:00 2024-04-22T23:34:27.684 INFO:teuthology.orchestra.run.smithi044.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 96 kB/s | 32 kB 00:00 2024-04-22T23:34:27.877 INFO:teuthology.orchestra.run.smithi044.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 426 kB/s | 203 kB 00:00 2024-04-22T23:34:27.901 INFO:teuthology.orchestra.run.smithi044.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 272 kB/s | 59 kB 00:00 2024-04-22T23:34:28.060 INFO:teuthology.orchestra.run.smithi044.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 489 kB/s | 90 kB 00:00 2024-04-22T23:34:28.194 INFO:teuthology.orchestra.run.smithi044.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-04-22T23:34:28.194 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:28.194 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-04-22T23:34:28.199 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-22T23:34:28.227 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-22T23:34:28.265 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:28.281 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:28.281 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:28.374 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:28.376 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:28.983 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-22T23:34:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-22T23:34:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-22T23:34:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-22T23:34:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-22T23:34:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-22T23:34:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-22T23:34:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-22T23:34:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-22T23:34:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-22T23:34:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-22T23:34:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-22T23:34:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-22T23:34:30.819 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:34:30.820 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-22T23:34:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-22T23:34:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T23:34:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T23:34:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T23:34:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T23:34:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:31.240 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-22T23:34:31.366 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-22T23:34:31.367 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-22T23:34:31.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T23:34:31.702 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:31.703 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:31.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-mgr-cephadm 2024-04-22T23:34:32.119 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:13 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:32.246 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T23:34:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-04-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:32.250 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-04-22T23:34:32.250 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-04-22T23:34:32.250 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:32.477 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:17 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:32.603 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T23:34:32.604 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout:Install 6 Packages 2024-04-22T23:34:32.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:32.606 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 7.0 M 2024-04-22T23:34:32.606 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 33 M 2024-04-22T23:34:32.606 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:32.960 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 815 kB/s | 50 kB 00:00 2024-04-22T23:34:33.043 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 132 kB/s | 19 kB 00:00 2024-04-22T23:34:33.069 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 150 kB/s | 26 kB 00:00 2024-04-22T23:34:33.138 INFO:teuthology.orchestra.run.smithi044.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-04-22T23:34:33.161 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-22T23:34:33.186 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 985 kB/s | 222 kB 00:00 2024-04-22T23:34:33.221 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-04-22T23:34:33.253 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-04-22T23:34:33.263 INFO:teuthology.orchestra.run.smithi044.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-22T23:34:33.412 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 143 kB/s | 32 kB 00:00 2024-04-22T23:34:33.462 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-04-22T23:34:33.464 INFO:teuthology.orchestra.run.smithi044.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-22T23:34:33.547 INFO:teuthology.orchestra.run.smithi044.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 204 kB/s | 58 kB 00:00 2024-04-22T23:34:33.629 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 354 kB/s | 59 kB 00:00 2024-04-22T23:34:33.639 INFO:teuthology.orchestra.run.smithi044.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-04-22T23:34:33.664 INFO:teuthology.orchestra.run.smithi044.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 255 kB/s | 51 kB 00:00 2024-04-22T23:34:33.666 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:33.666 INFO:teuthology.orchestra.run.smithi044.stdout:Total 6.6 MB/s | 7.0 MB 00:01 2024-04-22T23:34:33.704 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 450 kB/s | 203 kB 00:00 2024-04-22T23:34:33.739 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:33.752 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:33.752 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:33.796 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 538 kB/s | 90 kB 00:00 2024-04-22T23:34:33.838 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.4 MB/s | 1.0 MB 00:00 2024-04-22T23:34:33.838 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:33.839 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-04-22T23:34:33.855 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:33.855 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:33.904 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:33.919 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:34.010 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:34.011 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:34.166 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:34.332 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-22T23:34:34.392 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-22T23:34:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-22T23:34:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-22T23:34:34.889 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-22T23:34:34.933 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-22T23:34:34.956 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-22T23:34:34.983 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-22T23:34:35.047 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-22T23:34:35.092 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-22T23:34:35.116 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-22T23:34:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-22T23:34:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-22T23:34:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-22T23:34:35.860 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-22T23:34:35.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-04-22T23:34:35.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-22T23:34:35.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-22T23:34:35.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-22T23:34:35.860 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-22T23:34:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-22T23:34:36.092 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T23:34:36.109 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T23:34:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T23:34:36.110 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:36.110 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:36.125 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-22T23:34:36.174 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-22T23:34:36.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-22T23:34:36.330 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-fuse 2024-04-22T23:34:36.868 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:21 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-22T23:34:36.982 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-22T23:34:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-22T23:34:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-22T23:34:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-22T23:34:36.983 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-22T23:34:36.993 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:36.994 INFO:teuthology.orchestra.run.smithi044.stdout:Install 2 Packages 2024-04-22T23:34:36.995 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:36.995 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 920 k 2024-04-22T23:34:36.995 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 2.7 M 2024-04-22T23:34:36.995 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-22T23:34:37.297 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T23:34:37.298 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T23:34:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T23:34:37.299 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:37.299 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:37.393 INFO:teuthology.orchestra.run.smithi044.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-04-22T23:34:37.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-04-22T23:34:37.743 INFO:teuthology.orchestra.run.smithi044.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 188 kB/s | 80 kB 00:00 2024-04-22T23:34:37.743 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:37.744 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.2 MB/s | 920 kB 00:00 2024-04-22T23:34:37.756 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:37.780 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:37.780 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:37.861 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:37.862 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:38.048 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:38.055 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:19 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:38.178 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-22T23:34:38.179 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:38.179 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-22T23:34:38.179 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:38.179 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-04-22T23:34:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:38.181 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-04-22T23:34:38.181 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-04-22T23:34:38.181 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:38.251 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-22T23:34:38.475 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-04-22T23:34:38.575 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-22T23:34:38.634 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.4 MB/s | 548 kB 00:00 2024-04-22T23:34:38.668 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 627 kB/s | 58 kB 00:00 2024-04-22T23:34:38.693 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 876 kB/s | 51 kB 00:00 2024-04-22T23:34:38.768 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-22T23:34:38.768 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:38.769 INFO:teuthology.orchestra.run.smithi031.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-04-22T23:34:38.840 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:38.852 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:38.852 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:38.890 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-22T23:34:38.890 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-04-22T23:34:38.954 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:38.955 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:39.158 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-22T23:34:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-22T23:34:39.159 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:39.159 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:39.257 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:39.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados-devel 2024-04-22T23:34:39.427 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-22T23:34:39.480 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-22T23:34:39.942 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:24 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-22T23:34:40.066 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:40.067 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 127 k 2024-04-22T23:34:40.068 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 456 k 2024-04-22T23:34:40.068 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:40.123 INFO:teuthology.orchestra.run.smithi044.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.2 MB/s | 127 kB 00:00 2024-04-22T23:34:40.124 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:40.124 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.2 MB/s | 127 kB 00:00 2024-04-22T23:34:40.124 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:40.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-22T23:34:40.130 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:40.130 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:40.195 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-22T23:34:40.196 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:40.196 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:40.224 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-22T23:34:40.357 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:40.480 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:40.851 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-04-22T23:34:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-04-22T23:34:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-22T23:34:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-22T23:34:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-22T23:34:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-22T23:34:41.083 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:41.084 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:41.084 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:41.084 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:41.084 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:41.084 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:41.220 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:41.221 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:41.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs2 2024-04-22T23:34:41.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-04-22T23:34:41.858 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:26 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:41.941 INFO:teuthology.orchestra.run.smithi044.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:41.959 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:22 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:41.984 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:41.985 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:41.985 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:42.047 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install libcephfs-devel 2024-04-22T23:34:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-04-22T23:34:42.084 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:42.085 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 920 k 2024-04-22T23:34:42.085 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.7 M 2024-04-22T23:34:42.085 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:42.270 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-04-22T23:34:42.420 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 372 kB/s | 80 kB 00:00 2024-04-22T23:34:42.421 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:42.421 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.7 MB/s | 920 kB 00:00 2024-04-22T23:34:42.433 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:42.456 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:42.456 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:42.537 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:42.538 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:42.582 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:27 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:42.706 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:42.706 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 31 k 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 139 k 2024-04-22T23:34:42.707 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:42.727 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:42.770 INFO:teuthology.orchestra.run.smithi044.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 497 kB/s | 31 kB 00:00 2024-04-22T23:34:42.770 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:42.771 INFO:teuthology.orchestra.run.smithi044.stdout:Total 486 kB/s | 31 kB 00:00 2024-04-22T23:34:42.771 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:42.774 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:42.774 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:42.777 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:42.777 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-22T23:34:42.876 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:42.920 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-22T23:34:42.977 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:43.332 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:43.548 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:43.548 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:43.549 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:43.549 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-04-22T23:34:43.619 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-04-22T23:34:43.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librados2 2024-04-22T23:34:43.897 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-22T23:34:43.898 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:43.898 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:43.898 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-22T23:34:43.898 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:43.898 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:44.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-04-22T23:34:44.266 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:29 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:44.350 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:44.393 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:44.394 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:44.394 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:44.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install librbd1 2024-04-22T23:34:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:25 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:44.780 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-04-22T23:34:44.781 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:44.842 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.1 MB/s | 127 kB 00:00 2024-04-22T23:34:44.843 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:44.843 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.0 MB/s | 127 kB 00:00 2024-04-22T23:34:44.843 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:44.848 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:44.849 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:44.914 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:44.914 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:44.986 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:29 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:45.067 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:45.110 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:45.111 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:45.111 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:45.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rados 2024-04-22T23:34:45.526 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:45.721 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:30 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:45.770 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:45.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:45.771 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:45.771 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:45.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:45.771 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:45.803 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:45.845 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:45.847 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:45.847 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:45.916 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rgw 2024-04-22T23:34:45.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-04-22T23:34:46.456 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:31 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:46.515 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:27 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:46.538 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:46.580 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:46.582 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:46.582 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:46.597 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:46.640 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:46.641 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:46.642 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:46.650 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-cephfs 2024-04-22T23:34:46.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-04-22T23:34:47.178 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:32 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:47.259 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:47.264 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:28 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:47.302 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:47.303 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:47.303 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:47.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install python3-rbd 2024-04-22T23:34:47.390 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 31 k 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 139 k 2024-04-22T23:34:47.391 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:47.454 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 494 kB/s | 31 kB 00:00 2024-04-22T23:34:47.455 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:47.455 INFO:teuthology.orchestra.run.smithi031.stdout:Total 483 kB/s | 31 kB 00:00 2024-04-22T23:34:47.455 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:47.459 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:47.459 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:47.461 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:47.461 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:47.661 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:47.902 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:32 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:47.983 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:47.986 INFO:teuthology.orchestra.run.smithi044.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:48.029 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:48.030 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T23:34:48.030 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:48.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-fuse 2024-04-22T23:34:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:48.185 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:48.185 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:48.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-04-22T23:34:48.627 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:33 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:48.751 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:48.751 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 86 k 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 226 k 2024-04-22T23:34:48.752 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:48.814 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 86 kB 00:00 2024-04-22T23:34:48.815 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:48.815 INFO:teuthology.orchestra.run.smithi044.stdout:Total 1.3 MB/s | 86 kB 00:00 2024-04-22T23:34:48.815 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:48.823 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:48.823 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:48.889 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:48.889 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:48.891 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:29 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:48.974 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:49.018 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:49.019 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:49.019 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:49.050 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:49.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-04-22T23:34:49.147 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:49.508 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:30 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:49.694 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:49.737 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:49.749 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:49.749 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:49.749 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:49.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-04-22T23:34:49.974 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-mirror 2024-04-22T23:34:50.339 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:31 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:50.422 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:50.466 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:50.513 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:35 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:50.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-04-22T23:34:50.639 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:50.640 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:50.641 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:50.641 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:50.641 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:50.641 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 3.0 M 2024-04-22T23:34:50.641 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 12 M 2024-04-22T23:34:50.641 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:50.735 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 32 MB/s | 3.0 MB 00:00 2024-04-22T23:34:50.735 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:50.736 INFO:teuthology.orchestra.run.smithi044.stdout:Total 31 MB/s | 3.0 MB 00:00 2024-04-22T23:34:50.736 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:50.745 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:50.746 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:50.829 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:50.830 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:51.069 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:32 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:51.151 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:51.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-04-22T23:34:51.271 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:51.436 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:51.461 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:51.462 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:51.462 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T23:34:51.462 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:34:51.462 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:34:51.462 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:51.794 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:32 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:51.877 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:51.920 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:51.920 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:51.920 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:51.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-04-22T23:34:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:52.415 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:52.415 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:52.415 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:52.529 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:33 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:52.611 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-04-22T23:34:52.654 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:52.655 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-22T23:34:52.656 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:52.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install rbd-nbd 2024-04-22T23:34:52.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-04-22T23:34:53.214 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:01:38 ago on Mon 22 Apr 2024 11:33:15 PM UTC. 2024-04-22T23:34:53.257 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:34 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:53.338 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T23:34:53.338 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:Install 1 Package 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 171 k 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 494 k 2024-04-22T23:34:53.339 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T23:34:53.383 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:53.383 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:53.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:53.385 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 86 k 2024-04-22T23:34:53.385 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 226 k 2024-04-22T23:34:53.385 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:53.406 INFO:teuthology.orchestra.run.smithi044.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-04-22T23:34:53.407 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:53.407 INFO:teuthology.orchestra.run.smithi044.stdout:Total 2.5 MB/s | 171 kB 00:00 2024-04-22T23:34:53.407 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T23:34:53.415 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T23:34:53.415 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T23:34:53.457 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-04-22T23:34:53.457 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:53.457 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-04-22T23:34:53.458 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:53.465 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:53.465 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:53.481 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T23:34:53.481 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T23:34:53.530 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:53.531 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:53.634 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T23:34:53.677 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:53.746 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:53.772 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:54.122 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:54.133 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:54.346 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T23:34:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:54.346 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:34:54.346 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T23:34:54.379 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:54.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:54.379 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:54.379 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:54.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:54.379 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:54.580 DEBUG:teuthology.parallel:result is None 2024-04-22T23:34:54.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-04-22T23:34:55.165 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:36 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:55.289 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:55.290 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:55.290 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:55.290 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.0 M 2024-04-22T23:34:55.291 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 12 M 2024-04-22T23:34:55.292 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:55.393 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 29 MB/s | 3.0 MB 00:00 2024-04-22T23:34:55.393 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:55.394 INFO:teuthology.orchestra.run.smithi031.stdout:Total 29 MB/s | 3.0 MB 00:00 2024-04-22T23:34:55.394 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:55.403 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:55.404 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:55.487 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:55.487 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:55.940 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:56.103 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:56.130 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:56.130 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:34:56.131 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T23:34:56.131 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:34:56.131 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:34:56.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:57.119 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:57.120 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:57.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-04-22T23:34:57.930 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:38 ago on Mon 22 Apr 2024 11:33:19 PM UTC. 2024-04-22T23:34:58.057 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-22T23:34:58.058 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-22T23:34:58.058 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:58.058 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-22T23:34:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-04-22T23:34:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 171 k 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 494 k 2024-04-22T23:34:58.059 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-22T23:34:58.131 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.3 MB/s | 171 kB 00:00 2024-04-22T23:34:58.132 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:34:58.132 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.3 MB/s | 171 kB 00:00 2024-04-22T23:34:58.132 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-22T23:34:58.140 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-22T23:34:58.140 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-22T23:34:58.209 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-22T23:34:58.209 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-22T23:34:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-22T23:34:58.471 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-04-22T23:34:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:59.083 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-22T23:34:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-04-22T23:34:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:34:59.083 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-22T23:34:59.343 DEBUG:teuthology.parallel:result is None 2024-04-22T23:34:59.343 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-22T23:34:59.343 INFO:teuthology.packaging:ref: None 2024-04-22T23:34:59.343 INFO:teuthology.packaging:tag: v18.2.1 2024-04-22T23:34:59.343 INFO:teuthology.packaging:branch: None 2024-04-22T23:34:59.343 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:34:59.344 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-22T23:34:59.344 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-22T23:34:59.494 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T23:34:59.516 INFO:teuthology.orchestra.run.smithi031.stdout:18.2.1-0.el9 2024-04-22T23:34:59.517 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-04-22T23:34:59.517 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-22T23:34:59.519 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-22T23:34:59.519 INFO:teuthology.packaging:ref: None 2024-04-22T23:34:59.519 INFO:teuthology.packaging:tag: v18.2.1 2024-04-22T23:34:59.519 INFO:teuthology.packaging:branch: None 2024-04-22T23:34:59.519 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:34:59.519 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-04-22T23:34:59.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-22T23:34:59.659 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T23:34:59.684 INFO:teuthology.orchestra.run.smithi044.stdout:18.2.1-0.el9 2024-04-22T23:34:59.685 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-04-22T23:34:59.685 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-22T23:34:59.686 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T23:34:59.687 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:34:59.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T23:34:59.713 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:34:59.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T23:34:59.756 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T23:34:59.756 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:34:59.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T23:34:59.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T23:34:59.850 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:34:59.850 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T23:34:59.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T23:34:59.945 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T23:34:59.946 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:34:59.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T23:34:59.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T23:35:00.036 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:35:00.036 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T23:35:00.065 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T23:35:00.132 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T23:35:00.132 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:35:00.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T23:35:00.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T23:35:00.221 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:35:00.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T23:35:00.245 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T23:35:00.310 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:35:00.320 INFO:teuthology.task.print:**** done install task... 2024-04-22T23:35:00.320 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-22T23:35:00.423 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5'} 2024-04-22T23:35:00.424 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-04-22T23:35:00.424 INFO:tasks.cephadm:Cluster fsid is efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:00.424 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-22T23:35:00.424 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-22T23:35:00.424 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi031': '172.21.15.31', 'mon.smithi044': '172.21.15.44'} 2024-04-22T23:35:00.424 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-22T23:35:00.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-04-22T23:35:00.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-04-22T23:35:00.476 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-22T23:35:00.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:35:00.617 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78807/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3252-g43be0201', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3252.g43be0201'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-adk-testing-2024-04-22-1618/43be020184947e53516056c9931e1ac5bdbbb1a5/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 22:31:25.102991', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-adk-testing-2024-04-22-1618', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-adk-testing-2024-04-22-1618/43be020184947e53516056c9931e1ac5bdbbb1a5/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-22T23:35:00.827 INFO:tasks.util.chacra:got chacra host chacra.ceph.com, ref reef, sha1 9d98999acebbad2663cc4938e94c9ffd83485cbd from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-04-22T23:35:00.829 INFO:tasks.cephadm:Discovered cachra url: https://chacra.ceph.com/binaries/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/x86_64/flavors/default/cephadm 2024-04-22T23:35:00.829 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://chacra.ceph.com/binaries/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T23:35:01.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 212578 Apr 22 23:35 /home/ubuntu/cephtest/cephadm 2024-04-22T23:35:01.133 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://chacra.ceph.com/binaries/ceph/reef/9d98999acebbad2663cc4938e94c9ffd83485cbd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T23:35:01.430 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 212578 Apr 22 23:35 /home/ubuntu/cephtest/cephadm 2024-04-22T23:35:01.430 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T23:35:01.450 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T23:35:01.479 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-04-22T23:35:01.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-22T23:35:01.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-22T23:35:03.036 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-22T23:35:03.068 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-22T23:35:22.209 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T23:35:22.210 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-22T23:35:22.210 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-22T23:35:22.210 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-04-22T23:35:22.210 INFO:teuthology.orchestra.run.smithi044.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-22T23:35:22.210 INFO:teuthology.orchestra.run.smithi044.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-22T23:35:22.211 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-22T23:35:22.211 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-22T23:35:25.250 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:35:25.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-04-22T23:35:25.311 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-04-22T23:35:25.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-04-22T23:35:25.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-04-22T23:35:25.411 INFO:tasks.cephadm:Writing seed config... 2024-04-22T23:35:25.413 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-22T23:35:25.413 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-22T23:35:25.413 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-22T23:35:25.413 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-22T23:35:25.413 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-22T23:35:25.414 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-22T23:35:25.415 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-22T23:35:25.416 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-22T23:35:25.417 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-22T23:35:25.418 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-22T23:35:25.418 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-22T23:35:25.419 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:35:25.419 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-22T23:35:25.437 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = efa92c84-0100-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-22T23:35:25.437 DEBUG:teuthology.orchestra.run.smithi031:mon.smithi031> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service 2024-04-22T23:35:25.480 INFO:tasks.cephadm:Bootstrapping... 2024-04-22T23:35:25.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid efa92c84-0100-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:35:25.639 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:35:25.639 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', 'efa92c84-0100-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-04-22T23:35:25.665 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T23:35:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-22T23:35:25.665 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-04-22T23:35:25.689 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T23:35:25.689 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-04-22T23:35:25.689 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-04-22T23:35:25.697 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T23:35:25.697 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T23:35:25.703 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T23:35:25.703 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-22T23:35:25.709 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-22T23:35:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-22T23:35:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-22T23:35:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-04-22T23:35:25.740 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T23:35:25.740 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-22T23:35:25.740 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-04-22T23:35:25.740 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-04-22T23:35:25.746 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T23:35:25.746 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T23:35:25.752 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T23:35:25.752 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-22T23:35:25.758 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-22T23:35:25.764 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140296625470624 on /run/cephadm/efa92c84-0100-11ef-bc93-c7b262605968.lock 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140296625470624 acquired on /run/cephadm/efa92c84-0100-11ef-bc93-c7b262605968.lock 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-04-22T23:35:25.765 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-04-22T23:35:25.766 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-22T23:35:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-04-22T23:35:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-04-22T23:35:25.772 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-22T23:35:25.772 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-22T23:35:25.772 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-22T23:35:25.774 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-22T23:35:25.774 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-22T23:35:25.774 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T23:35:25.774 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-22T23:35:25.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-22T23:35:25.775 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T23:35:25.775 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-22T23:35:25.775 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-22T23:35:25.775 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-22T23:35:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-22T23:35:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-22T23:35:26.519 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-22T23:35:27.390 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-22T23:35:27.390 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-22T23:35:27.390 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-04-22T23:35:28.279 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-04-22T23:35:28.279 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-04-22T23:35:29.286 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBA9CZmDv99IxAA4QVxOdQ7f9nC+xW9JDPF5A== 2024-04-22T23:35:30.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBB9CZmdQaKJRAADN5Q58UPK7noBHyeKr8QhQ== 2024-04-22T23:35:31.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBC9CZmqR+cHxAAwre21Z/z0zb7QGNQ3p0CyQ== 2024-04-22T23:35:31.204 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for smithi031 [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2024-04-22T23:35:32.241 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T23:35:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:35:32.242 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-04-22T23:35:33.356 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.smithi031 on 2024-04-22T23:35:33.581 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-22T23:35:33.735 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-22T23:35:34.026 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-efa92c84-0100-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-efa92c84-0100-11ef-bc93-c7b262605968.target. 2024-04-22T23:35:34.026 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-efa92c84-0100-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-efa92c84-0100-11ef-bc93-c7b262605968.target. 2024-04-22T23:35:34.367 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031 2024-04-22T23:35:34.368 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service: Unit ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service not loaded. 2024-04-22T23:35:34.538 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-efa92c84-0100-11ef-bc93-c7b262605968.target.wants/ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service → /etc/systemd/system/ceph-efa92c84-0100-11ef-bc93-c7b262605968@.service. 2024-04-22T23:35:35.036 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:35:35.037 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-22T23:35:35.043 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:35:35.043 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-22T23:35:35.043 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-22T23:35:35.044 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T23:35:35.044 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-04-22T23:35:35.044 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-04-22T23:35:35.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:35 smithi031 systemd[1]: Started Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:35:36.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:36 smithi031 ceph-mon[26350]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-04-22T23:35:36.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:36 smithi031 ceph-mon[26350]: monmap e1: 1 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:35:36.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:36 smithi031 ceph-mon[26350]: fsmap 2024-04-22T23:35:36.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:36 smithi031 ceph-mon[26350]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T23:35:36.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:36 smithi031 ceph-mon[26350]: mgrmap e1: no daemons active 2024-04-22T23:35:36.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:36 smithi031 ceph-mon[26350]: from='client.? 172.21.15.31:0/358488917' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-22T23:35:36.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-04-22T23:35:36.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:36.256 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi031 (age 0.677693s) 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52dc250700 1 Processor -- start 2024-04-22T23:35:36.257 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52dc250700 1 -- start start 2024-04-22T23:35:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52dc250700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d4106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52dc250700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4106f50 con 0x7f52d4106560 2024-04-22T23:35:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52d9fec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d4106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52d9fec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d4106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42390/0 (socket says 172.21.15.31:42390) 2024-04-22T23:35:36.258 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.602+0000 7f52d9fec700 1 -- 172.21.15.31:0/2606270030 learned_addr learned my addr 172.21.15.31:0/2606270030 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.603+0000 7f52d9fec700 1 -- 172.21.15.31:0/2606270030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4107760 con 0x7f52d4106560 2024-04-22T23:35:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.603+0000 7f52d9fec700 1 --2- 172.21.15.31:0/2606270030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d4106980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009e80 tx=0x7f52cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26539ba7dd0774d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.603+0000 7f52d8fea700 1 -- 172.21.15.31:0/2606270030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52cc00e070 con 0x7f52d4106560 2024-04-22T23:35:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.603+0000 7f52d8fea700 1 -- 172.21.15.31:0/2606270030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc003a20 con 0x7f52d4106560 2024-04-22T23:35:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.603+0000 7f52d8fea700 1 -- 172.21.15.31:0/2606270030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52cc003bd0 con 0x7f52d4106560 2024-04-22T23:35:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 -- 172.21.15.31:0/2606270030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 msgr2=0x7f52d4106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 --2- 172.21.15.31:0/2606270030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d4106980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009e80 tx=0x7f52cc00b330 comp rx=0 tx=0).stop 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 -- 172.21.15.31:0/2606270030 shutdown_connections 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 --2- 172.21.15.31:0/2606270030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d4106980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 -- 172.21.15.31:0/2606270030 >> 172.21.15.31:0/2606270030 conn(0x7f52d4101b40 msgr2=0x7f52d4103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 -- 172.21.15.31:0/2606270030 shutdown_connections 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 -- 172.21.15.31:0/2606270030 wait complete. 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 Processor -- start 2024-04-22T23:35:36.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.604+0000 7f52dc250700 1 -- start start 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52dc250700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d407bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52dc250700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d407c170 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52d9fec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d407bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52d9fec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d407bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42398/0 (socket says 172.21.15.31:42398) 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52d9fec700 1 -- 172.21.15.31:0/358488917 learned_addr learned my addr 172.21.15.31:0/358488917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52d9fec700 1 -- 172.21.15.31:0/358488917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc009b30 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52d9fec700 1 --2- 172.21.15.31:0/358488917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d407bc30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004000 tx=0x7f52cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52cc004240 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc0043e0 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d407a280 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52cc0046e0 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.605+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d407a720 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.606+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f52cc019050 con 0x7f52d4106560 2024-04-22T23:35:36.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.606+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f52cc025790 con 0x7f52d4106560 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.606+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d407b080 con 0x7f52d4106560 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.609+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f52cc02a060 con 0x7f52d4106560 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.681+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f52d4192f90 con 0x7f52d4106560 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.681+0000 7f52caffd700 1 -- 172.21.15.31:0/358488917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f52cc020030 con 0x7f52d4106560 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 msgr2=0x7f52d407bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 --2- 172.21.15.31:0/358488917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d407bc30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004000 tx=0x7f52cc004ad0 comp rx=0 tx=0).stop 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 shutdown_connections 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 --2- 172.21.15.31:0/358488917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52d4106560 0x7f52d407bc30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 >> 172.21.15.31:0/358488917 conn(0x7f52d4101b40 msgr2=0x7f52d418df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 shutdown_connections 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:35.682+0000 7f52dc250700 1 -- 172.21.15.31:0/358488917 wait complete. 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-04-22T23:35:36.264 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-04-22T23:35:37.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:37 smithi031 ceph-mon[26350]: from='client.? 172.21.15.31:0/1424530122' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T23:35:37.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:37 smithi031 ceph-mon[26350]: from='client.? 172.21.15.31:0/1424530122' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-22T23:35:37.422 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:37.422 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T23:35:37.422 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:37.422 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T23:35:37.423 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.692+0000 7f3ff83ee700 1 Processor -- start 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.692+0000 7f3ff83ee700 1 -- start start 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.692+0000 7f3ff83ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff0105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.692+0000 7f3ff83ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0074620 con 0x7f3ff0104e30 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff618a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff0105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff618a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff0105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42400/0 (socket says 172.21.15.31:42400) 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff618a700 1 -- 172.21.15.31:0/2553975432 learned_addr learned my addr 172.21.15.31:0/2553975432 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff618a700 1 -- 172.21.15.31:0/2553975432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0105790 con 0x7f3ff0104e30 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff618a700 1 --2- 172.21.15.31:0/2553975432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff0105250 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009e80 tx=0x7f3fe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b8ff99bc4816aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff5188700 1 -- 172.21.15.31:0/2553975432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe800e070 con 0x7f3ff0104e30 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff5188700 1 -- 172.21.15.31:0/2553975432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8003a20 con 0x7f3ff0104e30 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.693+0000 7f3ff5188700 1 -- 172.21.15.31:0/2553975432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe8003bd0 con 0x7f3ff0104e30 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 -- 172.21.15.31:0/2553975432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 msgr2=0x7f3ff0105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 --2- 172.21.15.31:0/2553975432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff0105250 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009e80 tx=0x7f3fe800b330 comp rx=0 tx=0).stop 2024-04-22T23:35:37.424 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 -- 172.21.15.31:0/2553975432 shutdown_connections 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 --2- 172.21.15.31:0/2553975432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff0105250 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 -- 172.21.15.31:0/2553975432 >> 172.21.15.31:0/2553975432 conn(0x7f3ff01009f0 msgr2=0x7f3ff0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 -- 172.21.15.31:0/2553975432 shutdown_connections 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 -- 172.21.15.31:0/2553975432 wait complete. 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.694+0000 7f3ff83ee700 1 Processor -- start 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff83ee700 1 -- start start 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff83ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff83ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff019c0f0 con 0x7f3ff0104e30 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff618a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff618a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42404/0 (socket says 172.21.15.31:42404) 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff618a700 1 -- 172.21.15.31:0/1424530122 learned_addr learned my addr 172.21.15.31:0/1424530122 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff618a700 1 -- 172.21.15.31:0/1424530122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009b30 con 0x7f3ff0104e30 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3ff618a700 1 --2- 172.21.15.31:0/1424530122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff019bbb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0073a60 tx=0x7f3fe8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe8004240 con 0x7f3ff0104e30 2024-04-22T23:35:37.425 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe80043e0 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.695+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fe80046e0 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.696+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff019c2f0 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.696+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff019c790 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.696+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3fe8019050 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.696+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3fe80257e0 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.697+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff01958c0 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.699+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f3fe802a060 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.768+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3ff002cee0 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.772+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3fe8012d30 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.772+0000 7f3fef7fe700 1 -- 172.21.15.31:0/1424530122 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f3fe8020030 con 0x7f3ff0104e30 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 msgr2=0x7f3ff019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 --2- 172.21.15.31:0/1424530122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff019bbb0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0073a60 tx=0x7f3fe8004ad0 comp rx=0 tx=0).stop 2024-04-22T23:35:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 shutdown_connections 2024-04-22T23:35:37.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 --2- 172.21.15.31:0/1424530122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ff0104e30 0x7f3ff019bbb0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:37.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 >> 172.21.15.31:0/1424530122 conn(0x7f3ff01009f0 msgr2=0x7f3ff0192a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:37.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 shutdown_connections 2024-04-22T23:35:37.427 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:36.773+0000 7f3ff83ee700 1 -- 172.21.15.31:0/1424530122 wait complete. 2024-04-22T23:35:37.427 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-04-22T23:35:38.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:38 smithi031 ceph-mon[26350]: from='client.? 172.21.15.31:0/3464149970' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:35:38.411 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.869+0000 7f771f942700 1 Processor -- start 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.869+0000 7f771f942700 1 -- start start 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771f942700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f77181093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771f942700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718074680 con 0x7f7718106fe0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771d6de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f77181093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771d6de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f77181093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42410/0 (socket says 172.21.15.31:42410) 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771d6de700 1 -- 172.21.15.31:0/15077299 learned_addr learned my addr 172.21.15.31:0/15077299 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771d6de700 1 -- 172.21.15.31:0/15077299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718109910 con 0x7f7718106fe0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.870+0000 7f771d6de700 1 --2- 172.21.15.31:0/15077299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f77181093d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77080098a0 tx=0x7f7708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b40dc5d556706 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f770ffff700 1 -- 172.21.15.31:0/15077299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7708004180 con 0x7f7718106fe0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f770ffff700 1 -- 172.21.15.31:0/15077299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77080042e0 con 0x7f7718106fe0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f770ffff700 1 -- 172.21.15.31:0/15077299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7708011a50 con 0x7f7718106fe0 2024-04-22T23:35:38.412 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 -- 172.21.15.31:0/15077299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 msgr2=0x7f77181093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 --2- 172.21.15.31:0/15077299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f77181093d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f77080098a0 tx=0x7f7708009bb0 comp rx=0 tx=0).stop 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 -- 172.21.15.31:0/15077299 shutdown_connections 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 --2- 172.21.15.31:0/15077299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f77181093d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 -- 172.21.15.31:0/15077299 >> 172.21.15.31:0/15077299 conn(0x7f77181009f0 msgr2=0x7f7718102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 -- 172.21.15.31:0/15077299 shutdown_connections 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.871+0000 7f771f942700 1 -- 172.21.15.31:0/15077299 wait complete. 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771f942700 1 Processor -- start 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771f942700 1 -- start start 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771f942700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f7718106940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771f942700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718106e80 con 0x7f7718106fe0 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771d6de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f7718106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771d6de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f7718106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42424/0 (socket says 172.21.15.31:42424) 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771d6de700 1 -- 172.21.15.31:0/3464149970 learned_addr learned my addr 172.21.15.31:0/3464149970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:38.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.872+0000 7f771d6de700 1 -- 172.21.15.31:0/3464149970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708009550 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f771d6de700 1 --2- 172.21.15.31:0/3464149970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f7718106940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7708000c00 tx=0x7f7708011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7708003b90 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77080248b0 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7718104f90 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f770801a8e0 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7718105430 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f770802d3f0 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.873+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f77080194e0 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.874+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771810a690 con 0x7f7718106fe0 2024-04-22T23:35:38.414 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.876+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f7708024db0 con 0x7f7718106fe0 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.946+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f771810a8b0 con 0x7f7718106fe0 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.946+0000 7f770e7fc700 1 -- 172.21.15.31:0/3464149970 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f7708024ad0 con 0x7f7718106fe0 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 msgr2=0x7f7718106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 --2- 172.21.15.31:0/3464149970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f7718106940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7708000c00 tx=0x7f7708011ab0 comp rx=0 tx=0).stop 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 shutdown_connections 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 --2- 172.21.15.31:0/3464149970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7718106fe0 0x7f7718106940 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 >> 172.21.15.31:0/3464149970 conn(0x7f77181009f0 msgr2=0x7f7718101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 shutdown_connections 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:37.947+0000 7f771f942700 1 -- 172.21.15.31:0/3464149970 wait complete. 2024-04-22T23:35:38.415 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-04-22T23:35:38.657 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:38 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:35:38.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:38 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[26346]: 2024-04-22T23:35:38.671+0000 7fd09c157700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:35:38.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:38 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[26346]: 2024-04-22T23:35:38.671+0000 7fd09c157700 -1 mon.smithi031@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T23:35:38.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:38 smithi031 podman[26597]: 2024-04-22 23:35:38.702242843 +0000 UTC m=+0.123440194 container died 9b9bf1c8e3d36030474572ef0a80af882d9d7613a3be1c1331f178d300a14463 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T23:35:39.207 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 podman[26597]: 2024-04-22 23:35:39.012281289 +0000 UTC m=+0.433478639 container cleanup 9b9bf1c8e3d36030474572ef0a80af882d9d7613a3be1c1331f178d300a14463 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-22T23:35:39.207 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 bash[26597]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031 2024-04-22T23:35:39.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 podman[26611]: 2024-04-22 23:35:39.213401686 +0000 UTC m=+0.515365049 container remove 9b9bf1c8e3d36030474572ef0a80af882d9d7613a3be1c1331f178d300a14463 (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-22T23:35:39.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-04-22T23:35:39.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:35:39.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 systemd[1]: Starting Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:35:39.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 podman[26702]: 2024-04-22 23:35:39.453228346 +0000 UTC m=+0.018463535 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-04-22T23:35:39.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 podman[26702]: 2024-04-22 23:35:39.55580853 +0000 UTC m=+0.121043741 container create acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-04-22T23:35:39.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 podman[26702]: 2024-04-22 23:35:39.774983065 +0000 UTC m=+0.340218241 container init acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD) 2024-04-22T23:35:39.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 podman[26702]: 2024-04-22 23:35:39.778318282 +0000 UTC m=+0.343553452 container start acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=HEAD) 2024-04-22T23:35:39.840 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: pidfile_write: ignore empty --pid-file 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: load: jerasure load: lrc 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: RocksDB version: 7.9.2 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Git sha 0 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Compile date 2023-12-11 22:07:34 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: DB SUMMARY 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: DB Session ID: F3E0QHKC04R0MGZ6N7TP 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:35:40.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi031/store.db dir, Total Num: 1, files: 000008.sst 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi031/store.db: 000009.log size: 98357 ; 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.env: 0x55ee44d81720 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.info_log: 0x55ee47f1d2e0 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.statistics: (nil) 2024-04-22T23:35:40.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.use_fsync: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.db_log_dir: 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.wal_dir: 2024-04-22T23:35:40.063 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.write_buffer_manager: 0x55ee4718e140 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:35:40.064 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.unordered_write: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.row_cache: None 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.wal_filter: None 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.wal_compression: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:35:40.065 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:35:40.066 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_open_files: -1 2024-04-22T23:35:40.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Compression algorithms supported: 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kZSTD supported: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:35:40.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000010 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.merge_operator: 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_filter: None 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ee47f1d3e0) 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:35:40.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_type: 0 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_index_type: 0 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_shortening: 1 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: checksum: 4 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: no_block_cache: 0 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache: 0x55ee4722f350 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_options: 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: capacity : 536870912 2024-04-22T23:35:40.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_shard_bits : 4 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: strict_capacity_limit : 0 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_compressed: (nil) 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: persistent_cache: (nil) 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size: 4096 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size_deviation: 10 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_restart_interval: 16 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_block_restart_interval: 1 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_block_size: 4096 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: partition_filters: 0 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: use_delta_encoding: 1 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: filter_policy: bloomfilter 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: whole_key_filtering: 1 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: verify_compression: 0 2024-04-22T23:35:40.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: format_version: 5 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_index_compression: 1 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_align: 0 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_auto_readahead_size: 262144 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: prepopulate_block_cache: 0 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: initial_auto_readahead_size: 8192 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression: NoCompression 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.num_levels: 7 2024-04-22T23:35:40.073 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:35:40.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:35:40.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:35:40.076 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:35:40.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:35:40.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:35:40.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:35:40.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:35:40.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:35:40.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:35:40.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:35:40.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:35:40.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.table_properties_collectors: 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:35:40.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.ttl: 2592000 2024-04-22T23:35:40.082 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:35:40.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-22T23:35:40.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-22T23:35:40.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6880fd68-273d-40a1-b7b7-c9596e00fc4f 2024-04-22T23:35:40.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713828939807801, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-22T23:35:40.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-22T23:35:40.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713828939809501, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93601, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91623, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84232, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713828939, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6880fd68-273d-40a1-b7b7-c9596e00fc4f", "db_session_id": "F3E0QHKC04R0MGZ6N7TP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-22T23:35:40.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713828939809614, "job": 1, "event": "recovery_finished"} 2024-04-22T23:35:40.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi031/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ee472cc000 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: DB pointer 0x55ee472b8000 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** DB Stats ** 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:35:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 2/0 93.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 84.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 93.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 84.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 84.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:35:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 84.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:35:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 19.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 19.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache BinnedLRUCache@0x55ee4722f350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: starting mon.smithi031 rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi031 fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???) e1 preinit fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).mds e1 new map 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).mds e1 print_map 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: e1 2024-04-22T23:35:40.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: legacy client fscid: -1 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: No filesystems configured 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 bash[26702]: acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 systemd[1]: Started Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-04-22T23:35:40.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: monmap e1: 1 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:35:40.091 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: fsmap 2024-04-22T23:35:40.091 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T23:35:40.091 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:39 smithi031 ceph-mon[26721]: mgrmap e1: no daemons active 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.266+0000 7fc30d9de700 1 Processor -- start 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.266+0000 7fc30d9de700 1 -- start start 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.266+0000 7fc30d9de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc308109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.266+0000 7fc30d9de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080745e0 con 0x7fc308107080 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc306ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc308109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc306ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc308109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42428/0 (socket says 172.21.15.31:42428) 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc306ffd700 1 -- 172.21.15.31:0/1159507906 learned_addr learned my addr 172.21.15.31:0/1159507906 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc306ffd700 1 -- 172.21.15.31:0/1159507906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308074720 con 0x7fc308107080 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc306ffd700 1 --2- 172.21.15.31:0/1159507906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc308109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc300009e80 tx=0x7fc30000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89facca9b93c3e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc305ffb700 1 -- 172.21.15.31:0/1159507906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc30000e070 con 0x7fc308107080 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc305ffb700 1 -- 172.21.15.31:0/1159507906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc300003e90 con 0x7fc308107080 2024-04-22T23:35:40.815 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.267+0000 7fc305ffb700 1 -- 172.21.15.31:0/1159507906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc300004380 con 0x7fc308107080 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 -- 172.21.15.31:0/1159507906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 msgr2=0x7fc308109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 --2- 172.21.15.31:0/1159507906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc308109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc300009e80 tx=0x7fc30000b330 comp rx=0 tx=0).stop 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 -- 172.21.15.31:0/1159507906 shutdown_connections 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 --2- 172.21.15.31:0/1159507906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc308109470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 -- 172.21.15.31:0/1159507906 >> 172.21.15.31:0/1159507906 conn(0x7fc3081009f0 msgr2=0x7fc308102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 -- 172.21.15.31:0/1159507906 shutdown_connections 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 -- 172.21.15.31:0/1159507906 wait complete. 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.268+0000 7fc30d9de700 1 Processor -- start 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc30d9de700 1 -- start start 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc30d9de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc30819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc30d9de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30819a640 con 0x7fc308107080 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc306ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc30819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc306ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc30819a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42436/0 (socket says 172.21.15.31:42436) 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc306ffd700 1 -- 172.21.15.31:0/2936010381 learned_addr learned my addr 172.21.15.31:0/2936010381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:40.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc306ffd700 1 -- 172.21.15.31:0/2936010381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300009b30 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.269+0000 7fc306ffd700 1 --2- 172.21.15.31:0/2936010381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc30819a100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc300000f80 tx=0x7fc300004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc30000e040 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30819a840 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc300003cf0 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc30001b930 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30819ad40 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc300019040 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.270+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc30001a6d0 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.271+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30810a690 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.273+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc300029060 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.342+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc30810a850 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.344+0000 7fc30c9dc700 1 -- 172.21.15.31:0/2936010381 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc30001f070 con 0x7fc308107080 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 msgr2=0x7fc30819a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 --2- 172.21.15.31:0/2936010381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc30819a100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc300000f80 tx=0x7fc300004920 comp rx=0 tx=0).stop 2024-04-22T23:35:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 shutdown_connections 2024-04-22T23:35:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 --2- 172.21.15.31:0/2936010381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc308107080 0x7fc30819a100 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 >> 172.21.15.31:0/2936010381 conn(0x7fc3081009f0 msgr2=0x7fc308101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 shutdown_connections 2024-04-22T23:35:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:40.345+0000 7fc30d9de700 1 -- 172.21.15.31:0/2936010381 wait complete. 2024-04-22T23:35:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-22T23:35:40.832 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:35:40.832 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-04-22T23:35:40.833 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-22T23:35:40.833 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-22T23:35:40.833 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-22T23:35:41.098 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-efa92c84-0100-11ef-bc93-c7b262605968@mgr.smithi031.fazdcv 2024-04-22T23:35:41.098 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-efa92c84-0100-11ef-bc93-c7b262605968@mgr.smithi031.fazdcv.service: Unit ceph-efa92c84-0100-11ef-bc93-c7b262605968@mgr.smithi031.fazdcv.service not loaded. 2024-04-22T23:35:41.250 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-efa92c84-0100-11ef-bc93-c7b262605968.target.wants/ceph-efa92c84-0100-11ef-bc93-c7b262605968@mgr.smithi031.fazdcv.service → /etc/systemd/system/ceph-efa92c84-0100-11ef-bc93-c7b262605968@.service. 2024-04-22T23:35:41.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:41 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2936010381' entity='client.admin' 2024-04-22T23:35:41.752 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:35:41.752 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-22T23:35:41.759 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:35:41.759 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-22T23:35:41.760 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-22T23:35:41.760 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T23:35:41.769 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:35:41.769 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-22T23:35:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:35:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-22T23:35:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-22T23:35:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-22T23:35:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-04-22T23:35:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-04-22T23:35:42.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:42 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4224058546' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T23:35:42.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:42.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:35:42.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "efa92c84-0100-11ef-bc93-c7b262605968", 2024-04-22T23:35:42.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T23:35:42.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-22T23:35:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T23:35:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T23:35:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T23:35:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T23:35:42.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T23:35:42.896 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T23:35:42.897 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.898 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T23:35:35.006069+0000", 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:35:42.899 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.223+0000 7f6aca9a5700 1 Processor -- start 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.223+0000 7f6aca9a5700 1 -- start start 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.223+0000 7f6aca9a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac40791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.223+0000 7f6aca9a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac40797a0 con 0x7f6ac407ad70 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac40791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac40791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59160/0 (socket says 172.21.15.31:59160) 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac3fff700 1 -- 172.21.15.31:0/3881004778 learned_addr learned my addr 172.21.15.31:0/3881004778 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac3fff700 1 -- 172.21.15.31:0/3881004778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40798e0 con 0x7f6ac407ad70 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac3fff700 1 --2- 172.21.15.31:0/3881004778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac40791d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80098a0 tx=0x7f6ab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869bbd051ac08810 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac2ffd700 1 -- 172.21.15.31:0/3881004778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ab8004180 con 0x7f6ac407ad70 2024-04-22T23:35:42.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac2ffd700 1 -- 172.21.15.31:0/3881004778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab80042e0 con 0x7f6ac407ad70 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.224+0000 7f6ac2ffd700 1 -- 172.21.15.31:0/3881004778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ab8011a50 con 0x7f6ac407ad70 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 -- 172.21.15.31:0/3881004778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 msgr2=0x7f6ac40791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 --2- 172.21.15.31:0/3881004778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac40791d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80098a0 tx=0x7f6ab8009bb0 comp rx=0 tx=0).stop 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 -- 172.21.15.31:0/3881004778 shutdown_connections 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 --2- 172.21.15.31:0/3881004778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac40791d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 -- 172.21.15.31:0/3881004778 >> 172.21.15.31:0/3881004778 conn(0x7f6ac4101b20 msgr2=0x7f6ac4103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 -- 172.21.15.31:0/3881004778 shutdown_connections 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.225+0000 7f6aca9a5700 1 -- 172.21.15.31:0/3881004778 wait complete. 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6aca9a5700 1 Processor -- start 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6aca9a5700 1 -- start start 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6aca9a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6aca9a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac419c920 con 0x7f6ac407ad70 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:42.901 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59168/0 (socket says 172.21.15.31:59168) 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6ac3fff700 1 -- 172.21.15.31:0/4224058546 learned_addr learned my addr 172.21.15.31:0/4224058546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.226+0000 7f6ac3fff700 1 -- 172.21.15.31:0/4224058546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8009550 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6ac3fff700 1 --2- 172.21.15.31:0/4224058546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac419c3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006af0 tx=0x7f6ab8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ab8003b90 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6ab801a8a0 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac419cb20 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ab8019880 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac419cfc0 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6ab80243f0 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.227+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6ab800b950 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.228+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac410c8c0 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.230+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f6ab801aa10 con 0x7f6ac407ad70 2024-04-22T23:35:42.902 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.303+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6ac410cab0 con 0x7f6ac407ad70 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.304+0000 7f6ac17fa700 1 -- 172.21.15.31:0/4224058546 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f6ab801abb0 con 0x7f6ac407ad70 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 msgr2=0x7f6ac419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 --2- 172.21.15.31:0/4224058546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac419c3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8006af0 tx=0x7f6ab8011a90 comp rx=0 tx=0).stop 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 shutdown_connections 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 --2- 172.21.15.31:0/4224058546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ac407ad70 0x7f6ac419c3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 >> 172.21.15.31:0/4224058546 conn(0x7f6ac4101b20 msgr2=0x7f6ac4102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 shutdown_connections 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:42.305+0000 7f6aca9a5700 1 -- 172.21.15.31:0/4224058546 wait complete. 2024-04-22T23:35:42.903 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-04-22T23:35:48.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:48 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2776318325' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T23:35:49.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:35:49.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: mgrmap e2: smithi031.fazdcv(active, starting, since 0.00295721s) 2024-04-22T23:35:49.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:35:49.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:35:49.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:35:49.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' 2024-04-22T23:35:49.448 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:49 smithi031 ceph-mon[26721]: from='mgr.14100 172.21.15.31:0/2495138145' entity='mgr.smithi031.fazdcv' 2024-04-22T23:35:50.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:50 smithi031 ceph-mon[26721]: mgrmap e3: smithi031.fazdcv(active, since 1.00897s) 2024-04-22T23:35:52.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:51 smithi031 ceph-mon[26721]: mgrmap e4: smithi031.fazdcv(active, since 2s) 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "efa92c84-0100-11ef-bc93-c7b262605968", 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T23:35:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T23:35:53.367 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:53.368 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T23:35:53.368 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T23:35:53.368 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T23:35:53.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T23:35:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T23:35:35.006069+0000", 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.956+0000 7f93033bd700 1 Processor -- start 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f93033bd700 1 -- start start 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f93033bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc105890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f93033bd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc105e60 con 0x7f92fc105470 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f9301159700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc105890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f9301159700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc105890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59178/0 (socket says 172.21.15.31:59178) 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f9301159700 1 -- 172.21.15.31:0/946746752 learned_addr learned my addr 172.21.15.31:0/946746752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f9301159700 1 -- 172.21.15.31:0/946746752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc106670 con 0x7f92fc105470 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.957+0000 7f9301159700 1 --2- 172.21.15.31:0/946746752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc105890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92f800bb40 tx=0x7f92f800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322ce3e37999cca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f92effff700 1 -- 172.21.15.31:0/946746752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92f800dc00 con 0x7f92fc105470 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f92effff700 1 -- 172.21.15.31:0/946746752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92f80044a0 con 0x7f92fc105470 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f92effff700 1 -- 172.21.15.31:0/946746752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92f8010470 con 0x7f92fc105470 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 -- 172.21.15.31:0/946746752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 msgr2=0x7f92fc105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 --2- 172.21.15.31:0/946746752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc105890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92f800bb40 tx=0x7f92f800be50 comp rx=0 tx=0).stop 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 -- 172.21.15.31:0/946746752 shutdown_connections 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 --2- 172.21.15.31:0/946746752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc105890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 -- 172.21.15.31:0/946746752 >> 172.21.15.31:0/946746752 conn(0x7f92fc1009f0 msgr2=0x7f92fc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 -- 172.21.15.31:0/946746752 shutdown_connections 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.958+0000 7f93033bd700 1 -- 172.21.15.31:0/946746752 wait complete. 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f93033bd700 1 Processor -- start 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f93033bd700 1 -- start start 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f93033bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f93033bd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc19c8f0 con 0x7f92fc105470 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f9301159700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f9301159700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59184/0 (socket says 172.21.15.31:59184) 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.959+0000 7f9301159700 1 -- 172.21.15.31:0/2776318325 learned_addr learned my addr 172.21.15.31:0/2776318325 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f9301159700 1 -- 172.21.15.31:0/2776318325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f800b7f0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f9301159700 1 --2- 172.21.15.31:0/2776318325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc19c3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b340 tx=0x7f92f8003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92f80104e0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f92f8010ba0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc19caf0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92f8021b50 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.960+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc19cf30 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.961+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f92f802a3f0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.961+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92fc191d00 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.961+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f92f80193f0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:47.963+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f92f802f060 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.037+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f92fc0623c0 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.037+0000 7f92ee7fc700 1 -- 172.21.15.31:0/2776318325 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f92f8021410 con 0x7f92fc105470 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 msgr2=0x7f92fc19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 --2- 172.21.15.31:0/2776318325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc19c3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b340 tx=0x7f92f8003fa0 comp rx=0 tx=0).stop 2024-04-22T23:35:53.381 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 shutdown_connections 2024-04-22T23:35:53.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 --2- 172.21.15.31:0/2776318325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92fc105470 0x7f92fc19c3b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:53.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 >> 172.21.15.31:0/2776318325 conn(0x7f92fc1009f0 msgr2=0x7f92fc18ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:53.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 shutdown_connections 2024-04-22T23:35:53.382 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:48.038+0000 7f93033bd700 1 -- 172.21.15.31:0/2776318325 wait complete. 2024-04-22T23:35:53.382 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-04-22T23:35:56.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:56 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1572884680' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T23:35:56.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "efa92c84-0100-11ef-bc93-c7b262605968", 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-22T23:35:56.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T23:35:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T23:35:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T23:35:56.649 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T23:35:35.006069+0000", 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fea741700 1 Processor -- start 2024-04-22T23:35:56.650 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fea741700 1 -- start start 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fea741700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe4106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fea741700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4106f50 con 0x7f4fe4106560 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe4106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe4106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33558/0 (socket says 172.21.15.31:33558) 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fe3fff700 1 -- 172.21.15.31:0/3903512275 learned_addr learned my addr 172.21.15.31:0/3903512275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.781+0000 7f4fe3fff700 1 -- 172.21.15.31:0/3903512275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4107760 con 0x7f4fe4106560 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fe3fff700 1 --2- 172.21.15.31:0/3903512275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe4106980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40098a0 tx=0x7f4fd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15163d3779c16fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fe2ffd700 1 -- 172.21.15.31:0/3903512275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fd4004180 con 0x7f4fe4106560 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fe2ffd700 1 -- 172.21.15.31:0/3903512275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4fd40042e0 con 0x7f4fe4106560 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fe2ffd700 1 -- 172.21.15.31:0/3903512275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fd4011a50 con 0x7f4fe4106560 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 -- 172.21.15.31:0/3903512275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 msgr2=0x7f4fe4106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:56.651 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 --2- 172.21.15.31:0/3903512275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe4106980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40098a0 tx=0x7f4fd4009bb0 comp rx=0 tx=0).stop 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 -- 172.21.15.31:0/3903512275 shutdown_connections 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 --2- 172.21.15.31:0/3903512275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe4106980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 -- 172.21.15.31:0/3903512275 >> 172.21.15.31:0/3903512275 conn(0x7f4fe4101b40 msgr2=0x7f4fe4103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 -- 172.21.15.31:0/3903512275 shutdown_connections 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.782+0000 7f4fea741700 1 -- 172.21.15.31:0/3903512275 wait complete. 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.783+0000 7f4fea741700 1 Processor -- start 2024-04-22T23:35:56.652 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.783+0000 7f4fea741700 1 -- start start 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.783+0000 7f4fea741700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.783+0000 7f4fea741700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe419c930 con 0x7f4fe4106560 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.783+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33562/0 (socket says 172.21.15.31:33562) 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe3fff700 1 -- 172.21.15.31:0/1572884680 learned_addr learned my addr 172.21.15.31:0/1572884680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe3fff700 1 -- 172.21.15.31:0/1572884680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4009550 con 0x7f4fe4106560 2024-04-22T23:35:56.653 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe3fff700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe419c3f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009520 tx=0x7f4fd4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:56.654 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fd4003a80 con 0x7f4fe4106560 2024-04-22T23:35:56.654 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4fd4003be0 con 0x7f4fe4106560 2024-04-22T23:35:56.654 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe419cb30 con 0x7f4fe4106560 2024-04-22T23:35:56.654 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4fd40197b0 con 0x7f4fe4106560 2024-04-22T23:35:56.654 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.784+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe419cfd0 con 0x7f4fe4106560 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.785+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f4fd4019a20 con 0x7f4fe4106560 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.785+0000 7f4fe17fa700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7f4fcc0382d0 0x7f4fcc03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.785+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4fd404c490 con 0x7f4fe4106560 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.785+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40623c0 con 0x7f4fe4106560 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.785+0000 7f4fe37fe700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7f4fcc0382d0 0x7f4fcc03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.786+0000 7f4fe37fe700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7f4fcc0382d0 0x7f4fcc03a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006fd0 tx=0x7f4fd8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:55.791+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4fd402d440 con 0x7f4fe4106560 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.068+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4fe41a0070 con 0x7f4fe4106560 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.069+0000 7f4fe17fa700 1 -- 172.21.15.31:0/1572884680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f4fd4018350 con 0x7f4fe4106560 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7f4fcc0382d0 msgr2=0x7f4fcc03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7f4fcc0382d0 0x7f4fcc03a790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006fd0 tx=0x7f4fd8006e40 comp rx=0 tx=0).stop 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 msgr2=0x7f4fe419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe419c3f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009520 tx=0x7f4fd4011ab0 comp rx=0 tx=0).stop 2024-04-22T23:35:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 shutdown_connections 2024-04-22T23:35:56.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7f4fcc0382d0 0x7f4fcc03a790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:56.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 --2- 172.21.15.31:0/1572884680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fe4106560 0x7f4fe419c3f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:56.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 >> 172.21.15.31:0/1572884680 conn(0x7f4fe4101b40 msgr2=0x7f4fe41028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:56.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 shutdown_connections 2024-04-22T23:35:56.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:56.071+0000 7f4fea741700 1 -- 172.21.15.31:0/1572884680 wait complete. 2024-04-22T23:35:56.657 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-04-22T23:35:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:57 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3126937239' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T23:35:58.299 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:58.299 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T23:35:58.299 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:35:58.299 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T23:35:58.299 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1c5a43700 1 Processor -- start 2024-04-22T23:35:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1c5a43700 1 -- start start 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1c5a43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1c5a43700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c007a8b0 con 0x7fd1c007be80 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33576/0 (socket says 172.21.15.31:33576) 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.038+0000 7fd1beffd700 1 -- 172.21.15.31:0/3820597100 learned_addr learned my addr 172.21.15.31:0/3820597100 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.039+0000 7fd1beffd700 1 -- 172.21.15.31:0/3820597100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c007a9f0 con 0x7fd1c007be80 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.039+0000 7fd1beffd700 1 --2- 172.21.15.31:0/3820597100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c007a2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80098a0 tx=0x7fd1b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4aa8390daa0b5f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.039+0000 7fd1bdffb700 1 -- 172.21.15.31:0/3820597100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1b8004180 con 0x7fd1c007be80 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.039+0000 7fd1bdffb700 1 -- 172.21.15.31:0/3820597100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1b80042e0 con 0x7fd1c007be80 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.039+0000 7fd1bdffb700 1 -- 172.21.15.31:0/3820597100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1b8011a50 con 0x7fd1c007be80 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3820597100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 msgr2=0x7fd1c007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 --2- 172.21.15.31:0/3820597100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c007a2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80098a0 tx=0x7fd1b8009bb0 comp rx=0 tx=0).stop 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3820597100 shutdown_connections 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 --2- 172.21.15.31:0/3820597100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c007a2e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3820597100 >> 172.21.15.31:0/3820597100 conn(0x7fd1c0101b40 msgr2=0x7fd1c0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3820597100 shutdown_connections 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.040+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3820597100 wait complete. 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1c5a43700 1 Processor -- start 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1c5a43700 1 -- start start 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1c5a43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1c5a43700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c019c960 con 0x7fd1c007be80 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33588/0 (socket says 172.21.15.31:33588) 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1beffd700 1 -- 172.21.15.31:0/3126937239 learned_addr learned my addr 172.21.15.31:0/3126937239 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.041+0000 7fd1beffd700 1 -- 172.21.15.31:0/3126937239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009550 con 0x7fd1c007be80 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.042+0000 7fd1beffd700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c019c420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009520 tx=0x7fd1b8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.042+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1b8003a80 con 0x7fd1c007be80 2024-04-22T23:35:58.302 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.042+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd1b8003be0 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.042+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1b80197b0 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.042+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c019cb60 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.042+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c019d000 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.043+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fd1b8019a20 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.043+0000 7fd1c4a41700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fd1a80382d0 0x7fd1a803a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.043+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c010c8c0 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.043+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd1b804c620 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.043+0000 7fd1be7fc700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fd1a80382d0 0x7fd1a803a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.044+0000 7fd1be7fc700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fd1a80382d0 0x7fd1a803a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006fd0 tx=0x7fd1b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.048+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd1b802c3a0 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.244+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd1c010cae0 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.245+0000 7fd1c4a41700 1 -- 172.21.15.31:0/3126937239 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd1b8023390 con 0x7fd1c007be80 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fd1a80382d0 msgr2=0x7fd1a803a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:58.303 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fd1a80382d0 0x7fd1a803a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006fd0 tx=0x7fd1b4006e40 comp rx=0 tx=0).stop 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 msgr2=0x7fd1c019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c019c420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009520 tx=0x7fd1b8011ab0 comp rx=0 tx=0).stop 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 shutdown_connections 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fd1a80382d0 0x7fd1a803a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 --2- 172.21.15.31:0/3126937239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c007be80 0x7fd1c019c420 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 >> 172.21.15.31:0/3126937239 conn(0x7fd1c0101b40 msgr2=0x7fd1c01031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 shutdown_connections 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:57.247+0000 7fd1c5a43700 1 -- 172.21.15.31:0/3126937239 wait complete. 2024-04-22T23:35:58.304 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-04-22T23:35:59.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:35:59 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4254013752' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-22T23:36:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.155+0000 7fdb50a7a700 1 Processor -- start 2024-04-22T23:36:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.155+0000 7fdb50a7a700 1 -- start start 2024-04-22T23:36:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.155+0000 7fdb50a7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.155+0000 7fdb50a7a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c07a890 con 0x7fdb4c07be60 2024-04-22T23:36:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.155+0000 7fdb4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33590/0 (socket says 172.21.15.31:33590) 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4a59c700 1 -- 172.21.15.31:0/2461140955 learned_addr learned my addr 172.21.15.31:0/2461140955 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4a59c700 1 -- 172.21.15.31:0/2461140955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c07a9d0 con 0x7fdb4c07be60 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4a59c700 1 --2- 172.21.15.31:0/2461140955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c07a2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009e80 tx=0x7fdb3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d314795c7e1e9f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4959a700 1 -- 172.21.15.31:0/2461140955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb3c00e070 con 0x7fdb4c07be60 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4959a700 1 -- 172.21.15.31:0/2461140955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb3c003e90 con 0x7fdb4c07be60 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.156+0000 7fdb4959a700 1 -- 172.21.15.31:0/2461140955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb3c004380 con 0x7fdb4c07be60 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 -- 172.21.15.31:0/2461140955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 msgr2=0x7fdb4c07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 --2- 172.21.15.31:0/2461140955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c07a2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009e80 tx=0x7fdb3c00b330 comp rx=0 tx=0).stop 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 -- 172.21.15.31:0/2461140955 shutdown_connections 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 --2- 172.21.15.31:0/2461140955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c07a2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 -- 172.21.15.31:0/2461140955 >> 172.21.15.31:0/2461140955 conn(0x7fdb4c101b40 msgr2=0x7fdb4c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 -- 172.21.15.31:0/2461140955 shutdown_connections 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.157+0000 7fdb50a7a700 1 -- 172.21.15.31:0/2461140955 wait complete. 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb50a7a700 1 Processor -- start 2024-04-22T23:36:00.043 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb50a7a700 1 -- start start 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb50a7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb50a7a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c19c8b0 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb4a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c19c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33594/0 (socket says 172.21.15.31:33594) 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb4a59c700 1 -- 172.21.15.31:0/4254013752 learned_addr learned my addr 172.21.15.31:0/4254013752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.158+0000 7fdb4a59c700 1 -- 172.21.15.31:0/4254013752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c009b30 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.159+0000 7fdb4a59c700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c19c370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00b7b0 tx=0x7fdb3c004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.159+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb3c00e040 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.159+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c19cab0 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.159+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb3c003c30 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.159+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb3c01b740 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.159+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c19cfb0 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.160+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fdb3c019040 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.160+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c10c8e0 con 0x7fdb4c07be60 2024-04-22T23:36:00.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.160+0000 7fdb437fe700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fdb380382d0 0x7fdb3803a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.160+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdb3c04bbc0 con 0x7fdb4c07be60 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.160+0000 7fdb49d9b700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fdb380382d0 0x7fdb3803a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.160+0000 7fdb49d9b700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fdb380382d0 0x7fdb3803a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006fd0 tx=0x7fdb34006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.165+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb3c011980 con 0x7fdb4c07be60 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.418+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fdb4c10cad0 con 0x7fdb4c07be60 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.471+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fdb3c020d70 con 0x7fdb4c07be60 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.472+0000 7fdb437fe700 1 -- 172.21.15.31:0/4254013752 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fdb3c04b5e0 con 0x7fdb4c07be60 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.474+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fdb380382d0 msgr2=0x7fdb3803a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.474+0000 7fdb50a7a700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fdb380382d0 0x7fdb3803a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006fd0 tx=0x7fdb34006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.474+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 msgr2=0x7fdb4c19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.474+0000 7fdb50a7a700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c19c370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00b7b0 tx=0x7fdb3c004920 comp rx=0 tx=0).stop 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.475+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 shutdown_connections 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.475+0000 7fdb50a7a700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fdb380382d0 0x7fdb3803a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.475+0000 7fdb50a7a700 1 --2- 172.21.15.31:0/4254013752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb4c07be60 0x7fdb4c19c370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.475+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 >> 172.21.15.31:0/4254013752 conn(0x7fdb4c101b40 msgr2=0x7fdb4c1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:00.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.475+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 shutdown_connections 2024-04-22T23:36:00.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:35:59.475+0000 7fdb50a7a700 1 -- 172.21.15.31:0/4254013752 wait complete. 2024-04-22T23:36:00.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:00 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4254013752' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-22T23:36:00.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:00 smithi031 ceph-mon[26721]: mgrmap e5: smithi031.fazdcv(active, since 10s) 2024-04-22T23:36:01.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "smithi031.fazdcv", 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08cb55700 1 Processor -- start 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08cb55700 1 -- start start 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08cb55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb0881093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08cb55700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088074680 con 0x7fb088106fe0 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb0881093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb0881093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33616/0 (socket says 172.21.15.31:33616) 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.511+0000 7fb08659c700 1 -- 172.21.15.31:0/3884427895 learned_addr learned my addr 172.21.15.31:0/3884427895 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.512+0000 7fb08659c700 1 -- 172.21.15.31:0/3884427895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088109910 con 0x7fb088106fe0 2024-04-22T23:36:01.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.512+0000 7fb08659c700 1 --2- 172.21.15.31:0/3884427895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb0881093d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098a0 tx=0x7fb078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f8b7d7fdfa0e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.512+0000 7fb08559a700 1 -- 172.21.15.31:0/3884427895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb078004180 con 0x7fb088106fe0 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.512+0000 7fb08559a700 1 -- 172.21.15.31:0/3884427895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0780042e0 con 0x7fb088106fe0 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.512+0000 7fb08559a700 1 -- 172.21.15.31:0/3884427895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb078011a50 con 0x7fb088106fe0 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 -- 172.21.15.31:0/3884427895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 msgr2=0x7fb0881093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 --2- 172.21.15.31:0/3884427895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb0881093d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098a0 tx=0x7fb078009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 -- 172.21.15.31:0/3884427895 shutdown_connections 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 --2- 172.21.15.31:0/3884427895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb0881093d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 -- 172.21.15.31:0/3884427895 >> 172.21.15.31:0/3884427895 conn(0x7fb0881009f0 msgr2=0x7fb088102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 -- 172.21.15.31:0/3884427895 shutdown_connections 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.513+0000 7fb08cb55700 1 -- 172.21.15.31:0/3884427895 wait complete. 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08cb55700 1 Processor -- start 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08cb55700 1 -- start start 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08cb55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb08819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08cb55700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08819a670 con 0x7fb088106fe0 2024-04-22T23:36:01.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb08819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb08819a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33630/0 (socket says 172.21.15.31:33630) 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08659c700 1 -- 172.21.15.31:0/890880993 learned_addr learned my addr 172.21.15.31:0/890880993 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08659c700 1 -- 172.21.15.31:0/890880993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009550 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.514+0000 7fb08659c700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb08819a130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb078000c00 tx=0x7fb078011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.515+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb078003b90 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.515+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0780248b0 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.515+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08819a870 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.515+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb07801a8e0 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.515+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08819ad10 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.516+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb07801aba0 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.516+0000 7fb0777fe700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 0x7fb07003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.516+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08810a690 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.516+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb07804c120 con 0x7fb088106fe0 2024-04-22T23:36:01.377 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.516+0000 7fb085d9b700 1 -- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 msgr2=0x7fb07003a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.516+0000 7fb085d9b700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 0x7fb07003a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.521+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb0780237b0 con 0x7fb088106fe0 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.717+0000 7fb085d9b700 1 -- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 msgr2=0x7fb07003a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.717+0000 7fb085d9b700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 0x7fb07003a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.821+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb08819daf0 con 0x7fb088106fe0 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.822+0000 7fb0777fe700 1 -- 172.21.15.31:0/890880993 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fb0780172b0 con 0x7fb088106fe0 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 msgr2=0x7fb07003a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 0x7fb07003a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 msgr2=0x7fb08819a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb08819a130 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb078000c00 tx=0x7fb078011ab0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 shutdown_connections 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fb070038320 0x7fb07003a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.378 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 --2- 172.21.15.31:0/890880993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb088106fe0 0x7fb08819a130 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 >> 172.21.15.31:0/890880993 conn(0x7fb0881009f0 msgr2=0x7fb088101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 shutdown_connections 2024-04-22T23:36:01.379 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:00.824+0000 7fb08cb55700 1 -- 172.21.15.31:0/890880993 wait complete. 2024-04-22T23:36:01.379 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-22T23:36:01.379 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-04-22T23:36:01.743 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:01 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/890880993' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: Active manager daemon smithi031.fazdcv restarted 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: osdmap e2: 0 total, 0 up, 0 in 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: mgrmap e6: smithi031.fazdcv(active, starting, since 0.00319805s) 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:36:06.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:36:06.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:06 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:36:07.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:07 smithi031 ceph-mon[26721]: Found migration_current of "None". Setting to last migration. 2024-04-22T23:36:07.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:07 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:07.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:07 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:07.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:07 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:07.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:07 smithi031 ceph-mon[26721]: mgrmap e7: smithi031.fazdcv(active, since 1.0065s) 2024-04-22T23:36:07.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe78aae700 1 Processor -- start 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe78aae700 1 -- start start 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe78aae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe741093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:07.982 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe78aae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74074680 con 0x7fbe74106fe0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe741093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe741093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34964/0 (socket says 172.21.15.31:34964) 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe7259c700 1 -- 172.21.15.31:0/710492311 learned_addr learned my addr 172.21.15.31:0/710492311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.759+0000 7fbe7259c700 1 -- 172.21.15.31:0/710492311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74109910 con 0x7fbe74106fe0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.760+0000 7fbe7259c700 1 --2- 172.21.15.31:0/710492311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe741093d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098a0 tx=0x7fbe64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d78c57ed1fa7a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.760+0000 7fbe7159a700 1 -- 172.21.15.31:0/710492311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe64004180 con 0x7fbe74106fe0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.760+0000 7fbe7159a700 1 -- 172.21.15.31:0/710492311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe640042e0 con 0x7fbe74106fe0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.760+0000 7fbe7159a700 1 -- 172.21.15.31:0/710492311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe64011a50 con 0x7fbe74106fe0 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.760+0000 7fbe78aae700 1 -- 172.21.15.31:0/710492311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 msgr2=0x7fbe741093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 --2- 172.21.15.31:0/710492311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe741093d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe640098a0 tx=0x7fbe64009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 -- 172.21.15.31:0/710492311 shutdown_connections 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 --2- 172.21.15.31:0/710492311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe741093d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 -- 172.21.15.31:0/710492311 >> 172.21.15.31:0/710492311 conn(0x7fbe741009f0 msgr2=0x7fbe74102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 -- 172.21.15.31:0/710492311 shutdown_connections 2024-04-22T23:36:07.983 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 -- 172.21.15.31:0/710492311 wait complete. 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 Processor -- start 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.761+0000 7fbe78aae700 1 -- start start 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe78aae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe7419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe78aae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7419c8c0 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe7419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe7259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe7419c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34978/0 (socket says 172.21.15.31:34978) 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe7259c700 1 -- 172.21.15.31:0/2226117669 learned_addr learned my addr 172.21.15.31:0/2226117669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe7259c700 1 -- 172.21.15.31:0/2226117669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64009550 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe7259c700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe7419c380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe640119e0 tx=0x7fbe64011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe64003b90 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7419cac0 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbe6401a8a0 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.762+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe64019930 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.763+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7419cf60 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.764+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fbe6401aa10 con 0x7fbe74106fe0 2024-04-22T23:36:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.764+0000 7fbe637fe700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.764+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 --> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe5c03aeb0 con 0x7fbe5c0382e0 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.764+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbe6404c6d0 con 0x7fbe74106fe0 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.764+0000 7fbe71d9b700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 msgr2=0x7fbe5c03a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.764+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.965+0000 7fbe71d9b700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 msgr2=0x7fbe5c03a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:01.965+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:02.366+0000 7fbe71d9b700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 msgr2=0x7fbe5c03a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:02.366+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:03.168+0000 7fbe71d9b700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 msgr2=0x7fbe5c03a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:03.168+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:04.769+0000 7fbe71d9b700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 msgr2=0x7fbe5c03a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2198454822 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:04.769+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:06.363+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7fbe64023c40 con 0x7fbe74106fe0 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:06.363+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 msgr2=0x7fbe5c03a7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:36:07.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:06.363+0000 7fbe637fe700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.365+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fbe6404e4f0 con 0x7fbe74106fe0 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.365+0000 7fbe637fe700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe5c03bc60 0x7fbe5c03e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.365+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe5c03aeb0 con 0x7fbe5c03bc60 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.366+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe5c03bc60 0x7fbe5c03e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.367+0000 7fbe71d9b700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe5c03bc60 0x7fbe5c03e050 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbe68003a10 tx=0x7fbe68009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.368+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fbe5c03aeb0 con 0x7fbe5c03bc60 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.375+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbe7419d670 con 0x7fbe5c03bc60 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe637fe700 1 -- 172.21.15.31:0/2226117669 <== mgr.14118 v2:172.21.15.31:6800/863972114 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbe7419d670 con 0x7fbe5c03bc60 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe5c03bc60 msgr2=0x7fbe5c03e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe5c03bc60 0x7fbe5c03e050 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbe68003a10 tx=0x7fbe68009290 comp rx=0 tx=0).stop 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 msgr2=0x7fbe7419c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe7419c380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe640119e0 tx=0x7fbe64011ac0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 shutdown_connections 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe5c03bc60 0x7fbe5c03e050 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe74106fe0 0x7fbe7419c380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 --2- 172.21.15.31:0/2226117669 >> [v2:172.21.15.31:6800/2198454822,v1:172.21.15.31:6801/2198454822] conn(0x7fbe5c0382e0 0x7fbe5c03a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:07.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 >> 172.21.15.31:0/2226117669 conn(0x7fbe741009f0 msgr2=0x7fbe74101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:07.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 shutdown_connections 2024-04-22T23:36:07.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:07.376+0000 7fbe78aae700 1 -- 172.21.15.31:0/2226117669 wait complete. 2024-04-22T23:36:07.987 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-04-22T23:36:07.987 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-04-22T23:36:08.633 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:08 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:06] ENGINE Bus STARTING 2024-04-22T23:36:08.633 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:08 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:06] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:36:08.633 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:08 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:06] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:36:08.634 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:08 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:06] ENGINE Bus STARTED 2024-04-22T23:36:08.634 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:08 smithi031 ceph-mon[26721]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T23:36:08.634 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:08 smithi031 ceph-mon[26721]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9e5c3a700 1 Processor -- start 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9e5c3a700 1 -- start start 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9e5c3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e0106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9e5c3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0106f50 con 0x7fc9e0106560 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e0106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e0106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35046/0 (socket says 172.21.15.31:35046) 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.411+0000 7fc9df7fe700 1 -- 172.21.15.31:0/693868941 learned_addr learned my addr 172.21.15.31:0/693868941 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9df7fe700 1 -- 172.21.15.31:0/693868941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0107760 con 0x7fc9e0106560 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9df7fe700 1 --2- 172.21.15.31:0/693868941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e0106980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00098a0 tx=0x7fc9d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544561795cf0b6a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9de7fc700 1 -- 172.21.15.31:0/693868941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9d0004180 con 0x7fc9e0106560 2024-04-22T23:36:09.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9de7fc700 1 -- 172.21.15.31:0/693868941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9d00042e0 con 0x7fc9e0106560 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9de7fc700 1 -- 172.21.15.31:0/693868941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9d0011a50 con 0x7fc9e0106560 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/693868941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 msgr2=0x7fc9e0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.412+0000 7fc9e5c3a700 1 --2- 172.21.15.31:0/693868941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e0106980 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00098a0 tx=0x7fc9d0009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/693868941 shutdown_connections 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 --2- 172.21.15.31:0/693868941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e0106980 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/693868941 >> 172.21.15.31:0/693868941 conn(0x7fc9e0101b40 msgr2=0x7fc9e0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/693868941 shutdown_connections 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/693868941 wait complete. 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 Processor -- start 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.413+0000 7fc9e5c3a700 1 -- start start 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9e5c3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9e5c3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e019a730 con 0x7fc9e0106560 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35050/0 (socket says 172.21.15.31:35050) 2024-04-22T23:36:09.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9df7fe700 1 -- 172.21.15.31:0/3491967363 learned_addr learned my addr 172.21.15.31:0/3491967363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9df7fe700 1 -- 172.21.15.31:0/3491967363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0009550 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.414+0000 7fc9df7fe700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e019a1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009520 tx=0x7fc9d0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.418+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9d0003a80 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.418+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc9d0003be0 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.419+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e019a930 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.419+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9d00197b0 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.419+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e019add0 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.419+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fc9d0019af0 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.420+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e00623c0 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.420+0000 7fc9dcff9700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fc9cc0381a0 0x7fc9cc03a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.420+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc9d004c3d0 con 0x7fc9e0106560 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.420+0000 7fc9deffd700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fc9cc0381a0 0x7fc9cc03a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.420+0000 7fc9deffd700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fc9cc0381a0 0x7fc9cc03a660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006fd0 tx=0x7fc9c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:09.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.425+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9d002b750 con 0x7fc9e0106560 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.426+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc9d0016d40 con 0x7fc9e0106560 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.639+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9e010a960 con 0x7fc9cc0381a0 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.645+0000 7fc9dcff9700 1 -- 172.21.15.31:0/3491967363 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc9e010a960 con 0x7fc9cc0381a0 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fc9cc0381a0 msgr2=0x7fc9cc03a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fc9cc0381a0 0x7fc9cc03a660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8006fd0 tx=0x7fc9c8006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 msgr2=0x7fc9e019a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e019a1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009520 tx=0x7fc9d0011ab0 comp rx=0 tx=0).stop 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 shutdown_connections 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fc9cc0381a0 0x7fc9cc03a660 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 --2- 172.21.15.31:0/3491967363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9e0106560 0x7fc9e019a1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 >> 172.21.15.31:0/3491967363 conn(0x7fc9e0101b40 msgr2=0x7fc9e0102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 shutdown_connections 2024-04-22T23:36:09.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:08.647+0000 7fc9e5c3a700 1 -- 172.21.15.31:0/3491967363 wait complete. 2024-04-22T23:36:09.613 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:09 smithi031 ceph-mon[26721]: mgrmap e8: smithi031.fazdcv(active, since 2s) 2024-04-22T23:36:09.613 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:09 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:09.613 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:09 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:10.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:10 smithi031 ceph-mon[26721]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:10.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-22T23:36:10.491 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.632+0000 7f31e53b0700 1 Processor -- start 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.632+0000 7f31e53b0700 1 -- start start 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.632+0000 7f31e53b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e01093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.632+0000 7f31e53b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e0079e50 con 0x7f31e0107000 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.632+0000 7f31deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e01093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e01093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35064/0 (socket says 172.21.15.31:35064) 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31deffd700 1 -- 172.21.15.31:0/2979740648 learned_addr learned my addr 172.21.15.31:0/2979740648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31deffd700 1 -- 172.21.15.31:0/2979740648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0109930 con 0x7f31e0107000 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31deffd700 1 --2- 172.21.15.31:0/2979740648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e01093f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009e80 tx=0x7f31d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130a4a34c4f2e781 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31ddffb700 1 -- 172.21.15.31:0/2979740648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31d800e070 con 0x7f31e0107000 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31ddffb700 1 -- 172.21.15.31:0/2979740648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31d8003e90 con 0x7f31e0107000 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.633+0000 7f31ddffb700 1 -- 172.21.15.31:0/2979740648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31d8004380 con 0x7f31e0107000 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 -- 172.21.15.31:0/2979740648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 msgr2=0x7f31e01093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 --2- 172.21.15.31:0/2979740648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e01093f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009e80 tx=0x7f31d800b330 comp rx=0 tx=0).stop 2024-04-22T23:36:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 -- 172.21.15.31:0/2979740648 shutdown_connections 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 --2- 172.21.15.31:0/2979740648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e01093f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 -- 172.21.15.31:0/2979740648 >> 172.21.15.31:0/2979740648 conn(0x7f31e01009b0 msgr2=0x7f31e0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 -- 172.21.15.31:0/2979740648 shutdown_connections 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 -- 172.21.15.31:0/2979740648 wait complete. 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.634+0000 7f31e53b0700 1 Processor -- start 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31e53b0700 1 -- start start 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31e53b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31e53b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e019c8e0 con 0x7f31e0107000 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35066/0 (socket says 172.21.15.31:35066) 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31deffd700 1 -- 172.21.15.31:0/2888010139 learned_addr learned my addr 172.21.15.31:0/2888010139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31deffd700 1 -- 172.21.15.31:0/2888010139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d8009b30 con 0x7f31e0107000 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31deffd700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e019c3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b7b0 tx=0x7f31d8004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.635+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31d800e040 con 0x7f31e0107000 2024-04-22T23:36:10.493 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.636+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31d8003c30 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.636+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e019cae0 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.636+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31d801b740 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.636+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e019cfe0 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.636+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f31d8019040 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.637+0000 7f31d7fff700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f31cc038320 0x7f31cc03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.637+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f31d804bee0 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.637+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e010a680 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.637+0000 7f31de7fc700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f31cc038320 0x7f31cc03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.637+0000 7f31de7fc700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f31cc038320 0x7f31cc03a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f31c8006fd0 tx=0x7f31c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.642+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f31d8011980 con 0x7f31e0107000 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.843+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f31e019fd60 con 0x7f31cc038320 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.844+0000 7f31d7fff700 1 -- 172.21.15.31:0/2888010139 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f31e019fd60 con 0x7f31cc038320 2024-04-22T23:36:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f31cc038320 msgr2=0x7f31cc03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f31cc038320 0x7f31cc03a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f31c8006fd0 tx=0x7f31c8006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 msgr2=0x7f31e019c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e019c3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b7b0 tx=0x7f31d8004920 comp rx=0 tx=0).stop 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 shutdown_connections 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f31cc038320 0x7f31cc03a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 --2- 172.21.15.31:0/2888010139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e0107000 0x7f31e019c3a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 >> 172.21.15.31:0/2888010139 conn(0x7f31e01009b0 msgr2=0x7f31e0101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 shutdown_connections 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:09.846+0000 7f31e53b0700 1 -- 172.21.15.31:0/2888010139 wait complete. 2024-04-22T23:36:10.495 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-04-22T23:36:11.680 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:11 smithi031 ceph-mon[26721]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:11.681 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:11 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:11.681 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:11 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:11.972 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672c43c700 1 Processor -- start 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672c43c700 1 -- start start 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672c43c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672c43c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672407a890 con 0x7f672407be60 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672a1d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672a1d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51248/0 (socket says 172.21.15.31:51248) 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.903+0000 7f672a1d8700 1 -- 172.21.15.31:0/4164280187 learned_addr learned my addr 172.21.15.31:0/4164280187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f672a1d8700 1 -- 172.21.15.31:0/4164280187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672407a9d0 con 0x7f672407be60 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f672a1d8700 1 --2- 172.21.15.31:0/4164280187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672407a2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f67200098a0 tx=0x7f6720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8b4f8b55fcc113 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f67291d6700 1 -- 172.21.15.31:0/4164280187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6720004180 con 0x7f672407be60 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f67291d6700 1 -- 172.21.15.31:0/4164280187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67200042e0 con 0x7f672407be60 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f67291d6700 1 -- 172.21.15.31:0/4164280187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6720011a50 con 0x7f672407be60 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f672c43c700 1 -- 172.21.15.31:0/4164280187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 msgr2=0x7f672407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.904+0000 7f672c43c700 1 --2- 172.21.15.31:0/4164280187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672407a2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f67200098a0 tx=0x7f6720009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 -- 172.21.15.31:0/4164280187 shutdown_connections 2024-04-22T23:36:11.973 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 --2- 172.21.15.31:0/4164280187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672407a2c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 -- 172.21.15.31:0/4164280187 >> 172.21.15.31:0/4164280187 conn(0x7f6724101b40 msgr2=0x7f6724103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 -- 172.21.15.31:0/4164280187 shutdown_connections 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 -- 172.21.15.31:0/4164280187 wait complete. 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 Processor -- start 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 -- start start 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.905+0000 7f672c43c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672419c910 con 0x7f672407be60 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672a1d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672a1d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51256/0 (socket says 172.21.15.31:51256) 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672a1d8700 1 -- 172.21.15.31:0/2247798801 learned_addr learned my addr 172.21.15.31:0/2247798801 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672a1d8700 1 -- 172.21.15.31:0/2247798801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6720009550 con 0x7f672407be60 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672a1d8700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672419c3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6720003f80 tx=0x7f6720011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6720003a80 con 0x7f672407be60 2024-04-22T23:36:11.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672419cb10 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6720003be0 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6720019820 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.906+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672419cfb0 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.907+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6720019aa0 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.907+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672410c8e0 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.907+0000 7f671b7fe700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f67100382d0 0x7f671003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.907+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f672004c580 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.907+0000 7f67299d7700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f67100382d0 0x7f671003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.908+0000 7f67299d7700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f67100382d0 0x7f671003a790 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6714006fd0 tx=0x7f6714006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:10.912+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6720023b90 con 0x7f672407be60 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.110+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6724102fe0 con 0x7f67100382d0 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.375+0000 7f671b7fe700 1 -- 172.21.15.31:0/2247798801 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6724102fe0 con 0x7f67100382d0 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f67100382d0 msgr2=0x7f671003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f67100382d0 0x7f671003a790 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6714006fd0 tx=0x7f6714006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 msgr2=0x7f672419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:11.975 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672419c3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6720003f80 tx=0x7f6720011ab0 comp rx=0 tx=0).stop 2024-04-22T23:36:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 shutdown_connections 2024-04-22T23:36:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f67100382d0 0x7f671003a790 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 --2- 172.21.15.31:0/2247798801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f672407be60 0x7f672419c3d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 >> 172.21.15.31:0/2247798801 conn(0x7f6724101b40 msgr2=0x7f67241028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 shutdown_connections 2024-04-22T23:36:11.976 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:11.377+0000 7f672c43c700 1 -- 172.21.15.31:0/2247798801 wait complete. 2024-04-22T23:36:12.528 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:12 smithi031 ceph-mon[26721]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:12.528 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:12 smithi031 ceph-mon[26721]: Generating ssh key... 2024-04-22T23:36:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKc0zURdJ64of0unNZ2HniNNJwVczbDDpBIDpHi58qBg1qgFv+ZMnZ9k0W1DrA8ccxeCS1yEfSZ61b8R6GjsOGL1ejNgpVdjHbKlQ23B3fvykPeMaynnFj2RprhUy2l5PQk/iQZekR5VEJMx1GylQXlpz0X/GqDuIoVNerSk5rCSk3bWCmByl3IVFLH086nGN2wT53fc01KqasGBliwMw4tLbkKkIBUAnuq2U0eFvW2bOQQVPqQ4QmDTquZNWgtbhRW6JLTzlnNJrxAIP4PVeF+y1rKf4Figwo0T2HqTGbMnwts4+V5lN4O2Az60zCNXjGt5YBo/pWddtQeauJTlQPkmINCqxeMibo93ARWO1MSiJ4SbbQtSpWSJvC2Ir7BIqWGtB15AaIdSJT8CpCCZTZznHfxgx54L1HudIyHRhEgKtt0+234G0yQVvhz8hhJvT/reqiJjp0fl4QkzYDav6ODsdTSEGOL7LWwAJdMetF7Jy/Yd6BvoLKLjHJDGug7JU= ceph-efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:36:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48efed700 1 Processor -- start 2024-04-22T23:36:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48efed700 1 -- start start 2024-04-22T23:36:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48efed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48efed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48807a8b0 con 0x7fb48807be80 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48cd89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48cd89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51266/0 (socket says 172.21.15.31:51266) 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48cd89700 1 -- 172.21.15.31:0/83104859 learned_addr learned my addr 172.21.15.31:0/83104859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.324+0000 7fb48cd89700 1 -- 172.21.15.31:0/83104859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48807a9f0 con 0x7fb48807be80 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48cd89700 1 --2- 172.21.15.31:0/83104859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48807a2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0098a0 tx=0x7fb47c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf954f95c8e59511 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb4877fe700 1 -- 172.21.15.31:0/83104859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb47c004180 con 0x7fb48807be80 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb4877fe700 1 -- 172.21.15.31:0/83104859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb47c0042e0 con 0x7fb48807be80 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb4877fe700 1 -- 172.21.15.31:0/83104859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb47c011a50 con 0x7fb48807be80 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48efed700 1 -- 172.21.15.31:0/83104859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 msgr2=0x7fb48807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48efed700 1 --2- 172.21.15.31:0/83104859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48807a2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0098a0 tx=0x7fb47c009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48efed700 1 -- 172.21.15.31:0/83104859 shutdown_connections 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48efed700 1 --2- 172.21.15.31:0/83104859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48807a2e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:13.155 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48efed700 1 -- 172.21.15.31:0/83104859 >> 172.21.15.31:0/83104859 conn(0x7fb488101b40 msgr2=0x7fb488103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.325+0000 7fb48efed700 1 -- 172.21.15.31:0/83104859 shutdown_connections 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48efed700 1 -- 172.21.15.31:0/83104859 wait complete. 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48efed700 1 Processor -- start 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48efed700 1 -- start start 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48efed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48efed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48819c9a0 con 0x7fb48807be80 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48cd89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48cd89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51282/0 (socket says 172.21.15.31:51282) 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.326+0000 7fb48cd89700 1 -- 172.21.15.31:0/2158355257 learned_addr learned my addr 172.21.15.31:0/2158355257 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb48cd89700 1 -- 172.21.15.31:0/2158355257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c009550 con 0x7fb48807be80 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb48cd89700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48819c460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb47c003de0 tx=0x7fb47c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb47c003930 con 0x7fb48807be80 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb47c003a90 con 0x7fb48807be80 2024-04-22T23:36:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48819cba0 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb47c0196c0 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.327+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48819d040 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.328+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb47c019930 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.328+0000 7fb485ffb700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fb478038370 0x7fb47803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.328+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb47c04c370 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.328+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4880623c0 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.328+0000 7fb487fff700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fb478038370 0x7fb47803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.329+0000 7fb487fff700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fb478038370 0x7fb47803a830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb474006fd0 tx=0x7fb474006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.333+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb47c019c30 con 0x7fb48807be80 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.534+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb48819fa00 con 0x7fb478038370 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.534+0000 7fb485ffb700 1 -- 172.21.15.31:0/2158355257 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb48819fa00 con 0x7fb478038370 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fb478038370 msgr2=0x7fb47803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fb478038370 0x7fb47803a830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb474006fd0 tx=0x7fb474006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 msgr2=0x7fb48819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48819c460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb47c003de0 tx=0x7fb47c011ab0 comp rx=0 tx=0).stop 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 shutdown_connections 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fb478038370 0x7fb47803a830 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 --2- 172.21.15.31:0/2158355257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb48807be80 0x7fb48819c460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 >> 172.21.15.31:0/2158355257 conn(0x7fb488101b40 msgr2=0x7fb4881031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 shutdown_connections 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:12.536+0000 7fb48efed700 1 -- 172.21.15.31:0/2158355257 wait complete. 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-04-22T23:36:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-04-22T23:36:13.763 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:13 smithi031 ceph-mon[26721]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:14.659 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:14 smithi031 ceph-mon[26721]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:14.659 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:14 smithi031 ceph-mon[26721]: Deploying cephadm binary to smithi031 2024-04-22T23:36:16.781 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa9a1319700 1 Processor -- start 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa9a1319700 1 -- start start 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa9a1319700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa9a1319700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c106f50 con 0x7fa99c106560 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa99affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa99affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51284/0 (socket says 172.21.15.31:51284) 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.557+0000 7fa99affd700 1 -- 172.21.15.31:0/714861979 learned_addr learned my addr 172.21.15.31:0/714861979 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa99affd700 1 -- 172.21.15.31:0/714861979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c107760 con 0x7fa99c106560 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa99affd700 1 --2- 172.21.15.31:0/714861979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c106980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098a0 tx=0x7fa98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1975f95d3ab5494c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:16.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa999ffb700 1 -- 172.21.15.31:0/714861979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98c004180 con 0x7fa99c106560 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa999ffb700 1 -- 172.21.15.31:0/714861979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa98c0042e0 con 0x7fa99c106560 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa999ffb700 1 -- 172.21.15.31:0/714861979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98c011a50 con 0x7fa99c106560 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa9a1319700 1 -- 172.21.15.31:0/714861979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 msgr2=0x7fa99c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.558+0000 7fa9a1319700 1 --2- 172.21.15.31:0/714861979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c106980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0098a0 tx=0x7fa98c009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 -- 172.21.15.31:0/714861979 shutdown_connections 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 --2- 172.21.15.31:0/714861979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c106980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 -- 172.21.15.31:0/714861979 >> 172.21.15.31:0/714861979 conn(0x7fa99c101b40 msgr2=0x7fa99c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 -- 172.21.15.31:0/714861979 shutdown_connections 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 -- 172.21.15.31:0/714861979 wait complete. 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 Processor -- start 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 -- start start 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.559+0000 7fa9a1319700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c19c930 con 0x7fa99c106560 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa99affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:16.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa99affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51296/0 (socket says 172.21.15.31:51296) 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa99affd700 1 -- 172.21.15.31:0/2982424364 learned_addr learned my addr 172.21.15.31:0/2982424364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa99affd700 1 -- 172.21.15.31:0/2982424364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009550 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa99affd700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c19c3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa98c003de0 tx=0x7fa98c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98c003930 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa98c003a90 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c19cb30 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98c0196c0 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.560+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c19cfd0 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.561+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa98c01abc0 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.561+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c0623c0 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.561+0000 7fa983fff700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa984038320 0x7fa98403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.561+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa98c04c410 con 0x7fa99c106560 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.561+0000 7fa99a7fc700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa984038320 0x7fa98403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.562+0000 7fa99a7fc700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa984038320 0x7fa98403a7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa990006fd0 tx=0x7fa990006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.566+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa98c010770 con 0x7fa99c106560 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:13.769+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) v1 -- 0x7fa99c10a960 con 0x7fa984038320 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.227+0000 7fa983fff700 1 -- 172.21.15.31:0/2982424364 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fa99c10a960 con 0x7fa984038320 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa984038320 msgr2=0x7fa98403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa984038320 0x7fa98403a7e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa990006fd0 tx=0x7fa990006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 msgr2=0x7fa99c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c19c3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa98c003de0 tx=0x7fa98c011ab0 comp rx=0 tx=0).stop 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 shutdown_connections 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa984038320 0x7fa98403a7e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 --2- 172.21.15.31:0/2982424364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa99c106560 0x7fa99c19c3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 >> 172.21.15.31:0/2982424364 conn(0x7fa99c101b40 msgr2=0x7fa99c1028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 shutdown_connections 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:16.229+0000 7fa9a1319700 1 -- 172.21.15.31:0/2982424364 wait complete. 2024-04-22T23:36:16.789 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying mon service with default placement... 2024-04-22T23:36:17.442 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:17 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:17.442 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:17 smithi031 ceph-mon[26721]: Added host smithi031 2024-04-22T23:36:17.442 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:17 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:18.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:18 smithi031 ceph-mon[26721]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:18.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:18 smithi031 ceph-mon[26721]: Saving service mon spec with placement count:5 2024-04-22T23:36:18.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:18 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf5e0ad700 1 Processor -- start 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf5e0ad700 1 -- start start 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf5e0ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf58104ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf5e0ad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581052b0 con 0x7fbf581048c0 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf58104ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf58104ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51306/0 (socket says 172.21.15.31:51306) 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.164+0000 7fbf577fe700 1 -- 172.21.15.31:0/2818267034 learned_addr learned my addr 172.21.15.31:0/2818267034 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:18.823 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.165+0000 7fbf577fe700 1 -- 172.21.15.31:0/2818267034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf58107860 con 0x7fbf581048c0 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.165+0000 7fbf577fe700 1 --2- 172.21.15.31:0/2818267034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf58104ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf480098a0 tx=0x7fbf48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c95cc96cb200d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.165+0000 7fbf567fc700 1 -- 172.21.15.31:0/2818267034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf48004180 con 0x7fbf581048c0 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.165+0000 7fbf567fc700 1 -- 172.21.15.31:0/2818267034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf480042e0 con 0x7fbf581048c0 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.165+0000 7fbf567fc700 1 -- 172.21.15.31:0/2818267034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf48011a50 con 0x7fbf581048c0 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/2818267034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 msgr2=0x7fbf58104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 --2- 172.21.15.31:0/2818267034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf58104ce0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbf480098a0 tx=0x7fbf48009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/2818267034 shutdown_connections 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 --2- 172.21.15.31:0/2818267034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf58104ce0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/2818267034 >> 172.21.15.31:0/2818267034 conn(0x7fbf581001a0 msgr2=0x7fbf581025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/2818267034 shutdown_connections 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/2818267034 wait complete. 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 Processor -- start 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.166+0000 7fbf5e0ad700 1 -- start start 2024-04-22T23:36:18.824 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf5e0ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf5819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf5e0ad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5819a760 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf5819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf5819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51320/0 (socket says 172.21.15.31:51320) 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf577fe700 1 -- 172.21.15.31:0/4274340065 learned_addr learned my addr 172.21.15.31:0/4274340065 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf577fe700 1 -- 172.21.15.31:0/4274340065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009550 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf577fe700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf5819a220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf48003970 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5819a960 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbf48003ad0 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf48024750 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.167+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5819ae00 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.168+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fbf4802d3f0 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.168+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf580623c0 con 0x7fbf581048c0 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.168+0000 7fbf54ff9700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbf44038320 0x7fbf4403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:18.825 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.169+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbf4804c1e0 con 0x7fbf581048c0 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.169+0000 7fbf56ffd700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbf44038320 0x7fbf4403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.169+0000 7fbf56ffd700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbf44038320 0x7fbf4403a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006fd0 tx=0x7fbf40006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.173+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbf480226d0 con 0x7fbf581048c0 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.396+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf58101610 con 0x7fbf44038320 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.401+0000 7fbf54ff9700 1 -- 172.21.15.31:0/4274340065 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbf58101610 con 0x7fbf44038320 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbf44038320 msgr2=0x7fbf4403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbf44038320 0x7fbf4403a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006fd0 tx=0x7fbf40006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 msgr2=0x7fbf5819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf5819a220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48011ab0 comp rx=0 tx=0).stop 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 shutdown_connections 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbf44038320 0x7fbf4403a7e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 --2- 172.21.15.31:0/4274340065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf581048c0 0x7fbf5819a220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:18.826 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 >> 172.21.15.31:0/4274340065 conn(0x7fbf581001a0 msgr2=0x7fbf58100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:18.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 shutdown_connections 2024-04-22T23:36:18.827 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:17.403+0000 7fbf5e0ad700 1 -- 172.21.15.31:0/4274340065 wait complete. 2024-04-22T23:36:18.827 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying mgr service with default placement... 2024-04-22T23:36:19.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:19 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:19.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:19 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e2c31700 1 Processor -- start 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e2c31700 1 -- start start 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e2c31700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc104c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e2c31700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc105230 con 0x7fa3dc104840 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e09cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e09cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51328/0 (socket says 172.21.15.31:51328) 2024-04-22T23:36:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.259+0000 7fa3e09cd700 1 -- 172.21.15.31:0/2974002978 learned_addr learned my addr 172.21.15.31:0/2974002978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3e09cd700 1 -- 172.21.15.31:0/2974002978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc1077d0 con 0x7fa3dc104840 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3e09cd700 1 --2- 172.21.15.31:0/2974002978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc104c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00098a0 tx=0x7fa3d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e546727152f7918 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3db7fe700 1 -- 172.21.15.31:0/2974002978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3d0004180 con 0x7fa3dc104840 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3db7fe700 1 -- 172.21.15.31:0/2974002978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa3d00042e0 con 0x7fa3dc104840 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3db7fe700 1 -- 172.21.15.31:0/2974002978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3d0011af0 con 0x7fa3dc104840 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2974002978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 msgr2=0x7fa3dc104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.260+0000 7fa3e2c31700 1 --2- 172.21.15.31:0/2974002978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc104c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00098a0 tx=0x7fa3d0009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2974002978 shutdown_connections 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 --2- 172.21.15.31:0/2974002978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc104c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2974002978 >> 172.21.15.31:0/2974002978 conn(0x7fa3dc100170 msgr2=0x7fa3dc1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2974002978 shutdown_connections 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2974002978 wait complete. 2024-04-22T23:36:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 Processor -- start 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 -- start start 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.261+0000 7fa3e2c31700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e2c31700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc19a7b0 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e09cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e09cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51336/0 (socket says 172.21.15.31:51336) 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e09cd700 1 -- 172.21.15.31:0/2434950865 learned_addr learned my addr 172.21.15.31:0/2434950865 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e09cd700 1 -- 172.21.15.31:0/2434950865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009550 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e09cd700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc19a270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011670 tx=0x7fa3d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3d00039c0 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa3d0003b20 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc19a9b0 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3d00247f0 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.262+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc19ae50 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.263+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa3d0024ab0 con 0x7fa3dc104840 2024-04-22T23:36:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.263+0000 7fa3d9ffb700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa3cc0382e0 0x7fa3cc03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.263+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa3d004c2e0 con 0x7fa3dc104840 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.264+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc0623c0 con 0x7fa3dc104840 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.264+0000 7fa3dbfff700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa3cc0382e0 0x7fa3cc03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.264+0000 7fa3dbfff700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa3cc0382e0 0x7fa3cc03a7a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.268+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa3d0022310 con 0x7fa3dc104840 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.507+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3dc10a9d0 con 0x7fa3cc0382e0 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.511+0000 7fa3d9ffb700 1 -- 172.21.15.31:0/2434950865 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa3dc10a9d0 con 0x7fa3cc0382e0 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa3cc0382e0 msgr2=0x7fa3cc03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa3cc0382e0 0x7fa3cc03a7a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 msgr2=0x7fa3dc19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc19a270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011670 tx=0x7fa3d0011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 shutdown_connections 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa3cc0382e0 0x7fa3cc03a7a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 --2- 172.21.15.31:0/2434950865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3dc104840 0x7fa3dc19a270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 >> 172.21.15.31:0/2434950865 conn(0x7fa3dc100170 msgr2=0x7fa3dc101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:20.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 shutdown_connections 2024-04-22T23:36:20.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:19.513+0000 7fa3e2c31700 1 -- 172.21.15.31:0/2434950865 wait complete. 2024-04-22T23:36:20.445 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying crash service with default placement... 2024-04-22T23:36:20.566 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:20 smithi031 ceph-mon[26721]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:20.566 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:20 smithi031 ceph-mon[26721]: Saving service mgr spec with placement count:2 2024-04-22T23:36:20.566 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:20 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:21.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-22T23:36:21.688 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.845+0000 7fbbab526700 1 Processor -- start 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.845+0000 7fbbab526700 1 -- start start 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.845+0000 7fbbab526700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba41093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.845+0000 7fbbab526700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4079e50 con 0x7fbba4107000 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbba92c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba41093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbba92c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba41093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55420/0 (socket says 172.21.15.31:55420) 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbba92c2700 1 -- 172.21.15.31:0/159551944 learned_addr learned my addr 172.21.15.31:0/159551944 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbba92c2700 1 -- 172.21.15.31:0/159551944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba4109930 con 0x7fbba4107000 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbba92c2700 1 --2- 172.21.15.31:0/159551944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba41093f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbba00098a0 tx=0x7fbba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbc7294871d89c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbb9bfff700 1 -- 172.21.15.31:0/159551944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbba0004180 con 0x7fbba4107000 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbb9bfff700 1 -- 172.21.15.31:0/159551944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbba00042e0 con 0x7fbba4107000 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.846+0000 7fbb9bfff700 1 -- 172.21.15.31:0/159551944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbba0011af0 con 0x7fbba4107000 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 -- 172.21.15.31:0/159551944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 msgr2=0x7fbba41093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 --2- 172.21.15.31:0/159551944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba41093f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbba00098a0 tx=0x7fbba0009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 -- 172.21.15.31:0/159551944 shutdown_connections 2024-04-22T23:36:21.689 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 --2- 172.21.15.31:0/159551944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba41093f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 -- 172.21.15.31:0/159551944 >> 172.21.15.31:0/159551944 conn(0x7fbba41009b0 msgr2=0x7fbba4102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 -- 172.21.15.31:0/159551944 shutdown_connections 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.847+0000 7fbbab526700 1 -- 172.21.15.31:0/159551944 wait complete. 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbbab526700 1 Processor -- start 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbbab526700 1 -- start start 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbbab526700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba4106960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbbab526700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4106ea0 con 0x7fbba4107000 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbba92c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba4106960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbba92c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba4106960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55422/0 (socket says 172.21.15.31:55422) 2024-04-22T23:36:21.690 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbba92c2700 1 -- 172.21.15.31:0/3214591210 learned_addr learned my addr 172.21.15.31:0/3214591210 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbba92c2700 1 -- 172.21.15.31:0/3214591210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009550 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.848+0000 7fbba92c2700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba4106960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbba0011670 tx=0x7fbba0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.849+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbba0003a40 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.849+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbba0003ba0 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.849+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba4104fb0 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.849+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbba0024820 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.849+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba4105450 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.849+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fbba0024a90 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.850+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba410a6c0 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.850+0000 7fbb9a7fc700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbb90038330 0x7fbb9003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.850+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbba004c4c0 con 0x7fbba4107000 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.850+0000 7fbba8ac1700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbb90038330 0x7fbb9003a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.850+0000 7fbba8ac1700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbb90038330 0x7fbb9003a7f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbb94006fd0 tx=0x7fbb94006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:21.691 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:20.855+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbba0024d40 con 0x7fbba4107000 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.119+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fbba4101e90 con 0x7fbb90038330 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.124+0000 7fbb9a7fc700 1 -- 172.21.15.31:0/3214591210 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fbba4101e90 con 0x7fbb90038330 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbb90038330 msgr2=0x7fbb9003a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbb90038330 0x7fbb9003a7f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbb94006fd0 tx=0x7fbb94006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 msgr2=0x7fbba4106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba4106960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbba0011670 tx=0x7fbba0011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 shutdown_connections 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbb90038330 0x7fbb9003a7f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 --2- 172.21.15.31:0/3214591210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbba4107000 0x7fbba4106960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 >> 172.21.15.31:0/3214591210 conn(0x7fbba41009b0 msgr2=0x7fbba4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 shutdown_connections 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:21.126+0000 7fbbab526700 1 -- 172.21.15.31:0/3214591210 wait complete. 2024-04-22T23:36:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying ceph-exporter service with default placement... 2024-04-22T23:36:21.797 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:21 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:21.798 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:21 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:21.798 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:21 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:22.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:22 smithi031 ceph-mon[26721]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:22.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:22 smithi031 ceph-mon[26721]: Saving service crash spec with placement * 2024-04-22T23:36:22.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:22 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:23.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:23 smithi031 ceph-mon[26721]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:23.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:23 smithi031 ceph-mon[26721]: Saving service ceph-exporter spec with placement * 2024-04-22T23:36:23.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.109+0000 7f1808efa700 1 Processor -- start 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.109+0000 7f1808efa700 1 -- start start 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.109+0000 7f1808efa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f1804106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.109+0000 7f1808efa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1804106f50 con 0x7f1804106560 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.109+0000 7f180259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f1804106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.109+0000 7f180259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f1804106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55436/0 (socket says 172.21.15.31:55436) 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.110+0000 7f180259c700 1 -- 172.21.15.31:0/690869832 learned_addr learned my addr 172.21.15.31:0/690869832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.110+0000 7f180259c700 1 -- 172.21.15.31:0/690869832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804107760 con 0x7f1804106560 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.110+0000 7f180259c700 1 --2- 172.21.15.31:0/690869832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f1804106980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17f80098a0 tx=0x7f17f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de3ab3aa47016f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.110+0000 7f180159a700 1 -- 172.21.15.31:0/690869832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17f8004180 con 0x7f1804106560 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.110+0000 7f180159a700 1 -- 172.21.15.31:0/690869832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f17f80042e0 con 0x7f1804106560 2024-04-22T23:36:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.110+0000 7f180159a700 1 -- 172.21.15.31:0/690869832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17f8011af0 con 0x7f1804106560 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 -- 172.21.15.31:0/690869832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 msgr2=0x7f1804106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 --2- 172.21.15.31:0/690869832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f1804106980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17f80098a0 tx=0x7f17f8009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 -- 172.21.15.31:0/690869832 shutdown_connections 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 --2- 172.21.15.31:0/690869832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f1804106980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 -- 172.21.15.31:0/690869832 >> 172.21.15.31:0/690869832 conn(0x7f1804101b40 msgr2=0x7f1804103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 -- 172.21.15.31:0/690869832 shutdown_connections 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 -- 172.21.15.31:0/690869832 wait complete. 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.111+0000 7f1808efa700 1 Processor -- start 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f1808efa700 1 -- start start 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f1808efa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f180419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f1808efa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180419a780 con 0x7f1804106560 2024-04-22T23:36:23.738 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f180259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f180419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f180259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f180419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55440/0 (socket says 172.21.15.31:55440) 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f180259c700 1 -- 172.21.15.31:0/1056514364 learned_addr learned my addr 172.21.15.31:0/1056514364 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f180259c700 1 -- 172.21.15.31:0/1056514364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f8009550 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f180259c700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f180419a240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17f8011670 tx=0x7f17f8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17f80039f0 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f17f8003b50 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.112+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180419a980 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.113+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f17f80247c0 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.113+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180419ae20 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.114+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f17f802d3f0 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.114+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18040623c0 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.114+0000 7f17f37fe700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f17ec038370 0x7f17ec03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.114+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f17f804c540 con 0x7f1804106560 2024-04-22T23:36:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.114+0000 7f1801d9b700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f17ec038370 0x7f17ec03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.114+0000 7f1801d9b700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f17ec038370 0x7f17ec03a830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f17f4006fd0 tx=0x7f17f4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.119+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f17f801ac00 con 0x7f1804106560 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.341+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f180410a960 con 0x7f17ec038370 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.345+0000 7f17f37fe700 1 -- 172.21.15.31:0/1056514364 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f180410a960 con 0x7f17ec038370 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f17ec038370 msgr2=0x7f17ec03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f17ec038370 0x7f17ec03a830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f17f4006fd0 tx=0x7f17f4006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 msgr2=0x7f180419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f180419a240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17f8011670 tx=0x7f17f8011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 shutdown_connections 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f17ec038370 0x7f17ec03a830 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 --2- 172.21.15.31:0/1056514364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1804106560 0x7f180419a240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 >> 172.21.15.31:0/1056514364 conn(0x7f1804101b40 msgr2=0x7f1804102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 shutdown_connections 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:22.347+0000 7f1808efa700 1 -- 172.21.15.31:0/1056514364 wait complete. 2024-04-22T23:36:23.740 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying prometheus service with default placement... 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.155+0000 7f92f51a1700 1 Processor -- start 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.155+0000 7f92f51a1700 1 -- start start 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.155+0000 7f92f51a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f01067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.155+0000 7f92f51a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f0106d90 con 0x7f92f01063a0 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.155+0000 7f92effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f01067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.155+0000 7f92effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f01067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55448/0 (socket says 172.21.15.31:55448) 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.156+0000 7f92effff700 1 -- 172.21.15.31:0/2778852727 learned_addr learned my addr 172.21.15.31:0/2778852727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.156+0000 7f92effff700 1 -- 172.21.15.31:0/2778852727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f01075a0 con 0x7f92f01063a0 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.156+0000 7f92effff700 1 --2- 172.21.15.31:0/2778852727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f01067c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f92e00098a0 tx=0x7f92e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86da39daa6b1de87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.156+0000 7f92eeffd700 1 -- 172.21.15.31:0/2778852727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92e0004180 con 0x7f92f01063a0 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.156+0000 7f92eeffd700 1 -- 172.21.15.31:0/2778852727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f92e00042e0 con 0x7f92f01063a0 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.156+0000 7f92eeffd700 1 -- 172.21.15.31:0/2778852727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92e0011af0 con 0x7f92f01063a0 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 -- 172.21.15.31:0/2778852727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 msgr2=0x7f92f01067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 --2- 172.21.15.31:0/2778852727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f01067c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f92e00098a0 tx=0x7f92e0009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 -- 172.21.15.31:0/2778852727 shutdown_connections 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 --2- 172.21.15.31:0/2778852727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f01067c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 -- 172.21.15.31:0/2778852727 >> 172.21.15.31:0/2778852727 conn(0x7f92f0101980 msgr2=0x7f92f0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 -- 172.21.15.31:0/2778852727 shutdown_connections 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 -- 172.21.15.31:0/2778852727 wait complete. 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.157+0000 7f92f51a1700 1 Processor -- start 2024-04-22T23:36:24.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92f51a1700 1 -- start start 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92f51a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f007bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92f51a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f007c170 con 0x7f92f01063a0 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f007bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92effff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f007bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55450/0 (socket says 172.21.15.31:55450) 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92effff700 1 -- 172.21.15.31:0/1262391881 learned_addr learned my addr 172.21.15.31:0/1262391881 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.158+0000 7f92effff700 1 -- 172.21.15.31:0/1262391881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009550 con 0x7f92f01063a0 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.159+0000 7f92effff700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f007bc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92e0011670 tx=0x7f92e0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.159+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92e00039f0 con 0x7f92f01063a0 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.159+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f92e0003b50 con 0x7f92f01063a0 2024-04-22T23:36:24.986 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.159+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f007a280 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.159+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92e00247c0 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.159+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f007a720 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.160+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f92e002d3f0 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.160+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f00623c0 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.160+0000 7f92ed7fa700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f92d8038370 0x7f92d803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.160+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f92e004c4d0 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.160+0000 7f92ef7fe700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f92d8038370 0x7f92d803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.160+0000 7f92ef7fe700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f92d8038370 0x7f92d803a830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f92e4006fd0 tx=0x7f92e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.165+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f92e0010970 con 0x7f92f01063a0 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.378+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f92f010a7a0 con 0x7f92d8038370 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.382+0000 7f92ed7fa700 1 -- 172.21.15.31:0/1262391881 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f92f010a7a0 con 0x7f92d8038370 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f92d8038370 msgr2=0x7f92d803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f92d8038370 0x7f92d803a830 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f92e4006fd0 tx=0x7f92e4006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:24.987 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 msgr2=0x7f92f007bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f007bc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f92e0011670 tx=0x7f92e0011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 shutdown_connections 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f92d8038370 0x7f92d803a830 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 --2- 172.21.15.31:0/1262391881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92f01063a0 0x7f92f007bc30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 >> 172.21.15.31:0/1262391881 conn(0x7f92f0101980 msgr2=0x7f92f0102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 shutdown_connections 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:24.384+0000 7f92f51a1700 1 -- 172.21.15.31:0/1262391881 wait complete. 2024-04-22T23:36:24.988 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying grafana service with default placement... 2024-04-22T23:36:25.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:25 smithi031 ceph-mon[26721]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:25.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:25 smithi031 ceph-mon[26721]: Saving service prometheus spec with placement count:1 2024-04-22T23:36:25.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:25 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.405+0000 7f57b472e700 1 Processor -- start 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.405+0000 7f57b472e700 1 -- start start 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b472e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b472e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac079e30 con 0x7f57ac104e10 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b24ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b24ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55462/0 (socket says 172.21.15.31:55462) 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b24ca700 1 -- 172.21.15.31:0/3939701910 learned_addr learned my addr 172.21.15.31:0/3939701910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b24ca700 1 -- 172.21.15.31:0/3939701910 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac105770 con 0x7f57ac104e10 2024-04-22T23:36:26.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.406+0000 7f57b24ca700 1 --2- 172.21.15.31:0/3939701910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac105230 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f579c0098a0 tx=0x7f579c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75db606b53c2a2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b14c8700 1 -- 172.21.15.31:0/3939701910 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f579c004180 con 0x7f57ac104e10 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b14c8700 1 -- 172.21.15.31:0/3939701910 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f579c0042e0 con 0x7f57ac104e10 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b14c8700 1 -- 172.21.15.31:0/3939701910 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f579c011af0 con 0x7f57ac104e10 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 -- 172.21.15.31:0/3939701910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 msgr2=0x7f57ac105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 --2- 172.21.15.31:0/3939701910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac105230 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f579c0098a0 tx=0x7f579c009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 -- 172.21.15.31:0/3939701910 shutdown_connections 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 --2- 172.21.15.31:0/3939701910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac105230 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 -- 172.21.15.31:0/3939701910 >> 172.21.15.31:0/3939701910 conn(0x7f57ac1009f0 msgr2=0x7f57ac102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 -- 172.21.15.31:0/3939701910 shutdown_connections 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.407+0000 7f57b472e700 1 -- 172.21.15.31:0/3939701910 wait complete. 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b472e700 1 Processor -- start 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b472e700 1 -- start start 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b472e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:26.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b472e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac19c980 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b24ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b24ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55476/0 (socket says 172.21.15.31:55476) 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.408+0000 7f57b24ca700 1 -- 172.21.15.31:0/1602622507 learned_addr learned my addr 172.21.15.31:0/1602622507 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57b24ca700 1 -- 172.21.15.31:0/1602622507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009550 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57b24ca700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac19c440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f579c011670 tx=0x7f579c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f579c003a40 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f579c003ba0 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac19cb80 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f579c024820 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.409+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac19d020 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.410+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f579c01a430 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.410+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac195d20 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.410+0000 7f57a37fe700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f5798038330 0x7f579803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.410+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f579c04c290 con 0x7f57ac104e10 2024-04-22T23:36:26.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.410+0000 7f57b1cc9700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f5798038330 0x7f579803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.411+0000 7f57b1cc9700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f5798038330 0x7f579803a7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57a8006fd0 tx=0x7f57a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.415+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f579c01a6e0 con 0x7f57ac104e10 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.628+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f57ac0611d0 con 0x7f5798038330 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.633+0000 7f57a37fe700 1 -- 172.21.15.31:0/1602622507 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f57ac0611d0 con 0x7f5798038330 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f5798038330 msgr2=0x7f579803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f5798038330 0x7f579803a7f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57a8006fd0 tx=0x7f57a8006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 msgr2=0x7f57ac19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac19c440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f579c011670 tx=0x7f579c011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 shutdown_connections 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f5798038330 0x7f579803a7f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 --2- 172.21.15.31:0/1602622507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ac104e10 0x7f57ac19c440 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 >> 172.21.15.31:0/1602622507 conn(0x7f57ac1009f0 msgr2=0x7f57ac1909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 shutdown_connections 2024-04-22T23:36:26.210 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:25.635+0000 7f57b472e700 1 -- 172.21.15.31:0/1602622507 wait complete. 2024-04-22T23:36:26.211 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying node-exporter service with default placement... 2024-04-22T23:36:26.834 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:26 smithi031 ceph-mon[26721]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:26.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:26 smithi031 ceph-mon[26721]: Saving service grafana spec with placement count:1 2024-04-22T23:36:26.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:26 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:26.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:26 smithi031 ceph-mon[26721]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa06dd86700 1 Processor -- start 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa06dd86700 1 -- start start 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa06dd86700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa068106e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa06dd86700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068107410 con 0x7fa068106a20 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa068106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa068106e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55480/0 (socket says 172.21.15.31:55480) 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.621+0000 7fa0677fe700 1 -- 172.21.15.31:0/2802629614 learned_addr learned my addr 172.21.15.31:0/2802629614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.622+0000 7fa0677fe700 1 -- 172.21.15.31:0/2802629614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068107550 con 0x7fa068106a20 2024-04-22T23:36:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.622+0000 7fa0677fe700 1 --2- 172.21.15.31:0/2802629614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa068106e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098a0 tx=0x7fa058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606012338517487d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.622+0000 7fa0667fc700 1 -- 172.21.15.31:0/2802629614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa058004180 con 0x7fa068106a20 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.622+0000 7fa0667fc700 1 -- 172.21.15.31:0/2802629614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa0580042e0 con 0x7fa068106a20 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.622+0000 7fa0667fc700 1 -- 172.21.15.31:0/2802629614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa058011af0 con 0x7fa068106a20 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 -- 172.21.15.31:0/2802629614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 msgr2=0x7fa068106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 --2- 172.21.15.31:0/2802629614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa068106e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0580098a0 tx=0x7fa058009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 -- 172.21.15.31:0/2802629614 shutdown_connections 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 --2- 172.21.15.31:0/2802629614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa068106e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 -- 172.21.15.31:0/2802629614 >> 172.21.15.31:0/2802629614 conn(0x7fa068100190 msgr2=0x7fa0681025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 -- 172.21.15.31:0/2802629614 shutdown_connections 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 -- 172.21.15.31:0/2802629614 wait complete. 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.623+0000 7fa06dd86700 1 Processor -- start 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa06dd86700 1 -- start start 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa06dd86700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa06819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa06dd86700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06819c970 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa06819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa06819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55496/0 (socket says 172.21.15.31:55496) 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa0677fe700 1 -- 172.21.15.31:0/1456372491 learned_addr learned my addr 172.21.15.31:0/1456372491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa0677fe700 1 -- 172.21.15.31:0/1456372491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa058009550 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.624+0000 7fa0677fe700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa06819c430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa058011670 tx=0x7fa058011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.625+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa058003a40 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.625+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06819cb70 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.625+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa058003ba0 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.625+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa058024790 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.625+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06819d010 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.625+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa05802d3f0 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.626+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa068196110 con 0x7fa068106a20 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.626+0000 7fa064ff9700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa050038320 0x7fa05003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.626+0000 7fa066ffd700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa050038320 0x7fa05003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.626+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa05804c6c0 con 0x7fa068106a20 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.626+0000 7fa066ffd700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa050038320 0x7fa05003a7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa05c006fd0 tx=0x7fa05c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.631+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa058010970 con 0x7fa068106a20 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.840+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0680611d0 con 0x7fa050038320 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.846+0000 7fa064ff9700 1 -- 172.21.15.31:0/1456372491 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fa0680611d0 con 0x7fa050038320 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.848+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa050038320 msgr2=0x7fa05003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.848+0000 7fa06dd86700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa050038320 0x7fa05003a7e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa05c006fd0 tx=0x7fa05c006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.848+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 msgr2=0x7fa06819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.848+0000 7fa06dd86700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa06819c430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa058011670 tx=0x7fa058011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.849+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 shutdown_connections 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.849+0000 7fa06dd86700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fa050038320 0x7fa05003a7e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.849+0000 7fa06dd86700 1 --2- 172.21.15.31:0/1456372491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa068106a20 0x7fa06819c430 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.849+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 >> 172.21.15.31:0/1456372491 conn(0x7fa068100190 msgr2=0x7fa068193250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.849+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 shutdown_connections 2024-04-22T23:36:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:26.849+0000 7fa06dd86700 1 -- 172.21.15.31:0/1456372491 wait complete. 2024-04-22T23:36:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying alertmanager service with default placement... 2024-04-22T23:36:27.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:27 smithi031 ceph-mon[26721]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:27.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:27 smithi031 ceph-mon[26721]: Saving service node-exporter spec with placement * 2024-04-22T23:36:27.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:27 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.839+0000 7fbe86aa6700 1 Processor -- start 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.839+0000 7fbe86aa6700 1 -- start start 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.839+0000 7fbe86aa6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe801093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.839+0000 7fbe86aa6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80079e90 con 0x7fbe80106fd0 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe84842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe801093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe84842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe801093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55506/0 (socket says 172.21.15.31:55506) 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe84842700 1 -- 172.21.15.31:0/56608899 learned_addr learned my addr 172.21.15.31:0/56608899 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:28.636 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe84842700 1 -- 172.21.15.31:0/56608899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80109900 con 0x7fbe80106fd0 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe84842700 1 --2- 172.21.15.31:0/56608899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe801093c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe700098a0 tx=0x7fbe70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aeebab652bacb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe7f7fe700 1 -- 172.21.15.31:0/56608899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe70004180 con 0x7fbe80106fd0 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe7f7fe700 1 -- 172.21.15.31:0/56608899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe700042e0 con 0x7fbe80106fd0 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.840+0000 7fbe7f7fe700 1 -- 172.21.15.31:0/56608899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe70011af0 con 0x7fbe80106fd0 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 -- 172.21.15.31:0/56608899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 msgr2=0x7fbe801093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 --2- 172.21.15.31:0/56608899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe801093c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe700098a0 tx=0x7fbe70009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 -- 172.21.15.31:0/56608899 shutdown_connections 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 --2- 172.21.15.31:0/56608899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe801093c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 -- 172.21.15.31:0/56608899 >> 172.21.15.31:0/56608899 conn(0x7fbe80100a20 msgr2=0x7fbe80102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 -- 172.21.15.31:0/56608899 shutdown_connections 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.841+0000 7fbe86aa6700 1 -- 172.21.15.31:0/56608899 wait complete. 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe86aa6700 1 Processor -- start 2024-04-22T23:36:28.637 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe86aa6700 1 -- start start 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe86aa6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe80106930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe86aa6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80106e70 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe84842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe80106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe84842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe80106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55516/0 (socket says 172.21.15.31:55516) 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe84842700 1 -- 172.21.15.31:0/1170661859 learned_addr learned my addr 172.21.15.31:0/1170661859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe84842700 1 -- 172.21.15.31:0/1170661859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe70009550 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe84842700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe80106930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe70011670 tx=0x7fbe70011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.842+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe70003a40 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.843+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe70003ba0 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.843+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe80104f80 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.843+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe70024820 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.843+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe80105420 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.844+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fbe7001a430 con 0x7fbe80106fd0 2024-04-22T23:36:28.638 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.844+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8010a6f0 con 0x7fbe80106fd0 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.844+0000 7fbe7dffb700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe68038330 0x7fbe6803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.844+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbe7004c290 con 0x7fbe80106fd0 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.844+0000 7fbe7ffff700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe68038330 0x7fbe6803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.845+0000 7fbe7ffff700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe68038330 0x7fbe6803a7f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006fd0 tx=0x7fbe74006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:27.849+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbe7001a6e0 con 0x7fbe80106fd0 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.062+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe80101ec0 con 0x7fbe68038330 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.070+0000 7fbe7dffb700 1 -- 172.21.15.31:0/1170661859 <== mgr.14118 v2:172.21.15.31:6800/863972114 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fbe80101ec0 con 0x7fbe68038330 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe68038330 msgr2=0x7fbe6803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe68038330 0x7fbe6803a7f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbe74006fd0 tx=0x7fbe74006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 msgr2=0x7fbe80106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe80106930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe70011670 tx=0x7fbe70011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 shutdown_connections 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fbe68038330 0x7fbe6803a7f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:28.639 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 --2- 172.21.15.31:0/1170661859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80106fd0 0x7fbe80106930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:28.640 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 >> 172.21.15.31:0/1170661859 conn(0x7fbe80100a20 msgr2=0x7fbe801017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:28.640 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 shutdown_connections 2024-04-22T23:36:28.640 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:28.072+0000 7fbe86aa6700 1 -- 172.21.15.31:0/1170661859 wait complete. 2024-04-22T23:36:29.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:29 smithi031 ceph-mon[26721]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:29.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:29 smithi031 ceph-mon[26721]: Saving service alertmanager spec with placement count:1 2024-04-22T23:36:29.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:29 smithi031 ceph-mon[26721]: from='mgr.14118 172.21.15.31:0/1573865798' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:29.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:29 smithi031 ceph-mon[26721]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:36:29.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a3383700 1 Processor -- start 2024-04-22T23:36:29.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a3383700 1 -- start start 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a3383700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a3383700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c106f50 con 0x7f039c106560 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a111f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a111f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55524/0 (socket says 172.21.15.31:55524) 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.029+0000 7f03a111f700 1 -- 172.21.15.31:0/1378302460 learned_addr learned my addr 172.21.15.31:0/1378302460 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a111f700 1 -- 172.21.15.31:0/1378302460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c107760 con 0x7f039c106560 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a111f700 1 --2- 172.21.15.31:0/1378302460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c106980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03980098a0 tx=0x7f0398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5054e47eab30f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f0393fff700 1 -- 172.21.15.31:0/1378302460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0398004180 con 0x7f039c106560 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f0393fff700 1 -- 172.21.15.31:0/1378302460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03980042e0 con 0x7f039c106560 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f0393fff700 1 -- 172.21.15.31:0/1378302460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0398011af0 con 0x7f039c106560 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a3383700 1 -- 172.21.15.31:0/1378302460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 msgr2=0x7f039c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a3383700 1 --2- 172.21.15.31:0/1378302460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c106980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03980098a0 tx=0x7f0398009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a3383700 1 -- 172.21.15.31:0/1378302460 shutdown_connections 2024-04-22T23:36:29.817 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a3383700 1 --2- 172.21.15.31:0/1378302460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c106980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.030+0000 7f03a3383700 1 -- 172.21.15.31:0/1378302460 >> 172.21.15.31:0/1378302460 conn(0x7f039c101b40 msgr2=0x7f039c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a3383700 1 -- 172.21.15.31:0/1378302460 shutdown_connections 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a3383700 1 -- 172.21.15.31:0/1378302460 wait complete. 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a3383700 1 Processor -- start 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a3383700 1 -- start start 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a3383700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a3383700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c19c980 con 0x7f039c106560 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.031+0000 7f03a111f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03a111f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55530/0 (socket says 172.21.15.31:55530) 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03a111f700 1 -- 172.21.15.31:0/3992808301 learned_addr learned my addr 172.21.15.31:0/3992808301 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03a111f700 1 -- 172.21.15.31:0/3992808301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0398009550 con 0x7f039c106560 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03a111f700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c19c440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0398011670 tx=0x7f0398011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03980039c0 con 0x7f039c106560 2024-04-22T23:36:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0398003b20 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03980247f0 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c19cb80 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.032+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c19d020 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.033+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f039801ac70 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.033+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c0623c0 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.033+0000 7f03927fc700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f03880382e0 0x7f038803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.033+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f039804c290 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.033+0000 7f03a091e700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f03880382e0 0x7f038803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.034+0000 7f03a091e700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f03880382e0 0x7f038803a7a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.038+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0398018760 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.236+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f039c19ff20 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.241+0000 7f03927fc700 1 -- 172.21.15.31:0/3992808301 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f03980172b0 con 0x7f039c106560 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.243+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f03880382e0 msgr2=0x7f038803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.243+0000 7f03a3383700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f03880382e0 0x7f038803a7a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f038c006fd0 tx=0x7f038c006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.243+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 msgr2=0x7f039c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.243+0000 7f03a3383700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c19c440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0398011670 tx=0x7f0398011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.244+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 shutdown_connections 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.244+0000 7f03a3383700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f03880382e0 0x7f038803a7a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.244+0000 7f03a3383700 1 --2- 172.21.15.31:0/3992808301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f039c106560 0x7f039c19c440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.244+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 >> 172.21.15.31:0/3992808301 conn(0x7f039c101b40 msgr2=0x7f039c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.244+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 shutdown_connections 2024-04-22T23:36:29.820 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:29.244+0000 7f03a3383700 1 -- 172.21.15.31:0/3992808301 wait complete. 2024-04-22T23:36:30.477 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:30 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3992808301' entity='client.admin' 2024-04-22T23:36:31.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae6c89700 1 Processor -- start 2024-04-22T23:36:31.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae6c89700 1 -- start start 2024-04-22T23:36:31.006 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae6c89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae0109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae6c89700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0079e10 con 0x7f9ae0107040 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae4a25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae0109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae4a25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae0109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55542/0 (socket says 172.21.15.31:55542) 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae4a25700 1 -- 172.21.15.31:0/1878762527 learned_addr learned my addr 172.21.15.31:0/1878762527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.275+0000 7f9ae4a25700 1 -- 172.21.15.31:0/1878762527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0109970 con 0x7f9ae0107040 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae4a25700 1 --2- 172.21.15.31:0/1878762527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae0109430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40098a0 tx=0x7f9ad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c4f138e65ea1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9adf7fe700 1 -- 172.21.15.31:0/1878762527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ad4004180 con 0x7f9ae0107040 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9adf7fe700 1 -- 172.21.15.31:0/1878762527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ad40042e0 con 0x7f9ae0107040 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9adf7fe700 1 -- 172.21.15.31:0/1878762527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ad4011af0 con 0x7f9ae0107040 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 -- 172.21.15.31:0/1878762527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 msgr2=0x7f9ae0109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 --2- 172.21.15.31:0/1878762527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae0109430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40098a0 tx=0x7f9ad4009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 -- 172.21.15.31:0/1878762527 shutdown_connections 2024-04-22T23:36:31.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 --2- 172.21.15.31:0/1878762527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae0109430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 -- 172.21.15.31:0/1878762527 >> 172.21.15.31:0/1878762527 conn(0x7f9ae01009f0 msgr2=0x7f9ae0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 -- 172.21.15.31:0/1878762527 shutdown_connections 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.276+0000 7f9ae6c89700 1 -- 172.21.15.31:0/1878762527 wait complete. 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae6c89700 1 Processor -- start 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae6c89700 1 -- start start 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae6c89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae01069a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae6c89700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0106ee0 con 0x7f9ae0107040 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae4a25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae01069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae4a25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae01069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55554/0 (socket says 172.21.15.31:55554) 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.277+0000 7f9ae4a25700 1 -- 172.21.15.31:0/4216187509 learned_addr learned my addr 172.21.15.31:0/4216187509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9ae4a25700 1 -- 172.21.15.31:0/4216187509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009550 con 0x7f9ae0107040 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9ae4a25700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae01069a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4011670 tx=0x7f9ad4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ad4003a40 con 0x7f9ae0107040 2024-04-22T23:36:31.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ad4003ba0 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae0104ff0 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ad4024820 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.278+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae0105490 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.279+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9ad4024a90 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.279+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae0062430 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.279+0000 7f9addffb700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f9ad0038330 0x7f9ad003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.279+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ad404c290 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.279+0000 7f9adffff700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f9ad0038330 0x7f9ad003a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.280+0000 7f9adffff700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f9ad0038330 0x7f9ad003a7f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9acc006fd0 tx=0x7f9acc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.284+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9ae0062430 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.482+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f9ae0105ce0 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.486+0000 7f9addffb700 1 -- 172.21.15.31:0/4216187509 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f9ad4024d40 con 0x7f9ae0107040 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f9ad0038330 msgr2=0x7f9ad003a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:31.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f9ad0038330 0x7f9ad003a7f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9acc006fd0 tx=0x7f9acc006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 msgr2=0x7f9ae01069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae01069a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4011670 tx=0x7f9ad4011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 shutdown_connections 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7f9ad0038330 0x7f9ad003a7f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 --2- 172.21.15.31:0/4216187509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ae0107040 0x7f9ae01069a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 >> 172.21.15.31:0/4216187509 conn(0x7f9ae01009f0 msgr2=0x7f9ae01017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 shutdown_connections 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:30.488+0000 7f9ae6c89700 1 -- 172.21.15.31:0/4216187509 wait complete. 2024-04-22T23:36:31.010 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-04-22T23:36:31.689 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:31 smithi031 ceph-mon[26721]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:36:31.689 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:31 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4216187509' entity='client.admin' 2024-04-22T23:36:32.620 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:32 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1671396467' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-22T23:36:33.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc84117700 1 Processor -- start 2024-04-22T23:36:33.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc84117700 1 -- start start 2024-04-22T23:36:33.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc84117700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c105250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:33.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc84117700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c079e10 con 0x7efc7c104e30 2024-04-22T23:36:33.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc81eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c105250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:33.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc81eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c105250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42112/0 (socket says 172.21.15.31:42112) 2024-04-22T23:36:33.064 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.427+0000 7efc81eb3700 1 -- 172.21.15.31:0/264167609 learned_addr learned my addr 172.21.15.31:0/264167609 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc81eb3700 1 -- 172.21.15.31:0/264167609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7c105790 con 0x7efc7c104e30 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc81eb3700 1 --2- 172.21.15.31:0/264167609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c105250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc780098a0 tx=0x7efc78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a2cb38110db123 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc80eb1700 1 -- 172.21.15.31:0/264167609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc78004180 con 0x7efc7c104e30 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc80eb1700 1 -- 172.21.15.31:0/264167609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc780042e0 con 0x7efc7c104e30 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc80eb1700 1 -- 172.21.15.31:0/264167609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc78011af0 con 0x7efc7c104e30 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc84117700 1 -- 172.21.15.31:0/264167609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 msgr2=0x7efc7c105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.428+0000 7efc84117700 1 --2- 172.21.15.31:0/264167609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c105250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc780098a0 tx=0x7efc78009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 -- 172.21.15.31:0/264167609 shutdown_connections 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 --2- 172.21.15.31:0/264167609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c105250 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 -- 172.21.15.31:0/264167609 >> 172.21.15.31:0/264167609 conn(0x7efc7c1009b0 msgr2=0x7efc7c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 -- 172.21.15.31:0/264167609 shutdown_connections 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 -- 172.21.15.31:0/264167609 wait complete. 2024-04-22T23:36:33.065 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 Processor -- start 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 -- start start 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.429+0000 7efc84117700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c19c990 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc81eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc81eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42118/0 (socket says 172.21.15.31:42118) 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc81eb3700 1 -- 172.21.15.31:0/1671396467 learned_addr learned my addr 172.21.15.31:0/1671396467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc81eb3700 1 -- 172.21.15.31:0/1671396467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78009550 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc81eb3700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c19c450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efc78011670 tx=0x7efc78011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc78003a40 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc78003ba0 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c19cb90 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc78024790 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.430+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c19d030 con 0x7efc7c104e30 2024-04-22T23:36:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.431+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7efc7801a430 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.431+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c195d20 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.431+0000 7efc72ffd700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7efc68038370 0x7efc6803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.431+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efc7804c410 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.431+0000 7efc816b2700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7efc68038370 0x7efc6803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.432+0000 7efc816b2700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7efc68038370 0x7efc6803a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc6c006fd0 tx=0x7efc6c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.436+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efc78018ba0 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:31.694+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7efc7c02cc70 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.488+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7efc7802d770 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.489+0000 7efc72ffd700 1 -- 172.21.15.31:0/1671396467 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7efc78018dd0 con 0x7efc7c104e30 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7efc68038370 msgr2=0x7efc6803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7efc68038370 0x7efc6803a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc6c006fd0 tx=0x7efc6c006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 msgr2=0x7efc7c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c19c450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efc78011670 tx=0x7efc78011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 shutdown_connections 2024-04-22T23:36:33.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7efc68038370 0x7efc6803a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 --2- 172.21.15.31:0/1671396467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104e30 0x7efc7c19c450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.493+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 >> 172.21.15.31:0/1671396467 conn(0x7efc7c1009b0 msgr2=0x7efc7c1909f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:33.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.494+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 shutdown_connections 2024-04-22T23:36:33.068 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:32.494+0000 7efc84117700 1 -- 172.21.15.31:0/1671396467 wait complete. 2024-04-22T23:36:33.539 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:33 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1671396467' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-22T23:36:33.539 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:33 smithi031 ceph-mon[26721]: mgrmap e9: smithi031.fazdcv(active, since 26s) 2024-04-22T23:36:34.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:36:34.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-22T23:36:34.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T23:36:34.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "smithi031.fazdcv", 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.534+0000 7fea08a42700 1 Processor -- start 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.534+0000 7fea08a42700 1 -- start start 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.534+0000 7fea08a42700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea040791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.534+0000 7fea08a42700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040797a0 con 0x7fea0407ad70 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea040791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea040791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42150/0 (socket says 172.21.15.31:42150) 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0259c700 1 -- 172.21.15.31:0/1835236293 learned_addr learned my addr 172.21.15.31:0/1835236293 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0259c700 1 -- 172.21.15.31:0/1835236293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040798e0 con 0x7fea0407ad70 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0259c700 1 --2- 172.21.15.31:0/1835236293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea040791d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40098a0 tx=0x7fe9f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a64743ec88bf91 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0159a700 1 -- 172.21.15.31:0/1835236293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f4004180 con 0x7fea0407ad70 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0159a700 1 -- 172.21.15.31:0/1835236293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe9f40042e0 con 0x7fea0407ad70 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.535+0000 7fea0159a700 1 -- 172.21.15.31:0/1835236293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f4011af0 con 0x7fea0407ad70 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 -- 172.21.15.31:0/1835236293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 msgr2=0x7fea040791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 --2- 172.21.15.31:0/1835236293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea040791d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40098a0 tx=0x7fe9f4009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 -- 172.21.15.31:0/1835236293 shutdown_connections 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 --2- 172.21.15.31:0/1835236293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea040791d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 -- 172.21.15.31:0/1835236293 >> 172.21.15.31:0/1835236293 conn(0x7fea04101b20 msgr2=0x7fea04103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 -- 172.21.15.31:0/1835236293 shutdown_connections 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.536+0000 7fea08a42700 1 -- 172.21.15.31:0/1835236293 wait complete. 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea08a42700 1 Processor -- start 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea08a42700 1 -- start start 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea08a42700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea0419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea08a42700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0419c960 con 0x7fea0407ad70 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea0419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea0259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea0419c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42164/0 (socket says 172.21.15.31:42164) 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.537+0000 7fea0259c700 1 -- 172.21.15.31:0/2072477970 learned_addr learned my addr 172.21.15.31:0/2072477970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fea0259c700 1 -- 172.21.15.31:0/2072477970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f4009550 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fea0259c700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea0419c420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000c00 tx=0x7fe9f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f4003a70 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe9f4003bd0 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0419cb60 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe9f40238c0 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.538+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0419cf40 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.539+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fe9f4023b30 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.540+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea04062430 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.540+0000 7fe9f37fe700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 0x7fe9ec03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.540+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe9f404c1c0 con 0x7fea0407ad70 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.540+0000 7fea01d9b700 1 -- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 msgr2=0x7fe9ec03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.540+0000 7fea01d9b700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 0x7fe9ec03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:36:34.375 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.544+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fea04062430 con 0x7fea0407ad70 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.741+0000 7fea01d9b700 1 -- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 msgr2=0x7fe9ec03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.741+0000 7fea01d9b700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 0x7fe9ec03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.864+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fea040623c0 con 0x7fea0407ad70 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.864+0000 7fe9f37fe700 1 -- 172.21.15.31:0/2072477970 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fe9f4010b30 con 0x7fea0407ad70 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 msgr2=0x7fe9ec03a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 0x7fe9ec03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 msgr2=0x7fea0419c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea0419c420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4000c00 tx=0x7fe9f4011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 shutdown_connections 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe9ec0383c0 0x7fe9ec03a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 --2- 172.21.15.31:0/2072477970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea0407ad70 0x7fea0419c420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 >> 172.21.15.31:0/2072477970 conn(0x7fea04101b20 msgr2=0x7fea04102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 shutdown_connections 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:33.867+0000 7fea08a42700 1 -- 172.21.15.31:0/2072477970 wait complete. 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-22T23:36:34.376 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-04-22T23:36:34.784 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:34 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2072477970' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T23:36:39.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: Active manager daemon smithi031.fazdcv restarted 2024-04-22T23:36:39.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: osdmap e3: 0 total, 0 up, 0 in 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: mgrmap e10: smithi031.fazdcv(active, starting, since 0.00322509s) 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:39.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:39 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:36:40.652 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:40 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:36:40.652 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:40 smithi031 ceph-mon[26721]: mgrmap e11: smithi031.fazdcv(active, since 1.00546s) 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.789+0000 7fe1886ca700 1 Processor -- start 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.789+0000 7fe1886ca700 1 -- start start 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.789+0000 7fe1886ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe180104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.789+0000 7fe1886ca700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe180105250 con 0x7fe180104860 2024-04-22T23:36:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.790+0000 7fe186466700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe180104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.790+0000 7fe186466700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe180104c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42174/0 (socket says 172.21.15.31:42174) 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.790+0000 7fe186466700 1 -- 172.21.15.31:0/472966741 learned_addr learned my addr 172.21.15.31:0/472966741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.790+0000 7fe186466700 1 -- 172.21.15.31:0/472966741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1801077f0 con 0x7fe180104860 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.790+0000 7fe186466700 1 --2- 172.21.15.31:0/472966741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe180104c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098a0 tx=0x7fe174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cd32e15c5c79c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.790+0000 7fe185464700 1 -- 172.21.15.31:0/472966741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe174004180 con 0x7fe180104860 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe185464700 1 -- 172.21.15.31:0/472966741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1740042e0 con 0x7fe180104860 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe185464700 1 -- 172.21.15.31:0/472966741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe174011af0 con 0x7fe180104860 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 -- 172.21.15.31:0/472966741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 msgr2=0x7fe180104c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 --2- 172.21.15.31:0/472966741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe180104c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1740098a0 tx=0x7fe174009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 -- 172.21.15.31:0/472966741 shutdown_connections 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 --2- 172.21.15.31:0/472966741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe180104c80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 -- 172.21.15.31:0/472966741 >> 172.21.15.31:0/472966741 conn(0x7fe180100190 msgr2=0x7fe1801025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 -- 172.21.15.31:0/472966741 shutdown_connections 2024-04-22T23:36:41.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.791+0000 7fe1886ca700 1 -- 172.21.15.31:0/472966741 wait complete. 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe1886ca700 1 Processor -- start 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe1886ca700 1 -- start start 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe1886ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe18019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe1886ca700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019a770 con 0x7fe180104860 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe186466700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe18019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe186466700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe18019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42188/0 (socket says 172.21.15.31:42188) 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe186466700 1 -- 172.21.15.31:0/1976526625 learned_addr learned my addr 172.21.15.31:0/1976526625 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe186466700 1 -- 172.21.15.31:0/1976526625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009550 con 0x7fe180104860 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.792+0000 7fe186466700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe18019a230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe174006ba0 tx=0x7fe174011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.793+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe174003b00 con 0x7fe180104860 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.793+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18019a970 con 0x7fe180104860 2024-04-22T23:36:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.793+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe17401a8e0 con 0x7fe180104860 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.793+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe174019990 con 0x7fe180104860 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.793+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18019ae10 con 0x7fe180104860 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.793+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fe174019c50 con 0x7fe180104860 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.794+0000 7fe1737fe700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.794+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 --> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe16c03aef0 con 0x7fe16c038320 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.794+0000 7fe185c65700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 msgr2=0x7fe16c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.794+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe17404c750 con 0x7fe180104860 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.794+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.995+0000 7fe185c65700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 msgr2=0x7fe16c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:34.995+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:35.396+0000 7fe185c65700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 msgr2=0x7fe16c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:35.396+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T23:36:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:36.197+0000 7fe185c65700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 msgr2=0x7fe16c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:36.197+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:37.798+0000 7fe185c65700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 msgr2=0x7fe16c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/863972114 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:37.798+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:39.457+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7fe17401a5d0 con 0x7fe180104860 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:39.457+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 msgr2=0x7fe16c03a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:39.457+0000 7fe1737fe700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.458+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fe174018350 con 0x7fe180104860 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.459+0000 7fe1737fe700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fe16c03bd90 0x7fe16c03e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.459+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe16c03aef0 con 0x7fe16c03bd90 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.459+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fe16c03bd90 0x7fe16c03e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.460+0000 7fe185c65700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fe16c03bd90 0x7fe16c03e180 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe17c003a10 tx=0x7fe17c0092b0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.461+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fe16c03aef0 con 0x7fe16c03bd90 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.467+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe18019b570 con 0x7fe16c03bd90 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.467+0000 7fe1737fe700 1 -- 172.21.15.31:0/1976526625 <== mgr.14162 v2:172.21.15.31:6800/3579734707 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fe18019b570 con 0x7fe16c03bd90 2024-04-22T23:36:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.467+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fe16c03bd90 msgr2=0x7fe16c03e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.467+0000 7fe1886ca700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fe16c03bd90 0x7fe16c03e180 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe17c003a10 tx=0x7fe17c0092b0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 msgr2=0x7fe18019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe18019a230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe174006ba0 tx=0x7fe174011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 shutdown_connections 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fe16c03bd90 0x7fe16c03e180 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe180104860 0x7fe18019a230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 --2- 172.21.15.31:0/1976526625 >> [v2:172.21.15.31:6800/863972114,v1:172.21.15.31:6801/863972114] conn(0x7fe16c038320 0x7fe16c03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 >> 172.21.15.31:0/1976526625 conn(0x7fe180100190 msgr2=0x7fe180100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 shutdown_connections 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:40.468+0000 7fe1886ca700 1 -- 172.21.15.31:0/1976526625 wait complete. 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-04-22T23:36:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-04-22T23:36:41.657 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:41 smithi031 ceph-mon[26721]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T23:36:41.657 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:41 smithi031 ceph-mon[26721]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T23:36:41.657 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:41 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:42.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-22T23:36:42.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.708+0000 7fc9d9f7e700 1 Processor -- start 2024-04-22T23:36:42.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.708+0000 7fc9d9f7e700 1 -- start start 2024-04-22T23:36:42.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.708+0000 7fc9d9f7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d4105210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:42.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.708+0000 7fc9d9f7e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4079e30 con 0x7fc9d4104df0 2024-04-22T23:36:42.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d4105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d4105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49552/0 (socket says 172.21.15.31:49552) 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d37fe700 1 -- 172.21.15.31:0/303967985 learned_addr learned my addr 172.21.15.31:0/303967985 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d37fe700 1 -- 172.21.15.31:0/303967985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4105750 con 0x7fc9d4104df0 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d37fe700 1 --2- 172.21.15.31:0/303967985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d4105210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098a0 tx=0x7fc9cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ead1927320e2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d27fc700 1 -- 172.21.15.31:0/303967985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9cc004180 con 0x7fc9d4104df0 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d27fc700 1 -- 172.21.15.31:0/303967985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc9cc0042e0 con 0x7fc9d4104df0 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.709+0000 7fc9d27fc700 1 -- 172.21.15.31:0/303967985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9cc011af0 con 0x7fc9d4104df0 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/303967985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 msgr2=0x7fc9d4105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 --2- 172.21.15.31:0/303967985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d4105210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098a0 tx=0x7fc9cc009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/303967985 shutdown_connections 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 --2- 172.21.15.31:0/303967985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d4105210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/303967985 >> 172.21.15.31:0/303967985 conn(0x7fc9d41009b0 msgr2=0x7fc9d4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/303967985 shutdown_connections 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.710+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/303967985 wait complete. 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d9f7e700 1 Processor -- start 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d9f7e700 1 -- start start 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d9f7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d9f7e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d419c930 con 0x7fc9d4104df0 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49554/0 (socket says 172.21.15.31:49554) 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.711+0000 7fc9d37fe700 1 -- 172.21.15.31:0/3622475937 learned_addr learned my addr 172.21.15.31:0/3622475937 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d37fe700 1 -- 172.21.15.31:0/3622475937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009550 con 0x7fc9d4104df0 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d37fe700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d419c3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc006ba0 tx=0x7fc9cc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9cc003b00 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc9cc01a8e0 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d419cb30 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9cc0199c0 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.712+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d419cfd0 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.713+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fc9cc019c30 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.713+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d4195cd0 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.713+0000 7fc9d0ff9700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc9c0038240 0x7fc9c003a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.713+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc9cc04c500 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.714+0000 7fc9d2ffd700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc9c0038240 0x7fc9c003a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.714+0000 7fc9d2ffd700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc9c0038240 0x7fc9c003a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc006fd0 tx=0x7fc9bc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.719+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9cc02d440 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.940+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9d40611d0 con 0x7fc9c0038240 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:41.960+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fc9cc02c3a0 con 0x7fc9d4104df0 2024-04-22T23:36:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.040+0000 7fc9d0ff9700 1 -- 172.21.15.31:0/3622475937 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc9d40611d0 con 0x7fc9c0038240 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc9c0038240 msgr2=0x7fc9c003a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc9c0038240 0x7fc9c003a700 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc006fd0 tx=0x7fc9bc006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 msgr2=0x7fc9d419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d419c3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc006ba0 tx=0x7fc9cc011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 shutdown_connections 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc9c0038240 0x7fc9c003a700 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 --2- 172.21.15.31:0/3622475937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d4104df0 0x7fc9d419c3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 >> 172.21.15.31:0/3622475937 conn(0x7fc9d41009b0 msgr2=0x7fc9d41909b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 shutdown_connections 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:42.042+0000 7fc9d9f7e700 1 -- 172.21.15.31:0/3622475937 wait complete. 2024-04-22T23:36:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-04-22T23:36:42.686 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:40] ENGINE Bus STARTING 2024-04-22T23:36:42.686 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:40] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:36:42.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:40] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:36:42.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: [22/Apr/2024:23:36:40] ENGINE Bus STARTED 2024-04-22T23:36:42.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:42.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: mgrmap e12: smithi031.fazdcv(active, since 2s) 2024-04-22T23:36:42.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:42.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:42 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:43.812 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:43 smithi031 ceph-mon[26721]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:44.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$TfNO.wUSApUYvxtL2.wVZedAwjoPs7Zo18v01gJqaqxHRaIyqofO6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713829003, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-22T23:36:44.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.173+0000 7f5eece37700 1 Processor -- start 2024-04-22T23:36:44.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.173+0000 7f5eece37700 1 -- start start 2024-04-22T23:36:44.291 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.173+0000 7f5eece37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee8105d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.173+0000 7f5eece37700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee8106340 con 0x7f5ee8105950 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.173+0000 7f5ee659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee8105d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.173+0000 7f5ee659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee8105d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49562/0 (socket says 172.21.15.31:49562) 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.174+0000 7f5ee659c700 1 -- 172.21.15.31:0/2719041427 learned_addr learned my addr 172.21.15.31:0/2719041427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.174+0000 7f5ee659c700 1 -- 172.21.15.31:0/2719041427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8106670 con 0x7f5ee8105950 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.174+0000 7f5ee659c700 1 --2- 172.21.15.31:0/2719041427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee8105d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80098a0 tx=0x7f5ed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd5d5c422367941 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.174+0000 7f5ee559a700 1 -- 172.21.15.31:0/2719041427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed8004180 con 0x7f5ee8105950 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.174+0000 7f5ee559a700 1 -- 172.21.15.31:0/2719041427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ed80042e0 con 0x7f5ee8105950 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.174+0000 7f5ee559a700 1 -- 172.21.15.31:0/2719041427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed8011af0 con 0x7f5ee8105950 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 -- 172.21.15.31:0/2719041427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 msgr2=0x7f5ee8105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 --2- 172.21.15.31:0/2719041427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee8105d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80098a0 tx=0x7f5ed8009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 -- 172.21.15.31:0/2719041427 shutdown_connections 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 --2- 172.21.15.31:0/2719041427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee8105d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:44.292 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 -- 172.21.15.31:0/2719041427 >> 172.21.15.31:0/2719041427 conn(0x7f5ee8100170 msgr2=0x7f5ee81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 -- 172.21.15.31:0/2719041427 shutdown_connections 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.175+0000 7f5eece37700 1 -- 172.21.15.31:0/2719041427 wait complete. 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5eece37700 1 Processor -- start 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5eece37700 1 -- start start 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5eece37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee81906b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5eece37700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee8190bf0 con 0x7f5ee8105950 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ee659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee81906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ee659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee81906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49578/0 (socket says 172.21.15.31:49578) 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ee659c700 1 -- 172.21.15.31:0/2048537696 learned_addr learned my addr 172.21.15.31:0/2048537696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ee659c700 1 -- 172.21.15.31:0/2048537696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8009550 con 0x7f5ee8105950 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ee659c700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee81906b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8011670 tx=0x7f5ed8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed8003a40 con 0x7f5ee8105950 2024-04-22T23:36:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ed8003ba0 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee818ed00 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.176+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5ed8024820 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.177+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee818f1a0 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.177+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f5ed8024a90 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.177+0000 7f5ed77fe700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f5ed0038380 0x7f5ed003a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.178+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ed804c440 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.178+0000 7f5ee5d9b700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f5ed0038380 0x7f5ed003a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.178+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee80623c0 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.178+0000 7f5ee5d9b700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f5ed0038380 0x7f5ed003a840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5edc006fd0 tx=0x7f5edc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.183+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ed8024d90 con 0x7f5ee8105950 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.405+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5ee818fb80 con 0x7f5ed0038380 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.652+0000 7f5ed77fe700 1 -- 172.21.15.31:0/2048537696 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f5ee818fb80 con 0x7f5ed0038380 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f5ed0038380 msgr2=0x7f5ed003a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:44.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f5ed0038380 0x7f5ed003a840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5edc006fd0 tx=0x7f5edc006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 msgr2=0x7f5ee81906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee81906b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8011670 tx=0x7f5ed8011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 shutdown_connections 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f5ed0038380 0x7f5ed003a840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 --2- 172.21.15.31:0/2048537696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ee8105950 0x7f5ee81906b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 >> 172.21.15.31:0/2048537696 conn(0x7f5ee8100170 msgr2=0x7f5ee8104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 shutdown_connections 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:43.655+0000 7f5eece37700 1 -- 172.21.15.31:0/2048537696 wait complete. 2024-04-22T23:36:44.295 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-04-22T23:36:44.651 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:44 smithi031 ceph-mon[26721]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:44.651 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:44 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:45.475 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-04-22T23:36:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18feff8700 1 Processor -- start 2024-04-22T23:36:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18feff8700 1 -- start start 2024-04-22T23:36:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18feff8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f8106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18feff8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8106f50 con 0x7f18f8106560 2024-04-22T23:36:45.481 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18fcd94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f8106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18fcd94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f8106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49588/0 (socket says 172.21.15.31:49588) 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.762+0000 7f18fcd94700 1 -- 172.21.15.31:0/3571605082 learned_addr learned my addr 172.21.15.31:0/3571605082 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18fcd94700 1 -- 172.21.15.31:0/3571605082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8107760 con 0x7f18f8106560 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18fcd94700 1 --2- 172.21.15.31:0/3571605082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f8106980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098a0 tx=0x7f18ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b716f5a3ac63fef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18f77fe700 1 -- 172.21.15.31:0/3571605082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18ec004180 con 0x7f18f8106560 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18f77fe700 1 -- 172.21.15.31:0/3571605082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f18ec0042e0 con 0x7f18f8106560 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18f77fe700 1 -- 172.21.15.31:0/3571605082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18ec011af0 con 0x7f18f8106560 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18feff8700 1 -- 172.21.15.31:0/3571605082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 msgr2=0x7f18f8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.763+0000 7f18feff8700 1 --2- 172.21.15.31:0/3571605082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f8106980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0098a0 tx=0x7f18ec009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 -- 172.21.15.31:0/3571605082 shutdown_connections 2024-04-22T23:36:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 --2- 172.21.15.31:0/3571605082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f8106980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 -- 172.21.15.31:0/3571605082 >> 172.21.15.31:0/3571605082 conn(0x7f18f8101b40 msgr2=0x7f18f8103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 -- 172.21.15.31:0/3571605082 shutdown_connections 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 -- 172.21.15.31:0/3571605082 wait complete. 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 Processor -- start 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.764+0000 7f18feff8700 1 -- start start 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18feff8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18feff8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f819a780 con 0x7f18f8106560 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18fcd94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18fcd94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49598/0 (socket says 172.21.15.31:49598) 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18fcd94700 1 -- 172.21.15.31:0/3292812208 learned_addr learned my addr 172.21.15.31:0/3292812208 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18fcd94700 1 -- 172.21.15.31:0/3292812208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec009550 con 0x7f18f8106560 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18fcd94700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f819a240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f18ec011670 tx=0x7f18ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18ec0039c0 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f18ec003b20 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f819a980 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.765+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f18ec0247f0 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.766+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f819ae20 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.766+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f18ec024a60 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.766+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f80623c0 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.767+0000 7f18f5ffb700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f18e8038380 0x7f18e803a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.767+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f18ec04c500 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.767+0000 7f18f7fff700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f18e8038380 0x7f18e803a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.767+0000 7f18f7fff700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f18e8038380 0x7f18e803a840 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18e4006fd0 tx=0x7f18e4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.771+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f18ec022310 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.968+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f18f819dc60 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.969+0000 7f18f5ffb700 1 -- 172.21.15.31:0/3292812208 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18ec018b00 con 0x7f18f8106560 2024-04-22T23:36:45.484 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f18e8038380 msgr2=0x7f18e803a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f18e8038380 0x7f18e803a840 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18e4006fd0 tx=0x7f18e4006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 msgr2=0x7f18f819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f819a240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f18ec011670 tx=0x7f18ec011aa0 comp rx=0 tx=0).stop 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 shutdown_connections 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f18e8038380 0x7f18e803a840 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 --2- 172.21.15.31:0/3292812208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18f8106560 0x7f18f819a240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 >> 172.21.15.31:0/3292812208 conn(0x7f18f8101b40 msgr2=0x7f18f8102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 shutdown_connections 2024-04-22T23:36:45.485 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:44.971+0000 7f18feff8700 1 -- 172.21.15.31:0/3292812208 wait complete. 2024-04-22T23:36:45.489 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:36:45.489 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-22T23:36:45.497 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:36:45.497 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-22T23:36:45.497 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-22T23:36:45.497 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-22T23:36:45.498 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031:8443/ 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: Password: 4tzpqert1p 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config directory 2024-04-22T23:36:45.499 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2024-04-22T23:36:46.100 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:45 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3292812208' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-22T23:36:46.669 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f7acf700 1 Processor -- start 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f7acf700 1 -- start start 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f7acf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f01093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f7acf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0079e90 con 0x7f16f0106fc0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f586b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f01093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f586b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f01093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49612/0 (socket says 172.21.15.31:49612) 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.900+0000 7f16f586b700 1 -- 172.21.15.31:0/1590127401 learned_addr learned my addr 172.21.15.31:0/1590127401 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.901+0000 7f16f586b700 1 -- 172.21.15.31:0/1590127401 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f01098f0 con 0x7f16f0106fc0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.901+0000 7f16f586b700 1 --2- 172.21.15.31:0/1590127401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f01093b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098a0 tx=0x7f16ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edd131b4caf5c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.901+0000 7f16f4869700 1 -- 172.21.15.31:0/1590127401 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16ec004180 con 0x7f16f0106fc0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.901+0000 7f16f4869700 1 -- 172.21.15.31:0/1590127401 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f16ec0042e0 con 0x7f16f0106fc0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.901+0000 7f16f4869700 1 -- 172.21.15.31:0/1590127401 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16ec011af0 con 0x7f16f0106fc0 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.901+0000 7f16f7acf700 1 -- 172.21.15.31:0/1590127401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 msgr2=0x7f16f01093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:46.670 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 --2- 172.21.15.31:0/1590127401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f01093b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0098a0 tx=0x7f16ec009bb0 comp rx=0 tx=0).stop 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 -- 172.21.15.31:0/1590127401 shutdown_connections 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 --2- 172.21.15.31:0/1590127401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f01093b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 -- 172.21.15.31:0/1590127401 >> 172.21.15.31:0/1590127401 conn(0x7f16f01009b0 msgr2=0x7f16f0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 -- 172.21.15.31:0/1590127401 shutdown_connections 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 -- 172.21.15.31:0/1590127401 wait complete. 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 Processor -- start 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 -- start start 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.902+0000 7f16f7acf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f7acf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f019a740 con 0x7f16f0106fc0 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f586b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f586b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49626/0 (socket says 172.21.15.31:49626) 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f586b700 1 -- 172.21.15.31:0/4073647287 learned_addr learned my addr 172.21.15.31:0/4073647287 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f586b700 1 -- 172.21.15.31:0/4073647287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec009550 con 0x7f16f0106fc0 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f586b700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f019a200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16ec011670 tx=0x7f16ec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:46.671 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16ec003a40 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f019a940 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f16ec003ba0 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16ec024790 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.903+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f019ade0 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.904+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f16ec02d3f0 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.904+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f010a6e0 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.904+0000 7f16e6ffd700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f16dc038410 0x7f16dc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.904+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f16ec04c830 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.905+0000 7f16f506a700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f16dc038410 0x7f16dc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.905+0000 7f16f506a700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f16dc038410 0x7f16dc03a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006fd0 tx=0x7f16e0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:45.909+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f16ec0109d0 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.106+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f16f019dbc0 con 0x7f16f0106fc0 2024-04-22T23:36:46.672 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.106+0000 7f16e6ffd700 1 -- 172.21.15.31:0/4073647287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f16ec04c0c0 con 0x7f16f0106fc0 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f16dc038410 msgr2=0x7f16dc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f16dc038410 0x7f16dc03a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006fd0 tx=0x7f16e0006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 msgr2=0x7f16f019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f019a200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16ec011670 tx=0x7f16ec011b50 comp rx=0 tx=0).stop 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 shutdown_connections 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f16dc038410 0x7f16dc03a8d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 --2- 172.21.15.31:0/4073647287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16f0106fc0 0x7f16f019a200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 >> 172.21.15.31:0/4073647287 conn(0x7f16f01009b0 msgr2=0x7f16f0101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 shutdown_connections 2024-04-22T23:36:46.673 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:46.108+0000 7f16f7acf700 1 -- 172.21.15.31:0/4073647287 wait complete. 2024-04-22T23:36:46.940 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:46 smithi031 ceph-mon[26721]: mgrmap e13: smithi031.fazdcv(active, since 6s) 2024-04-22T23:36:47.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.168+0000 7f07d2253700 1 Processor -- start 2024-04-22T23:36:47.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.168+0000 7f07d2253700 1 -- start start 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.168+0000 7f07d2253700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.168+0000 7f07d2253700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc07a890 con 0x7f07cc07be60 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49638/0 (socket says 172.21.15.31:49638) 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07cb7fe700 1 -- 172.21.15.31:0/3879508810 learned_addr learned my addr 172.21.15.31:0/3879508810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07cb7fe700 1 -- 172.21.15.31:0/3879508810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc07a9d0 con 0x7f07cc07be60 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07cb7fe700 1 --2- 172.21.15.31:0/3879508810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc07a2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009e80 tx=0x7f07c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df938aad20b9b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07ca7fc700 1 -- 172.21.15.31:0/3879508810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07c400e070 con 0x7f07cc07be60 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07ca7fc700 1 -- 172.21.15.31:0/3879508810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f07c400b8c0 con 0x7f07cc07be60 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.169+0000 7f07ca7fc700 1 -- 172.21.15.31:0/3879508810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07c40044d0 con 0x7f07cc07be60 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 -- 172.21.15.31:0/3879508810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 msgr2=0x7f07cc07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 --2- 172.21.15.31:0/3879508810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc07a2c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07c4009e80 tx=0x7f07c400b330 comp rx=0 tx=0).stop 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 -- 172.21.15.31:0/3879508810 shutdown_connections 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 --2- 172.21.15.31:0/3879508810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc07a2c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:47.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 -- 172.21.15.31:0/3879508810 >> 172.21.15.31:0/3879508810 conn(0x7f07cc101b40 msgr2=0x7f07cc103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 -- 172.21.15.31:0/3879508810 shutdown_connections 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.170+0000 7f07d2253700 1 -- 172.21.15.31:0/3879508810 wait complete. 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07d2253700 1 Processor -- start 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07d2253700 1 -- start start 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07d2253700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07d2253700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc19c8f0 con 0x7f07cc07be60 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49654/0 (socket says 172.21.15.31:49654) 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07cb7fe700 1 -- 172.21.15.31:0/2778150991 learned_addr learned my addr 172.21.15.31:0/2778150991 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.171+0000 7f07cb7fe700 1 -- 172.21.15.31:0/2778150991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4009b30 con 0x7f07cc07be60 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.172+0000 7f07cb7fe700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc19c3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07c4000f80 tx=0x7f07c40049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.172+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07c400e040 con 0x7f07cc07be60 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.172+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f07c400bd00 con 0x7f07cc07be60 2024-04-22T23:36:47.994 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.172+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc19caf0 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.172+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07c4012770 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.172+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc19cff0 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.173+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f07c4019040 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.173+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc10c940 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.173+0000 7f07c8ff9700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f07b8038460 0x7f07b803a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.173+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f07c404bdf0 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.173+0000 7f07caffd700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f07b8038460 0x7f07b803a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.174+0000 7f07caffd700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f07b8038460 0x7f07b803a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006fd0 tx=0x7f07b4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.178+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f07c4011750 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.489+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f07cc19fd70 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.492+0000 7f07c8ff9700 1 -- 172.21.15.31:0/2778150991 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f07c401f0c0 con 0x7f07cc07be60 2024-04-22T23:36:47.995 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f07b8038460 msgr2=0x7f07b803a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f07b8038460 0x7f07b803a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006fd0 tx=0x7f07b4006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 msgr2=0x7f07cc19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc19c3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07c4000f80 tx=0x7f07c40049f0 comp rx=0 tx=0).stop 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 shutdown_connections 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f07b8038460 0x7f07b803a920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 --2- 172.21.15.31:0/2778150991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07cc07be60 0x7f07cc19c3b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 >> 172.21.15.31:0/2778150991 conn(0x7f07cc101b40 msgr2=0x7f07cc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 shutdown_connections 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-22T23:36:47.494+0000 7f07d2253700 1 -- 172.21.15.31:0/2778150991 wait complete. 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.996 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:36:47.997 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-04-22T23:36:48.022 INFO:tasks.cephadm:Fetching config... 2024-04-22T23:36:48.022 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:36:48.023 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-22T23:36:48.078 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-22T23:36:48.078 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:36:48.078 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-22T23:36:48.133 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-22T23:36:48.133 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:36:48.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/keyring of=/dev/stdout 2024-04-22T23:36:48.197 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-22T23:36:48.197 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:36:48.197 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-22T23:36:48.252 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-22T23:36:48.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKc0zURdJ64of0unNZ2HniNNJwVczbDDpBIDpHi58qBg1qgFv+ZMnZ9k0W1DrA8ccxeCS1yEfSZ61b8R6GjsOGL1ejNgpVdjHbKlQ23B3fvykPeMaynnFj2RprhUy2l5PQk/iQZekR5VEJMx1GylQXlpz0X/GqDuIoVNerSk5rCSk3bWCmByl3IVFLH086nGN2wT53fc01KqasGBliwMw4tLbkKkIBUAnuq2U0eFvW2bOQQVPqQ4QmDTquZNWgtbhRW6JLTzlnNJrxAIP4PVeF+y1rKf4Figwo0T2HqTGbMnwts4+V5lN4O2Az60zCNXjGt5YBo/pWddtQeauJTlQPkmINCqxeMibo93ARWO1MSiJ4SbbQtSpWSJvC2Ir7BIqWGtB15AaIdSJT8CpCCZTZznHfxgx54L1HudIyHRhEgKtt0+234G0yQVvhz8hhJvT/reqiJjp0fl4QkzYDav6ODsdTSEGOL7LWwAJdMetF7Jy/Yd6BvoLKLjHJDGug7JU= ceph-efa92c84-0100-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T23:36:48.326 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKc0zURdJ64of0unNZ2HniNNJwVczbDDpBIDpHi58qBg1qgFv+ZMnZ9k0W1DrA8ccxeCS1yEfSZ61b8R6GjsOGL1ejNgpVdjHbKlQ23B3fvykPeMaynnFj2RprhUy2l5PQk/iQZekR5VEJMx1GylQXlpz0X/GqDuIoVNerSk5rCSk3bWCmByl3IVFLH086nGN2wT53fc01KqasGBliwMw4tLbkKkIBUAnuq2U0eFvW2bOQQVPqQ4QmDTquZNWgtbhRW6JLTzlnNJrxAIP4PVeF+y1rKf4Figwo0T2HqTGbMnwts4+V5lN4O2Az60zCNXjGt5YBo/pWddtQeauJTlQPkmINCqxeMibo93ARWO1MSiJ4SbbQtSpWSJvC2Ir7BIqWGtB15AaIdSJT8CpCCZTZznHfxgx54L1HudIyHRhEgKtt0+234G0yQVvhz8hhJvT/reqiJjp0fl4QkzYDav6ODsdTSEGOL7LWwAJdMetF7Jy/Yd6BvoLKLjHJDGug7JU= ceph-efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:36:48.341 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKc0zURdJ64of0unNZ2HniNNJwVczbDDpBIDpHi58qBg1qgFv+ZMnZ9k0W1DrA8ccxeCS1yEfSZ61b8R6GjsOGL1ejNgpVdjHbKlQ23B3fvykPeMaynnFj2RprhUy2l5PQk/iQZekR5VEJMx1GylQXlpz0X/GqDuIoVNerSk5rCSk3bWCmByl3IVFLH086nGN2wT53fc01KqasGBliwMw4tLbkKkIBUAnuq2U0eFvW2bOQQVPqQ4QmDTquZNWgtbhRW6JLTzlnNJrxAIP4PVeF+y1rKf4Figwo0T2HqTGbMnwts4+V5lN4O2Az60zCNXjGt5YBo/pWddtQeauJTlQPkmINCqxeMibo93ARWO1MSiJ4SbbQtSpWSJvC2Ir7BIqWGtB15AaIdSJT8CpCCZTZznHfxgx54L1HudIyHRhEgKtt0+234G0yQVvhz8hhJvT/reqiJjp0fl4QkzYDav6ODsdTSEGOL7LWwAJdMetF7Jy/Yd6BvoLKLjHJDGug7JU= ceph-efa92c84-0100-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T23:36:48.381 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKc0zURdJ64of0unNZ2HniNNJwVczbDDpBIDpHi58qBg1qgFv+ZMnZ9k0W1DrA8ccxeCS1yEfSZ61b8R6GjsOGL1ejNgpVdjHbKlQ23B3fvykPeMaynnFj2RprhUy2l5PQk/iQZekR5VEJMx1GylQXlpz0X/GqDuIoVNerSk5rCSk3bWCmByl3IVFLH086nGN2wT53fc01KqasGBliwMw4tLbkKkIBUAnuq2U0eFvW2bOQQVPqQ4QmDTquZNWgtbhRW6JLTzlnNJrxAIP4PVeF+y1rKf4Figwo0T2HqTGbMnwts4+V5lN4O2Az60zCNXjGt5YBo/pWddtQeauJTlQPkmINCqxeMibo93ARWO1MSiJ4SbbQtSpWSJvC2Ir7BIqWGtB15AaIdSJT8CpCCZTZznHfxgx54L1HudIyHRhEgKtt0+234G0yQVvhz8hhJvT/reqiJjp0fl4QkzYDav6ODsdTSEGOL7LWwAJdMetF7Jy/Yd6BvoLKLjHJDGug7JU= ceph-efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:36:48.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-22T23:36:48.599 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:36:48.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:48 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2778150991' entity='client.admin' 2024-04-22T23:36:50.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.899+0000 7f4e0f38e700 1 -- 172.21.15.31:0/1237635080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 msgr2=0x7f4e08100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.899+0000 7f4e0f38e700 1 --2- 172.21.15.31:0/1237635080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08100cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009910 tx=0x7f4df8009c20 comp rx=0 tx=0).stop 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 -- 172.21.15.31:0/1237635080 shutdown_connections 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 --2- 172.21.15.31:0/1237635080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08100cc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 -- 172.21.15.31:0/1237635080 >> 172.21.15.31:0/1237635080 conn(0x7f4e080fbe20 msgr2=0x7f4e080fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 -- 172.21.15.31:0/1237635080 shutdown_connections 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 -- 172.21.15.31:0/1237635080 wait complete. 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 Processor -- start 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 -- start start 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.900+0000 7f4e0f38e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08198110 con 0x7f4e081008a0 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0d12a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0d12a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38346/0 (socket says 172.21.15.31:38346) 2024-04-22T23:36:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0d12a700 1 -- 172.21.15.31:0/250095437 learned_addr learned my addr 172.21.15.31:0/250095437 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0d12a700 1 -- 172.21.15.31:0/250095437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80095f0 con 0x7f4e081008a0 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0d12a700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08197bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005950 tx=0x7f4df8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4df801d070 con 0x7f4e081008a0 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4df800be10 con 0x7f4e081008a0 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e08198310 con 0x7f4e081008a0 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4df800f460 con 0x7f4e081008a0 2024-04-22T23:36:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.901+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e081987b0 con 0x7f4e081008a0 2024-04-22T23:36:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.902+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f4df800f5c0 con 0x7f4e081008a0 2024-04-22T23:36:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.902+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e081918c0 con 0x7f4e081008a0 2024-04-22T23:36:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.902+0000 7f4dfe7fc700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f4df4038410 0x7f4df403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.902+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4df804c4b0 con 0x7f4e081008a0 2024-04-22T23:36:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.902+0000 7f4e0c929700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f4df4038410 0x7f4df403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.903+0000 7f4e0c929700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f4df4038410 0x7f4df403a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e04006fd0 tx=0x7f4e04006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:50.907+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4df8020b10 con 0x7f4e081008a0 2024-04-22T23:36:51.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:51.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:51.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:51.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:51.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:36:51.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:36:51.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:51.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:50 smithi031 ceph-mon[26721]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:36:51.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.109+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f4e0802cc70 con 0x7f4e081008a0 2024-04-22T23:36:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.113+0000 7f4dfe7fc700 1 -- 172.21.15.31:0/250095437 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4df8027030 con 0x7f4e081008a0 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f4df4038410 msgr2=0x7f4df403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f4df4038410 0x7f4df403a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e04006fd0 tx=0x7f4e04006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 msgr2=0x7f4e08197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08197bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4df8005950 tx=0x7f4df8009e60 comp rx=0 tx=0).stop 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 shutdown_connections 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f4df4038410 0x7f4df403a8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 --2- 172.21.15.31:0/250095437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e081008a0 0x7f4e08197bd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 >> 172.21.15.31:0/250095437 conn(0x7f4e080fbe20 msgr2=0x7f4e080fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.117+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 shutdown_connections 2024-04-22T23:36:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:51.118+0000 7f4e0f38e700 1 -- 172.21.15.31:0/250095437 wait complete. 2024-04-22T23:36:51.782 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-22T23:36:51.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-22T23:36:52.304 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:52 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/250095437' entity='client.admin' 2024-04-22T23:36:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.801+0000 7fa7edda7700 1 -- 172.21.15.31:0/1380078300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 msgr2=0x7fa7e8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.801+0000 7fa7edda7700 1 --2- 172.21.15.31:0/1380078300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e8100ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009910 tx=0x7fa7d8009c20 comp rx=0 tx=0).stop 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.802+0000 7fa7edda7700 1 -- 172.21.15.31:0/1380078300 shutdown_connections 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.802+0000 7fa7edda7700 1 --2- 172.21.15.31:0/1380078300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e8100ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.802+0000 7fa7edda7700 1 -- 172.21.15.31:0/1380078300 >> 172.21.15.31:0/1380078300 conn(0x7fa7e80fbe00 msgr2=0x7fa7e80fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.802+0000 7fa7edda7700 1 -- 172.21.15.31:0/1380078300 shutdown_connections 2024-04-22T23:36:54.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.802+0000 7fa7edda7700 1 -- 172.21.15.31:0/1380078300 wait complete. 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.802+0000 7fa7edda7700 1 Processor -- start 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7edda7700 1 -- start start 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7edda7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e81959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7edda7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e8195f10 con 0x7fa7e8100880 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e81959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e81959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38354/0 (socket says 172.21.15.31:38354) 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7e77fe700 1 -- 172.21.15.31:0/2082673676 learned_addr learned my addr 172.21.15.31:0/2082673676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7e77fe700 1 -- 172.21.15.31:0/2082673676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80095f0 con 0x7fa7e8100880 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7e77fe700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e81959d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8005950 tx=0x7fa7d8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.803+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7d801d070 con 0x7fa7e8100880 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.804+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa7d800be10 con 0x7fa7e8100880 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.804+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e8196110 con 0x7fa7e8100880 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.804+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7d800f460 con 0x7fa7e8100880 2024-04-22T23:36:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.804+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e81965b0 con 0x7fa7e8100880 2024-04-22T23:36:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.804+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fa7d800f690 con 0x7fa7e8100880 2024-04-22T23:36:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.805+0000 7fa7e4ff9700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fa7d40383c0 0x7fa7d403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.805+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e818f4a0 con 0x7fa7e8100880 2024-04-22T23:36:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.805+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa7d804c480 con 0x7fa7e8100880 2024-04-22T23:36:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.805+0000 7fa7e6ffd700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fa7d40383c0 0x7fa7d403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.805+0000 7fa7e6ffd700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fa7d40383c0 0x7fa7d403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0006fd0 tx=0x7fa7d0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:54.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:54.810+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa7d8027020 con 0x7fa7e8100880 2024-04-22T23:36:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.025+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e8061260 con 0x7fa7d40383c0 2024-04-22T23:36:55.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.030+0000 7fa7e4ff9700 1 -- 172.21.15.31:0/2082673676 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa7e8061260 con 0x7fa7d40383c0 2024-04-22T23:36:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.032+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fa7d40383c0 msgr2=0x7fa7d403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.032+0000 7fa7edda7700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fa7d40383c0 0x7fa7d403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0006fd0 tx=0x7fa7d0006e40 comp rx=0 tx=0).stop 2024-04-22T23:36:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.032+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 msgr2=0x7fa7e81959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.032+0000 7fa7edda7700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e81959d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8005950 tx=0x7fa7d8009e60 comp rx=0 tx=0).stop 2024-04-22T23:36:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.033+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 shutdown_connections 2024-04-22T23:36:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.033+0000 7fa7edda7700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fa7d40383c0 0x7fa7d403a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.033+0000 7fa7edda7700 1 --2- 172.21.15.31:0/2082673676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e8100880 0x7fa7e81959d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.033+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 >> 172.21.15.31:0/2082673676 conn(0x7fa7e80fbe00 msgr2=0x7fa7e80fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.033+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 shutdown_connections 2024-04-22T23:36:55.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:55.033+0000 7fa7edda7700 1 -- 172.21.15.31:0/2082673676 wait complete. 2024-04-22T23:36:55.757 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi044 2024-04-22T23:36:55.757 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:36:55.757 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.conf 2024-04-22T23:36:55.777 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:36:55.778 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:36:55.836 INFO:tasks.cephadm:Adding host smithi044 to orchestrator... 2024-04-22T23:36:55.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch host add smithi044 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:56.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:56 smithi031 ceph-mon[26721]: Deploying daemon crash.smithi031 on smithi031 2024-04-22T23:36:56.868 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:36:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 -- 172.21.15.31:0/106881069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 msgr2=0x7fb3780ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 --2- 172.21.15.31:0/106881069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3780ffba0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb370009910 tx=0x7fb370009c20 comp rx=0 tx=0).stop 2024-04-22T23:36:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 -- 172.21.15.31:0/106881069 shutdown_connections 2024-04-22T23:36:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 --2- 172.21.15.31:0/106881069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3780ffba0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 -- 172.21.15.31:0/106881069 >> 172.21.15.31:0/106881069 conn(0x7fb3780fad20 msgr2=0x7fb3780fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 -- 172.21.15.31:0/106881069 shutdown_connections 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.330+0000 7fb37ccfe700 1 -- 172.21.15.31:0/106881069 wait complete. 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37ccfe700 1 Processor -- start 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37ccfe700 1 -- start start 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37ccfe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3781959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37ccfe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb378195ee0 con 0x7fb3780ff780 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3781959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3781959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38388/0 (socket says 172.21.15.31:38388) 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37659c700 1 -- 172.21.15.31:0/3914195386 learned_addr learned my addr 172.21.15.31:0/3914195386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.331+0000 7fb37659c700 1 -- 172.21.15.31:0/3914195386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700095f0 con 0x7fb3780ff780 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.332+0000 7fb37659c700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3781959a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb370005b40 tx=0x7fb370004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.332+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb37001d070 con 0x7fb3780ff780 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.332+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb37000be10 con 0x7fb3780ff780 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.332+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3781960e0 con 0x7fb3780ff780 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.332+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb37000f460 con 0x7fb3780ff780 2024-04-22T23:36:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.332+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb378196580 con 0x7fb3780ff780 2024-04-22T23:36:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.333+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fb37000f6d0 con 0x7fb3780ff780 2024-04-22T23:36:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.333+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37818f3b0 con 0x7fb3780ff780 2024-04-22T23:36:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.333+0000 7fb36f7fe700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fb3640383c0 0x7fb36403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.333+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb37004c3e0 con 0x7fb3780ff780 2024-04-22T23:36:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.333+0000 7fb375d9b700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fb3640383c0 0x7fb36403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.334+0000 7fb375d9b700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fb3640383c0 0x7fb36403a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:58.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.338+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb37002a360 con 0x7fb3780ff780 2024-04-22T23:36:58.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:36:58.549+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3780611d0 con 0x7fb3640383c0 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: Deploying cephadm binary to smithi044 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:36:59.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:36:59 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:01.047 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi044' with addr '172.21.15.44' 2024-04-22T23:37:01.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.051+0000 7fb36f7fe700 1 -- 172.21.15.31:0/3914195386 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fb3780611d0 con 0x7fb3640383c0 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fb3640383c0 msgr2=0x7fb36403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fb3640383c0 0x7fb36403a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 msgr2=0x7fb3781959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3781959a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb370005b40 tx=0x7fb370004740 comp rx=0 tx=0).stop 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 shutdown_connections 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fb3640383c0 0x7fb36403a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 --2- 172.21.15.31:0/3914195386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3780ff780 0x7fb3781959a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.053+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 >> 172.21.15.31:0/3914195386 conn(0x7fb3780fad20 msgr2=0x7fb378068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.054+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 shutdown_connections 2024-04-22T23:37:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:01.054+0000 7fb37ccfe700 1 -- 172.21.15.31:0/3914195386 wait complete. 2024-04-22T23:37:01.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T23:37:02.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:02 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:02.287 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:02 smithi031 ceph-mon[26721]: Added host smithi044 2024-04-22T23:37:02.287 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:02 smithi031 ceph-mon[26721]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:02.594 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:37:03.770 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:03 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:03.770 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:03 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:03.770 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:03 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:03.771 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:03 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:03.771 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:03 smithi031 ceph-mon[26721]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-22T23:37:03.771 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:03 smithi031 ceph-mon[26721]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 -- 172.21.15.31:0/775650379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 msgr2=0x7f32b00fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 --2- 172.21.15.31:0/775650379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b00fef70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009910 tx=0x7f32a4009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 -- 172.21.15.31:0/775650379 shutdown_connections 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 --2- 172.21.15.31:0/775650379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b00fef70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 -- 172.21.15.31:0/775650379 >> 172.21.15.31:0/775650379 conn(0x7f32b00fa490 msgr2=0x7f32b00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 -- 172.21.15.31:0/775650379 shutdown_connections 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.010+0000 7f32b6f67700 1 -- 172.21.15.31:0/775650379 wait complete. 2024-04-22T23:37:04.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b6f67700 1 Processor -- start 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b6f67700 1 -- start start 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b6f67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b01959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b6f67700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0195ef0 con 0x7f32b00feb50 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b4d03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b01959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b4d03700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b01959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35876/0 (socket says 172.21.15.31:35876) 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b4d03700 1 -- 172.21.15.31:0/622468891 learned_addr learned my addr 172.21.15.31:0/622468891 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b4d03700 1 -- 172.21.15.31:0/622468891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a40095f0 con 0x7f32b00feb50 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.011+0000 7f32b4d03700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b01959b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f32a4005b40 tx=0x7f32a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.012+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32a401d070 con 0x7f32b00feb50 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.012+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32a400be10 con 0x7f32b00feb50 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.012+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b01960f0 con 0x7f32b00feb50 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.012+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32a400f460 con 0x7f32b00feb50 2024-04-22T23:37:04.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.012+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b0196590 con 0x7f32b00feb50 2024-04-22T23:37:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.013+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f32a400f5c0 con 0x7f32b00feb50 2024-04-22T23:37:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.013+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b018f4c0 con 0x7f32b00feb50 2024-04-22T23:37:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.013+0000 7f32adffb700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f32a0038410 0x7f32a003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.013+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f32a404d260 con 0x7f32b00feb50 2024-04-22T23:37:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.013+0000 7f32affff700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f32a0038410 0x7f32a003a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.013+0000 7f32affff700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f32a0038410 0x7f32a003a8d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:04.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.022+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f32a4020730 con 0x7f32b00feb50 2024-04-22T23:37:04.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.234+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32b00ff680 con 0x7f32a0038410 2024-04-22T23:37:04.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.235+0000 7f32adffb700 1 -- 172.21.15.31:0/622468891 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f32b00ff680 con 0x7f32a0038410 2024-04-22T23:37:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:37:04.230 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}] 2024-04-22T23:37:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f32a0038410 msgr2=0x7f32a003a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f32a0038410 0x7f32a003a8d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f329c006fd0 tx=0x7f329c006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 msgr2=0x7f32b01959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b01959b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f32a4005b40 tx=0x7f32a4004740 comp rx=0 tx=0).stop 2024-04-22T23:37:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 shutdown_connections 2024-04-22T23:37:04.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f32a0038410 0x7f32a003a8d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:04.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 --2- 172.21.15.31:0/622468891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32b00feb50 0x7f32b01959b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:04.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 >> 172.21.15.31:0/622468891 conn(0x7f32b00fa490 msgr2=0x7f32b00fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:04.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 shutdown_connections 2024-04-22T23:37:04.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:04.238+0000 7f32b6f67700 1 -- 172.21.15.31:0/622468891 wait complete. 2024-04-22T23:37:05.208 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-22T23:37:05.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-22T23:37:05.416 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:37:05.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:05 smithi031 ceph-mon[26721]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:37:05.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:05 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:06.924 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:06 smithi031 ceph-mon[26721]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.423+0000 7f2ce908d700 1 -- 172.21.15.31:0/4139114421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 msgr2=0x7f2ce4100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.423+0000 7f2ce908d700 1 --2- 172.21.15.31:0/4139114421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4100ca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009910 tx=0x7f2cd4009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.423+0000 7f2ce908d700 1 -- 172.21.15.31:0/4139114421 shutdown_connections 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.424+0000 7f2ce908d700 1 --2- 172.21.15.31:0/4139114421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4100ca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.424+0000 7f2ce908d700 1 -- 172.21.15.31:0/4139114421 >> 172.21.15.31:0/4139114421 conn(0x7f2ce40fbe20 msgr2=0x7f2ce40fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.424+0000 7f2ce908d700 1 -- 172.21.15.31:0/4139114421 shutdown_connections 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.424+0000 7f2ce908d700 1 -- 172.21.15.31:0/4139114421 wait complete. 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.424+0000 7f2ce908d700 1 Processor -- start 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce908d700 1 -- start start 2024-04-22T23:37:08.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce908d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce908d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4075070 con 0x7f2ce4100880 2024-04-22T23:37:08.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35892/0 (socket says 172.21.15.31:35892) 2024-04-22T23:37:08.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce2d9d700 1 -- 172.21.15.31:0/366366641 learned_addr learned my addr 172.21.15.31:0/366366641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:37:08.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce2d9d700 1 -- 172.21.15.31:0/366366641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40095f0 con 0x7f2ce4100880 2024-04-22T23:37:08.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ce2d9d700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4074b30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4005b40 tx=0x7f2cd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:08.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.425+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cd401d070 con 0x7f2ce4100880 2024-04-22T23:37:08.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.426+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2cd400be10 con 0x7f2ce4100880 2024-04-22T23:37:08.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.426+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce4073180 con 0x7f2ce4100880 2024-04-22T23:37:08.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.426+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cd400f460 con 0x7f2ce4100880 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.426+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce4073620 con 0x7f2ce4100880 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.426+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f2cd400f5c0 con 0x7f2ce4100880 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.427+0000 7f2ccbfff700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f2ccc038410 0x7f2ccc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.427+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce40623c0 con 0x7f2ce4100880 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.427+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2cd404c3a0 con 0x7f2ce4100880 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.427+0000 7f2ce259c700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f2ccc038410 0x7f2ccc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.427+0000 7f2ce259c700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f2ccc038410 0x7f2ccc03a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006fd0 tx=0x7f2cd8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:08.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.432+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2cd4027070 con 0x7f2ce4100880 2024-04-22T23:37:08.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:08.638+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f2ce418ee80 con 0x7f2ce4100880 2024-04-22T23:37:08.732 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:08 smithi031 ceph-mon[26721]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:09.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.509+0000 7f2ccbfff700 1 -- 172.21.15.31:0/366366641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2cd402c440 con 0x7f2ce4100880 2024-04-22T23:37:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f2ccc038410 msgr2=0x7f2ccc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f2ccc038410 0x7f2ccc03a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006fd0 tx=0x7f2cd8006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 msgr2=0x7f2ce4074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4074b30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4005b40 tx=0x7f2cd4004740 comp rx=0 tx=0).stop 2024-04-22T23:37:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 shutdown_connections 2024-04-22T23:37:09.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f2ccc038410 0x7f2ccc03a8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 --2- 172.21.15.31:0/366366641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce4100880 0x7f2ce4074b30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 >> 172.21.15.31:0/366366641 conn(0x7f2ce40fbe20 msgr2=0x7f2ce40fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 shutdown_connections 2024-04-22T23:37:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:37:09.512+0000 7f2ce908d700 1 -- 172.21.15.31:0/366366641 wait complete. 2024-04-22T23:37:09.507 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-04-22T23:37:09.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:09 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/366366641' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T23:37:10.738 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:10 smithi031 ceph-mon[26721]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:10.738 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:10 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/366366641' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T23:37:10.738 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:10 smithi031 ceph-mon[26721]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T23:37:10.765 INFO:tasks.cephadm:Adding mon.smithi031 on smithi031 2024-04-22T23:37:10.766 INFO:tasks.cephadm:Adding mon.smithi044 on smithi044 2024-04-22T23:37:10.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044' 2024-04-22T23:37:10.986 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:11.035 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:12.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 -- 172.21.15.44:0/361720360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 msgr2=0x7fadd4100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 --2- 172.21.15.44:0/361720360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd4100cc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b0a0 tx=0x7fadc400b3b0 comp rx=0 tx=0).stop 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 -- 172.21.15.44:0/361720360 shutdown_connections 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 --2- 172.21.15.44:0/361720360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd4100cc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 -- 172.21.15.44:0/361720360 >> 172.21.15.44:0/361720360 conn(0x7fadd40fbe20 msgr2=0x7fadd40fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 -- 172.21.15.44:0/361720360 shutdown_connections 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.481+0000 7fadd8daa700 1 -- 172.21.15.44:0/361720360 wait complete. 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd8daa700 1 Processor -- start 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd8daa700 1 -- start start 2024-04-22T23:37:12.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd8daa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd41959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd8daa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4195ee0 con 0x7fadd41008a0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd41959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd41959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:45456/0 (socket says 172.21.15.44:45456) 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.482+0000 7fadd259c700 1 -- 172.21.15.44:0/1398211067 learned_addr learned my addr 172.21.15.44:0/1398211067 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadd259c700 1 -- 172.21.15.44:0/1398211067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4009d00 con 0x7fadd41008a0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadd259c700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd41959a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b830 tx=0x7fadc400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadc400e040 con 0x7fadd41008a0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd41960e0 con 0x7fadd41008a0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fadc4012af0 con 0x7fadd41008a0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadc401d9f0 con 0x7fadd41008a0 2024-04-22T23:37:12.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.483+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd4196500 con 0x7fadd41008a0 2024-04-22T23:37:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.484+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fadc401dcb0 con 0x7fadd41008a0 2024-04-22T23:37:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.484+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd418f4c0 con 0x7fadd41008a0 2024-04-22T23:37:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.484+0000 7fadcb7fe700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fadc0038410 0x7fadc003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.484+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fadc404cac0 con 0x7fadd41008a0 2024-04-22T23:37:12.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.484+0000 7fadd1d9b700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fadc0038410 0x7fadc003a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:12.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.485+0000 7fadd1d9b700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fadc0038410 0x7fadc003a8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fadbc006fd0 tx=0x7fadbc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:12.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.490+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fadc4025770 con 0x7fadd41008a0 2024-04-22T23:37:12.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.699+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 --> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044", "target": ["mon-mgr", ""]}) v1 -- 0x7fadd4061280 con 0x7fadc0038410 2024-04-22T23:37:12.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.706+0000 7fadcb7fe700 1 -- 172.21.15.44:0/1398211067 <== mgr.14162 v2:172.21.15.31:6800/3579734707 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fadd4061280 con 0x7fadc0038410 2024-04-22T23:37:12.701 INFO:teuthology.orchestra.run.smithi044.stdout:Scheduled mon update... 2024-04-22T23:37:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fadc0038410 msgr2=0x7fadc003a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:12.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fadc0038410 0x7fadc003a8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fadbc006fd0 tx=0x7fadbc006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 msgr2=0x7fadd41959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd41959a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b830 tx=0x7fadc400bd40 comp rx=0 tx=0).stop 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 shutdown_connections 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fadc0038410 0x7fadc003a8d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 --2- 172.21.15.44:0/1398211067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd41008a0 0x7fadd41959a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 >> 172.21.15.44:0/1398211067 conn(0x7fadd40fbe20 msgr2=0x7fadd40fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 shutdown_connections 2024-04-22T23:37:12.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:12.708+0000 7fadd8daa700 1 -- 172.21.15.44:0/1398211067 wait complete. 2024-04-22T23:37:12.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:12.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-22T23:37:12.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:12 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:13.231 DEBUG:teuthology.orchestra.run.smithi044:mon.smithi044> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi044.service 2024-04-22T23:37:13.234 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:13.234 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:13.479 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:13.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:13 smithi031 ceph-mon[26721]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-22T23:37:13.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:13 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-22T23:37:13.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:13 smithi031 ceph-mon[26721]: Deploying daemon grafana.smithi031 on smithi031 2024-04-22T23:37:13.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:13 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:14.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:14 smithi031 ceph-mon[26721]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:14.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:14 smithi031 ceph-mon[26721]: Saving service mon spec with placement smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044;count:2 2024-04-22T23:37:14.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:14 smithi031 ceph-mon[26721]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:14.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:14 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.981+0000 7f27604ad700 1 -- 172.21.15.44:0/2642309754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 msgr2=0x7f2758103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.981+0000 7f27604ad700 1 --2- 172.21.15.44:0/2642309754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758103700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2754009910 tx=0x7f2754009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.982+0000 7f27604ad700 1 -- 172.21.15.44:0/2642309754 shutdown_connections 2024-04-22T23:37:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.982+0000 7f27604ad700 1 --2- 172.21.15.44:0/2642309754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758103700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:14.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.982+0000 7f27604ad700 1 -- 172.21.15.44:0/2642309754 >> 172.21.15.44:0/2642309754 conn(0x7f27580fad20 msgr2=0x7f27580fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:14.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.982+0000 7f27604ad700 1 -- 172.21.15.44:0/2642309754 shutdown_connections 2024-04-22T23:37:14.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.982+0000 7f27604ad700 1 -- 172.21.15.44:0/2642309754 wait complete. 2024-04-22T23:37:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f27604ad700 1 Processor -- start 2024-04-22T23:37:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f27604ad700 1 -- start start 2024-04-22T23:37:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f27604ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:14.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f27604ad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758198070 con 0x7f2758101310 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f275e249700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f275e249700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:45472/0 (socket says 172.21.15.44:45472) 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f275e249700 1 -- 172.21.15.44:0/3171010987 learned_addr learned my addr 172.21.15.44:0/3171010987 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.983+0000 7f275e249700 1 -- 172.21.15.44:0/3171010987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540095f0 con 0x7f2758101310 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.984+0000 7f275e249700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758197b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2754005b40 tx=0x7f2754004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.984+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f275401d070 con 0x7f2758101310 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.984+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f275400be10 con 0x7f2758101310 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.984+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2758198270 con 0x7f2758101310 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.984+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f275400f460 con 0x7f2758101310 2024-04-22T23:37:14.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.984+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2758198710 con 0x7f2758101310 2024-04-22T23:37:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.985+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f27540227e0 con 0x7f2758101310 2024-04-22T23:37:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.985+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27581918a0 con 0x7f2758101310 2024-04-22T23:37:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.985+0000 7f274f7fe700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f27440383c0 0x7f274403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.985+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f275404c4c0 con 0x7f2758101310 2024-04-22T23:37:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.985+0000 7f275da48700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f27440383c0 0x7f274403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:14.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.986+0000 7f275da48700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f27440383c0 0x7f274403a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2748006fd0 tx=0x7f2748006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:14.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:14.994+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f275402bba0 con 0x7f2758101310 2024-04-22T23:37:15.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.274+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f275802cc70 con 0x7f2758101310 2024-04-22T23:37:15.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.274+0000 7f274f7fe700 1 -- 172.21.15.44:0/3171010987 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2754027020 con 0x7f2758101310 2024-04-22T23:37:15.270 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:15.270 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f27440383c0 msgr2=0x7f274403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f27440383c0 0x7f274403a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2748006fd0 tx=0x7f2748006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 msgr2=0x7f2758197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758197b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2754005b40 tx=0x7f2754004740 comp rx=0 tx=0).stop 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 shutdown_connections 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f27440383c0 0x7f274403a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:15.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 --2- 172.21.15.44:0/3171010987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2758101310 0x7f2758197b30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 >> 172.21.15.44:0/3171010987 conn(0x7f27580fad20 msgr2=0x7f27580fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 shutdown_connections 2024-04-22T23:37:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:15.276+0000 7f27604ad700 1 -- 172.21.15.44:0/3171010987 wait complete. 2024-04-22T23:37:15.272 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:16.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:15 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/3171010987' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:16.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:15 smithi031 ceph-mon[26721]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:16.787 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:16.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:16.995 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 -- 172.21.15.44:0/1696801883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 msgr2=0x7fc6cc100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 --2- 172.21.15.44:0/1696801883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc100870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009910 tx=0x7fc6bc009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:18.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 -- 172.21.15.44:0/1696801883 shutdown_connections 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 --2- 172.21.15.44:0/1696801883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc100870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 -- 172.21.15.44:0/1696801883 >> 172.21.15.44:0/1696801883 conn(0x7fc6cc0fa050 msgr2=0x7fc6cc0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 -- 172.21.15.44:0/1696801883 shutdown_connections 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 -- 172.21.15.44:0/1696801883 wait complete. 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.492+0000 7fc6d1d75700 1 Processor -- start 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6d1d75700 1 -- start start 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6d1d75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6d1d75700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc195ec0 con 0x7fc6cc100450 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:18.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:45492/0 (socket says 172.21.15.44:45492) 2024-04-22T23:37:18.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6cb7fe700 1 -- 172.21.15.44:0/205051826 learned_addr learned my addr 172.21.15.44:0/205051826 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6cb7fe700 1 -- 172.21.15.44:0/205051826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0095f0 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.493+0000 7fc6cb7fe700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc195980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005b40 tx=0x7fc6bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.494+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6bc01d070 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.494+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc1960c0 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.494+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc6bc00be10 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.494+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6bc00f460 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.494+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc196560 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.494+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc6bc00f5c0 con 0x7fc6cc100450 2024-04-22T23:37:18.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.495+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc18fcd0 con 0x7fc6cc100450 2024-04-22T23:37:18.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.495+0000 7fc6c8ff9700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc6b4038410 0x7fc6b403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:18.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.495+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc6bc04c390 con 0x7fc6cc100450 2024-04-22T23:37:18.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.495+0000 7fc6caffd700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc6b4038410 0x7fc6b403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:18.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.495+0000 7fc6caffd700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc6b4038410 0x7fc6b403a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006fd0 tx=0x7fc6c0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:18.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.500+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc6bc02bb60 con 0x7fc6cc100450 2024-04-22T23:37:18.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.781+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6cc02cc70 con 0x7fc6cc100450 2024-04-22T23:37:18.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.782+0000 7fc6c8ff9700 1 -- 172.21.15.44:0/205051826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc6bc027020 con 0x7fc6cc100450 2024-04-22T23:37:18.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:18.777 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc6b4038410 msgr2=0x7fc6b403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc6b4038410 0x7fc6b403a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0006fd0 tx=0x7fc6c0006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 msgr2=0x7fc6cc195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc195980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc005b40 tx=0x7fc6bc004740 comp rx=0 tx=0).stop 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 shutdown_connections 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc6b4038410 0x7fc6b403a8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 --2- 172.21.15.44:0/205051826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6cc100450 0x7fc6cc195980 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 >> 172.21.15.44:0/205051826 conn(0x7fc6cc0fa050 msgr2=0x7fc6cc0fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 shutdown_connections 2024-04-22T23:37:18.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:18.784+0000 7fc6d1d75700 1 -- 172.21.15.44:0/205051826 wait complete. 2024-04-22T23:37:18.780 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:18.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:18 smithi031 ceph-mon[26721]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:19.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:19 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/205051826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:20.341 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:20.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:20.539 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:20.590 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:20 smithi031 ceph-mon[26721]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:21.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.961+0000 7ff324cdc700 1 -- 172.21.15.44:0/2527587985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 msgr2=0x7ff320100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.961+0000 7ff324cdc700 1 --2- 172.21.15.44:0/2527587985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320100cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff314009910 tx=0x7ff314009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 -- 172.21.15.44:0/2527587985 shutdown_connections 2024-04-22T23:37:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 --2- 172.21.15.44:0/2527587985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320100cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 -- 172.21.15.44:0/2527587985 >> 172.21.15.44:0/2527587985 conn(0x7ff3200fbe20 msgr2=0x7ff3200fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 -- 172.21.15.44:0/2527587985 shutdown_connections 2024-04-22T23:37:21.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 -- 172.21.15.44:0/2527587985 wait complete. 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 Processor -- start 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.962+0000 7ff324cdc700 1 -- start start 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff324cdc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff324cdc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320075070 con 0x7ff3201008a0 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff31e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff31e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:55120/0 (socket says 172.21.15.44:55120) 2024-04-22T23:37:21.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff31e59c700 1 -- 172.21.15.44:0/2848238467 learned_addr learned my addr 172.21.15.44:0/2848238467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff31e59c700 1 -- 172.21.15.44:0/2848238467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3140095f0 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff31e59c700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320074b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff314005950 tx=0x7ff314009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.963+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff31401d070 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.964+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff31400be10 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.964+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320073180 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.964+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff31400f460 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.964+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff320073620 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.964+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7ff31400b960 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.964+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3200623c0 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.965+0000 7ff30f7fe700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff308038410 0x7ff30803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.965+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff31404c490 con 0x7ff3201008a0 2024-04-22T23:37:21.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.965+0000 7ff31dd9b700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff308038410 0x7ff30803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:21.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.965+0000 7ff31dd9b700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff308038410 0x7ff30803a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff310006fd0 tx=0x7ff310006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:21.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:21.970+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff31402bd40 con 0x7ff3201008a0 2024-04-22T23:37:22.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.247+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff32018eea0 con 0x7ff3201008a0 2024-04-22T23:37:22.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.248+0000 7ff30f7fe700 1 -- 172.21.15.44:0/2848238467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff314027030 con 0x7ff3201008a0 2024-04-22T23:37:22.243 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:22.243 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:22.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff308038410 msgr2=0x7ff30803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:22.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff308038410 0x7ff30803a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff310006fd0 tx=0x7ff310006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 msgr2=0x7ff320074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320074b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff314005950 tx=0x7ff314009e60 comp rx=0 tx=0).stop 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 shutdown_connections 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff308038410 0x7ff30803a8d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 --2- 172.21.15.44:0/2848238467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3201008a0 0x7ff320074b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 >> 172.21.15.44:0/2848238467 conn(0x7ff3200fbe20 msgr2=0x7ff3200fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 shutdown_connections 2024-04-22T23:37:22.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:22.250+0000 7ff324cdc700 1 -- 172.21.15.44:0/2848238467 wait complete. 2024-04-22T23:37:22.247 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:22.769 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:22 smithi031 ceph-mon[26721]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:22.769 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:22 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/2848238467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:23.805 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:23.805 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:24.058 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:24.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:24 smithi031 ceph-mon[26721]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:25.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 -- 172.21.15.44:0/2438640417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 msgr2=0x7ff6f00fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 --2- 172.21.15.44:0/2438640417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f00fef70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009910 tx=0x7ff6e0009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 -- 172.21.15.44:0/2438640417 shutdown_connections 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 --2- 172.21.15.44:0/2438640417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f00fef70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 -- 172.21.15.44:0/2438640417 >> 172.21.15.44:0/2438640417 conn(0x7ff6f00fa490 msgr2=0x7ff6f00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 -- 172.21.15.44:0/2438640417 shutdown_connections 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 -- 172.21.15.44:0/2438640417 wait complete. 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.423+0000 7ff6f49a9700 1 Processor -- start 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6f49a9700 1 -- start start 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6f49a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6f49a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f01019d0 con 0x7ff6f00feb50 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6ee59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6ee59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f0101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:55134/0 (socket says 172.21.15.44:55134) 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6ee59c700 1 -- 172.21.15.44:0/3636670449 learned_addr learned my addr 172.21.15.44:0/3636670449 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:25.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6ee59c700 1 -- 172.21.15.44:0/3636670449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00095f0 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.424+0000 7ff6ee59c700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f0101490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0005950 tx=0x7ff6e0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.425+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6e001d070 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.425+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff6e000be10 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.425+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f00ffae0 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.425+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6e000f460 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.425+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f00fff80 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.425+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7ff6e000f690 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.426+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f00623c0 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.426+0000 7ff6df7fe700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff6d80383c0 0x7ff6d803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.426+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff6e004c370 con 0x7ff6f00feb50 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.426+0000 7ff6edd9b700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff6d80383c0 0x7ff6d803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:25.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.426+0000 7ff6edd9b700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff6d80383c0 0x7ff6d803a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4006fd0 tx=0x7ff6e4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:25.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.431+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff6e0020730 con 0x7ff6f00feb50 2024-04-22T23:37:25.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.710+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff6f00ff680 con 0x7ff6f00feb50 2024-04-22T23:37:25.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.710+0000 7ff6df7fe700 1 -- 172.21.15.44:0/3636670449 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff6f00ff680 con 0x7ff6f00feb50 2024-04-22T23:37:25.705 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:25.705 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff6d80383c0 msgr2=0x7ff6d803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff6d80383c0 0x7ff6d803a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4006fd0 tx=0x7ff6e4006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 msgr2=0x7ff6f0101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f0101490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0005950 tx=0x7ff6e0009e60 comp rx=0 tx=0).stop 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 shutdown_connections 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff6d80383c0 0x7ff6d803a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 --2- 172.21.15.44:0/3636670449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6f00feb50 0x7ff6f0101490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:25.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.712+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 >> 172.21.15.44:0/3636670449 conn(0x7ff6f00fa490 msgr2=0x7ff6f00fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.713+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 shutdown_connections 2024-04-22T23:37:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:25.713+0000 7ff6f49a9700 1 -- 172.21.15.44:0/3636670449 wait complete. 2024-04-22T23:37:25.708 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:26.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:26.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:25 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/3636670449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:27.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:26 smithi031 ceph-mon[26721]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-22T23:37:27.235 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:27.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:27.434 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:28.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:27 smithi031 ceph-mon[26721]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 -- 172.21.15.44:0/182707252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 msgr2=0x7ff9bc0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 --2- 172.21.15.44:0/182707252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc0ffba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009910 tx=0x7ff9b8009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 -- 172.21.15.44:0/182707252 shutdown_connections 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 --2- 172.21.15.44:0/182707252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc0ffba0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 -- 172.21.15.44:0/182707252 >> 172.21.15.44:0/182707252 conn(0x7ff9bc0fad20 msgr2=0x7ff9bc0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 -- 172.21.15.44:0/182707252 shutdown_connections 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 -- 172.21.15.44:0/182707252 wait complete. 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 Processor -- start 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.958+0000 7ff9c44c2700 1 -- start start 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c44c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:28.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c44c2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc1986f0 con 0x7ff9bc0ff780 2024-04-22T23:37:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c225e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c225e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc069400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:55152/0 (socket says 172.21.15.44:55152) 2024-04-22T23:37:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c225e700 1 -- 172.21.15.44:0/3084548066 learned_addr learned my addr 172.21.15.44:0/3084548066 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c225e700 1 -- 172.21.15.44:0/3084548066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80095f0 con 0x7ff9bc0ff780 2024-04-22T23:37:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9c225e700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc069400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005950 tx=0x7ff9b8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:28.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.959+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9b801d070 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.960+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9b800be10 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.960+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc198830 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.960+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff9b800f460 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.960+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc198cd0 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.960+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7ff9b800b960 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.960+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc18e3a0 con 0x7ff9bc0ff780 2024-04-22T23:37:28.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.961+0000 7ff9b37fe700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff9a80383c0 0x7ff9a803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.961+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff9b804c480 con 0x7ff9bc0ff780 2024-04-22T23:37:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.961+0000 7ff9c1a5d700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff9a80383c0 0x7ff9a803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:28.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.961+0000 7ff9c1a5d700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff9a80383c0 0x7ff9a803a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac006fd0 tx=0x7ff9ac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:28.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:28.965+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff9b802bba0 con 0x7ff9bc0ff780 2024-04-22T23:37:29.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.243+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9bc02cc70 con 0x7ff9bc0ff780 2024-04-22T23:37:29.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.243+0000 7ff9b37fe700 1 -- 172.21.15.44:0/3084548066 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff9b8027030 con 0x7ff9bc0ff780 2024-04-22T23:37:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:29.238 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:29.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff9a80383c0 msgr2=0x7ff9a803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff9a80383c0 0x7ff9a803a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac006fd0 tx=0x7ff9ac006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 msgr2=0x7ff9bc069400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc069400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8005950 tx=0x7ff9b8009e60 comp rx=0 tx=0).stop 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 shutdown_connections 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff9a80383c0 0x7ff9a803a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 --2- 172.21.15.44:0/3084548066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9bc0ff780 0x7ff9bc069400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 >> 172.21.15.44:0/3084548066 conn(0x7ff9bc0fad20 msgr2=0x7ff9bc0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 shutdown_connections 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:29.245+0000 7ff9c44c2700 1 -- 172.21.15.44:0/3084548066 wait complete. 2024-04-22T23:37:29.240 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:29.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:29 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/3084548066' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:30.779 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:30.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:30.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:30 smithi031 ceph-mon[26721]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:30.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:30 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:30.982 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:31.030 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:32.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 -- 172.21.15.44:0/2792957643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 msgr2=0x7f315c0fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 --2- 172.21.15.44:0/2792957643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c0fef70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f314c009910 tx=0x7f314c009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 -- 172.21.15.44:0/2792957643 shutdown_connections 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 --2- 172.21.15.44:0/2792957643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c0fef70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 -- 172.21.15.44:0/2792957643 >> 172.21.15.44:0/2792957643 conn(0x7f315c0fa4b0 msgr2=0x7f315c0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 -- 172.21.15.44:0/2792957643 shutdown_connections 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.395+0000 7f31618d0700 1 -- 172.21.15.44:0/2792957643 wait complete. 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.396+0000 7f31618d0700 1 Processor -- start 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.396+0000 7f31618d0700 1 -- start start 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.396+0000 7f31618d0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:32.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.396+0000 7f31618d0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c1019d0 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f315affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f315affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:43422/0 (socket says 172.21.15.44:43422) 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f315affd700 1 -- 172.21.15.44:0/2967856242 learned_addr learned my addr 172.21.15.44:0/2967856242 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f315affd700 1 -- 172.21.15.44:0/2967856242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0095f0 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f315affd700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c101490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f314c005950 tx=0x7f314c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f314c01d070 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f314c00be10 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315c0ffae0 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.397+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315c0fff80 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.398+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f314c00f460 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.398+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f314c00f5c0 con 0x7f315c0feb50 2024-04-22T23:37:32.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.398+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3148005320 con 0x7f315c0feb50 2024-04-22T23:37:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.398+0000 7f31608ce700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f3144038410 0x7f314403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.398+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f314c04c250 con 0x7f315c0feb50 2024-04-22T23:37:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.399+0000 7f315a7fc700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f3144038410 0x7f314403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:32.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.399+0000 7f315a7fc700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f3144038410 0x7f314403a8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3150006fd0 tx=0x7f3150006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:32.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.403+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f314c020730 con 0x7f315c0feb50 2024-04-22T23:37:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.684+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3148005190 con 0x7f315c0feb50 2024-04-22T23:37:32.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.685+0000 7f31608ce700 1 -- 172.21.15.44:0/2967856242 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f314c027020 con 0x7f315c0feb50 2024-04-22T23:37:32.679 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:32.679 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:32.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f3144038410 msgr2=0x7f314403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:32.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f3144038410 0x7f314403a8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3150006fd0 tx=0x7f3150006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:32.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 msgr2=0x7f315c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:32.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c101490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f314c005950 tx=0x7f314c009e60 comp rx=0 tx=0).stop 2024-04-22T23:37:32.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 shutdown_connections 2024-04-22T23:37:32.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f3144038410 0x7f314403a8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:32.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 --2- 172.21.15.44:0/2967856242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f315c0feb50 0x7f315c101490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:32.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 >> 172.21.15.44:0/2967856242 conn(0x7f315c0fa4b0 msgr2=0x7f315c0fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:32.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 shutdown_connections 2024-04-22T23:37:32.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:32.687+0000 7f31618d0700 1 -- 172.21.15.44:0/2967856242 wait complete. 2024-04-22T23:37:32.682 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:32.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:32 smithi031 ceph-mon[26721]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:33.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:33 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/2967856242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:34.260 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:34.260 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:34.480 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:34.527 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:34.783 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:34 smithi031 ceph-mon[26721]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 -- 172.21.15.44:0/2974791369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 msgr2=0x7ff388100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 --2- 172.21.15.44:0/2974791369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff388100cc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009910 tx=0x7ff37c009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 -- 172.21.15.44:0/2974791369 shutdown_connections 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 --2- 172.21.15.44:0/2974791369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff388100cc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 -- 172.21.15.44:0/2974791369 >> 172.21.15.44:0/2974791369 conn(0x7ff3880fbe20 msgr2=0x7ff3880fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 -- 172.21.15.44:0/2974791369 shutdown_connections 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.908+0000 7ff38ec9a700 1 -- 172.21.15.44:0/2974791369 wait complete. 2024-04-22T23:37:35.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ec9a700 1 Processor -- start 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ec9a700 1 -- start start 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ec9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff3881959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ec9a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff388195ee0 con 0x7ff3881008a0 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ca36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff3881959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ca36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff3881959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:43448/0 (socket says 172.21.15.44:43448) 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ca36700 1 -- 172.21.15.44:0/1177221197 learned_addr learned my addr 172.21.15.44:0/1177221197 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.909+0000 7ff38ca36700 1 -- 172.21.15.44:0/1177221197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0095f0 con 0x7ff3881008a0 2024-04-22T23:37:35.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.910+0000 7ff38ca36700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff3881959a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005b40 tx=0x7ff37c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:35.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.910+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff37c01d070 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.910+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3881960e0 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.910+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff37c00be10 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.910+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff37c00f460 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.910+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff388196580 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.911+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38818f4c0 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.911+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7ff37c0227e0 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.911+0000 7ff385ffb700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff3780383c0 0x7ff37803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.911+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff37c04c470 con 0x7ff3881008a0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.912+0000 7ff387fff700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff3780383c0 0x7ff37803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:35.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.912+0000 7ff387fff700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff3780383c0 0x7ff37803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff374006fd0 tx=0x7ff374006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:35.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:35.920+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff37c02bba0 con 0x7ff3881008a0 2024-04-22T23:37:36.105 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:35 smithi031 ceph-mon[26721]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.201+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff38802cc70 con 0x7ff3881008a0 2024-04-22T23:37:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.201+0000 7ff385ffb700 1 -- 172.21.15.44:0/1177221197 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff37c027020 con 0x7ff3881008a0 2024-04-22T23:37:36.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:36.196 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff3780383c0 msgr2=0x7ff37803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff3780383c0 0x7ff37803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff374006fd0 tx=0x7ff374006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 msgr2=0x7ff3881959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff3881959a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff37c005b40 tx=0x7ff37c004740 comp rx=0 tx=0).stop 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 shutdown_connections 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7ff3780383c0 0x7ff37803a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 --2- 172.21.15.44:0/1177221197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3881008a0 0x7ff3881959a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 >> 172.21.15.44:0/1177221197 conn(0x7ff3880fbe20 msgr2=0x7ff3880fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 shutdown_connections 2024-04-22T23:37:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:36.203+0000 7ff38ec9a700 1 -- 172.21.15.44:0/1177221197 wait complete. 2024-04-22T23:37:36.199 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:37.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:36 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/1177221197' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:37.778 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:37.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:37 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:37 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:37 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:37 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-22T23:37:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:37 smithi031 ceph-mon[26721]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:37:37.993 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:38.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:38 smithi031 ceph-mon[26721]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-22T23:37:38.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:38 smithi031 ceph-mon[26721]: mgrmap e14: smithi031.fazdcv(active, since 58s) 2024-04-22T23:37:39.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 -- 172.21.15.44:0/2546389952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 msgr2=0x7f51ac103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 --2- 172.21.15.44:0/2546389952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac103710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009910 tx=0x7f51a0009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:39.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 -- 172.21.15.44:0/2546389952 shutdown_connections 2024-04-22T23:37:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 --2- 172.21.15.44:0/2546389952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac103710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 -- 172.21.15.44:0/2546389952 >> 172.21.15.44:0/2546389952 conn(0x7f51ac0fad30 msgr2=0x7f51ac0fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 -- 172.21.15.44:0/2546389952 shutdown_connections 2024-04-22T23:37:39.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 -- 172.21.15.44:0/2546389952 wait complete. 2024-04-22T23:37:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 Processor -- start 2024-04-22T23:37:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.355+0000 7f51b2f20700 1 -- start start 2024-04-22T23:37:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b2f20700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b2f20700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac195ea0 con 0x7f51ac101320 2024-04-22T23:37:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b0cbc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b0cbc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:43456/0 (socket says 172.21.15.44:43456) 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b0cbc700 1 -- 172.21.15.44:0/1661022376 learned_addr learned my addr 172.21.15.44:0/1661022376 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b0cbc700 1 -- 172.21.15.44:0/1661022376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00095f0 con 0x7f51ac101320 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b0cbc700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac195960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51a0005b40 tx=0x7f51a0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a001d070 con 0x7f51ac101320 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51a000be10 con 0x7f51ac101320 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac1960a0 con 0x7f51ac101320 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.356+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a000f460 con 0x7f51ac101320 2024-04-22T23:37:39.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.357+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac196540 con 0x7f51ac101320 2024-04-22T23:37:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.357+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f51a00227e0 con 0x7f51ac101320 2024-04-22T23:37:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.357+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac18f4b0 con 0x7f51ac101320 2024-04-22T23:37:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.358+0000 7f51a9ffb700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 0x7f519c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.358+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51a004c4a0 con 0x7f51ac101320 2024-04-22T23:37:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.358+0000 7f51abfff700 1 -- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 msgr2=0x7f519c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3579734707 2024-04-22T23:37:39.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.358+0000 7f51abfff700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 0x7f519c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:37:39.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.362+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f51a002bba0 con 0x7f51ac101320 2024-04-22T23:37:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.559+0000 7f51abfff700 1 -- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 msgr2=0x7f519c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3579734707 2024-04-22T23:37:39.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.559+0000 7f51abfff700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 0x7f519c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:37:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.645+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51ac02cc70 con 0x7f51ac101320 2024-04-22T23:37:39.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.646+0000 7f51a9ffb700 1 -- 172.21.15.44:0/1661022376 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f51a0027030 con 0x7f51ac101320 2024-04-22T23:37:39.641 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:39.641 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 msgr2=0x7f519c03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 0x7f519c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 msgr2=0x7f51ac195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac195960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f51a0005b40 tx=0x7f51a0004740 comp rx=0 tx=0).stop 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 shutdown_connections 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7f519c038410 0x7f519c03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 --2- 172.21.15.44:0/1661022376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51ac101320 0x7f51ac195960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 >> 172.21.15.44:0/1661022376 conn(0x7f51ac0fad30 msgr2=0x7f51ac0fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 shutdown_connections 2024-04-22T23:37:39.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:39.648+0000 7f51b2f20700 1 -- 172.21.15.44:0/1661022376 wait complete. 2024-04-22T23:37:39.644 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:40.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:39 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/1661022376' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:41.216 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:41.217 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:41.427 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:41.475 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 -- 172.21.15.44:0/641404316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 msgr2=0x7fc07c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 --2- 172.21.15.44:0/641404316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c0731a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc078009910 tx=0x7fc078009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 -- 172.21.15.44:0/641404316 shutdown_connections 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 --2- 172.21.15.44:0/641404316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c0731a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 -- 172.21.15.44:0/641404316 >> 172.21.15.44:0/641404316 conn(0x7fc07c0fbe20 msgr2=0x7fc07c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 -- 172.21.15.44:0/641404316 shutdown_connections 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 -- 172.21.15.44:0/641404316 wait complete. 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 Processor -- start 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.907+0000 7fc083eed700 1 -- start start 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc083eed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:42.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc083eed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c198100 con 0x7fc07c074d40 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc081c89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc081c89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:42378/0 (socket says 172.21.15.44:42378) 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc081c89700 1 -- 172.21.15.44:0/3079550682 learned_addr learned my addr 172.21.15.44:0/3079550682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc081c89700 1 -- 172.21.15.44:0/3079550682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780095f0 con 0x7fc07c074d40 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc081c89700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c197bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0780092e0 tx=0x7fc078009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc07801d070 con 0x7fc07c074d40 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc07800be10 con 0x7fc07c074d40 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc07800f460 con 0x7fc07c074d40 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.908+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c198300 con 0x7fc07c074d40 2024-04-22T23:37:42.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.909+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c1987a0 con 0x7fc07c074d40 2024-04-22T23:37:42.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.909+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7fc07800b960 con 0x7fc07c074d40 2024-04-22T23:37:42.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.909+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c1916c0 con 0x7fc07c074d40 2024-04-22T23:37:42.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.910+0000 7fc072ffd700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 0x7fc06803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:42.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.910+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc07804c570 con 0x7fc07c074d40 2024-04-22T23:37:42.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.910+0000 7fc081488700 1 -- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 msgr2=0x7fc06803a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3579734707 2024-04-22T23:37:42.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.910+0000 7fc081488700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 0x7fc06803a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:37:42.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:42.915+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc078005c00 con 0x7fc07c074d40 2024-04-22T23:37:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.111+0000 7fc081488700 1 -- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 msgr2=0x7fc06803a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/3579734707 2024-04-22T23:37:43.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.111+0000 7fc081488700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 0x7fc06803a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:37:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.192+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc07c02cc70 con 0x7fc07c074d40 2024-04-22T23:37:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.193+0000 7fc072ffd700 1 -- 172.21.15.44:0/3079550682 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc078027070 con 0x7fc07c074d40 2024-04-22T23:37:43.187 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:43.188 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 msgr2=0x7fc06803a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 0x7fc06803a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 msgr2=0x7fc07c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c197bc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc0780092e0 tx=0x7fc078009e60 comp rx=0 tx=0).stop 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 shutdown_connections 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:6800/3579734707,v1:172.21.15.31:6801/3579734707] conn(0x7fc068038410 0x7fc06803a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 --2- 172.21.15.44:0/3079550682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c074d40 0x7fc07c197bc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 >> 172.21.15.44:0/3079550682 conn(0x7fc07c0fbe20 msgr2=0x7fc07c0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 shutdown_connections 2024-04-22T23:37:43.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:43.195+0000 7fc083eed700 1 -- 172.21.15.44:0/3079550682 wait complete. 2024-04-22T23:37:43.191 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:43.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:43 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/3079550682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:44.753 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:44.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:44.961 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:45.019 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:46.143 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: Active manager daemon smithi031.fazdcv restarted 2024-04-22T23:37:46.143 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:37:46.143 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: mgrmap e15: smithi031.fazdcv(active, starting, since 0.0043203s) 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:37:46.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 -- 172.21.15.44:0/1045660333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 msgr2=0x7fb2c00ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 --2- 172.21.15.44:0/1045660333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c00ffb40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009910 tx=0x7fb2b0009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 -- 172.21.15.44:0/1045660333 shutdown_connections 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 --2- 172.21.15.44:0/1045660333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c00ffb40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 -- 172.21.15.44:0/1045660333 >> 172.21.15.44:0/1045660333 conn(0x7fb2c00facc0 msgr2=0x7fb2c00fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 -- 172.21.15.44:0/1045660333 shutdown_connections 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.539+0000 7fb2c60a1700 1 -- 172.21.15.44:0/1045660333 wait complete. 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2c60a1700 1 Processor -- start 2024-04-22T23:37:46.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2c60a1700 1 -- start start 2024-04-22T23:37:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2c60a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c018c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2c60a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c018cc40 con 0x7fb2c00ff720 2024-04-22T23:37:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c018c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c018c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:42408/0 (socket says 172.21.15.44:42408) 2024-04-22T23:37:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.540+0000 7fb2bf7fe700 1 -- 172.21.15.44:0/144794986 learned_addr learned my addr 172.21.15.44:0/144794986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:46.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2bf7fe700 1 -- 172.21.15.44:0/144794986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b00095f0 con 0x7fb2c00ff720 2024-04-22T23:37:46.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2bf7fe700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c018c700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0005b40 tx=0x7fb2b0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:46.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b001d070 con 0x7fb2c00ff720 2024-04-22T23:37:46.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c018ad50 con 0x7fb2c00ff720 2024-04-22T23:37:46.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb2b000be10 con 0x7fb2c00ff720 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b000f460 con 0x7fb2c00ff720 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.541+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c018b1f0 con 0x7fb2c00ff720 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.542+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7fb2b000f6d0 con 0x7fb2c00ff720 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.542+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c0105560 con 0x7fb2c00ff720 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.542+0000 7fb2bcff9700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb2a8038260 0x7fb2a803a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.542+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb2b004c400 con 0x7fb2c00ff720 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.542+0000 7fb2beffd700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb2a8038260 0x7fb2a803a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:46.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.543+0000 7fb2beffd700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb2a8038260 0x7fb2a803a720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4006fd0 tx=0x7fb2b4006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:46.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.548+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb2b002a360 con 0x7fb2c00ff720 2024-04-22T23:37:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.836+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb2c0105750 con 0x7fb2c00ff720 2024-04-22T23:37:46.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.837+0000 7fb2bcff9700 1 -- 172.21.15.44:0/144794986 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb2b0027020 con 0x7fb2c00ff720 2024-04-22T23:37:46.832 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:46.832 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb2a8038260 msgr2=0x7fb2a803a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb2a8038260 0x7fb2a803a720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4006fd0 tx=0x7fb2b4006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 msgr2=0x7fb2c018c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c018c700 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0005b40 tx=0x7fb2b0004740 comp rx=0 tx=0).stop 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 shutdown_connections 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb2a8038260 0x7fb2a803a720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 --2- 172.21.15.44:0/144794986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00ff720 0x7fb2c018c700 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 >> 172.21.15.44:0/144794986 conn(0x7fb2c00facc0 msgr2=0x7fb2c00fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 shutdown_connections 2024-04-22T23:37:46.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:46.839+0000 7fb2c60a1700 1 -- 172.21.15.44:0/144794986 wait complete. 2024-04-22T23:37:46.835 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:47.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: [22/Apr/2024:23:37:45] ENGINE Bus STARTING 2024-04-22T23:37:47.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: [22/Apr/2024:23:37:45] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:37:47.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: [22/Apr/2024:23:37:45] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:37:47.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: [22/Apr/2024:23:37:45] ENGINE Bus STARTED 2024-04-22T23:37:47.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: mgrmap e16: smithi031.fazdcv(active, since 1.00645s) 2024-04-22T23:37:47.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:47.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:46 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/144794986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:48.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:48 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:48.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:48 smithi031 ceph-mon[26721]: mgrmap e17: smithi031.fazdcv(active, since 2s) 2024-04-22T23:37:48.494 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:48.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:49.042 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:49.100 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:37:50.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:50.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:50.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:50.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:50.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:37:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 -- 172.21.15.44:0/1687854818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 msgr2=0x7fef1c106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 --2- 172.21.15.44:0/1687854818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c106bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef10009910 tx=0x7fef10009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 -- 172.21.15.44:0/1687854818 shutdown_connections 2024-04-22T23:37:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 --2- 172.21.15.44:0/1687854818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c106bf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 -- 172.21.15.44:0/1687854818 >> 172.21.15.44:0/1687854818 conn(0x7fef1c0fc320 msgr2=0x7fef1c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:51.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 -- 172.21.15.44:0/1687854818 shutdown_connections 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 -- 172.21.15.44:0/1687854818 wait complete. 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 Processor -- start 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.319+0000 7fef23172700 1 -- start start 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef23172700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef23172700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c075010 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef20f0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef20f0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:57686/0 (socket says 172.21.15.44:57686) 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef20f0e700 1 -- 172.21.15.44:0/37099818 learned_addr learned my addr 172.21.15.44:0/37099818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef20f0e700 1 -- 172.21.15.44:0/37099818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100095f0 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef20f0e700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c073090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef10005b40 tx=0x7fef10004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef1001d070 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c0735d0 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef100044d0 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.320+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef10003930 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.321+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c073a50 con 0x7fef1c106810 2024-04-22T23:37:51.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.322+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c104ac0 con 0x7fef1c106810 2024-04-22T23:37:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.322+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fef10003bf0 con 0x7fef1c106810 2024-04-22T23:37:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.322+0000 7fef19ffb700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0c0384b0 0x7fef0c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.322+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fef1004c4f0 con 0x7fef1c106810 2024-04-22T23:37:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.322+0000 7fef1bfff700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0c0384b0 0x7fef0c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.323+0000 7fef1bfff700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0c0384b0 0x7fef0c03a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef08006fd0 tx=0x7fef08006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:51.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.327+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef1002a360 con 0x7fef1c106810 2024-04-22T23:37:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.615+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef1c0623c0 con 0x7fef1c106810 2024-04-22T23:37:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.615+0000 7fef19ffb700 1 -- 172.21.15.44:0/37099818 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fef10020b20 con 0x7fef1c106810 2024-04-22T23:37:51.609 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:51.609 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0c0384b0 msgr2=0x7fef0c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0c0384b0 0x7fef0c03a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef08006fd0 tx=0x7fef08006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 msgr2=0x7fef1c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c073090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef10005b40 tx=0x7fef10004c80 comp rx=0 tx=0).stop 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 shutdown_connections 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0c0384b0 0x7fef0c03a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 --2- 172.21.15.44:0/37099818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef1c106810 0x7fef1c073090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 >> 172.21.15.44:0/37099818 conn(0x7fef1c0fc320 msgr2=0x7fef1c108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 shutdown_connections 2024-04-22T23:37:51.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:51.617+0000 7fef23172700 1 -- 172.21.15.44:0/37099818 wait complete. 2024-04-22T23:37:51.612 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:51.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:51.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:51.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:51.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:51.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:37:51.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:37:51.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:51 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:37:52.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:37:52.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:37:52.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:37:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:52 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/37099818' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:53.192 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:53.192 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:53.405 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 -- 172.21.15.44:0/3540117772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 msgr2=0x7fc640106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 --2- 172.21.15.44:0/3540117772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc640106c00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc63000b0a0 tx=0x7fc63000b3b0 comp rx=0 tx=0).stop 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 -- 172.21.15.44:0/3540117772 shutdown_connections 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 --2- 172.21.15.44:0/3540117772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc640106c00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 -- 172.21.15.44:0/3540117772 >> 172.21.15.44:0/3540117772 conn(0x7fc640075290 msgr2=0x7fc6400756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 -- 172.21.15.44:0/3540117772 shutdown_connections 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.122+0000 7fc64487c700 1 -- 172.21.15.44:0/3540117772 wait complete. 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.123+0000 7fc64487c700 1 Processor -- start 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.123+0000 7fc64487c700 1 -- start start 2024-04-22T23:37:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.123+0000 7fc64487c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc64019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:55.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.123+0000 7fc64487c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64019c060 con 0x7fc640106820 2024-04-22T23:37:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc63e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc64019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc63e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc64019bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:57698/0 (socket says 172.21.15.44:57698) 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc63e59c700 1 -- 172.21.15.44:0/1402648942 learned_addr learned my addr 172.21.15.44:0/1402648942 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc63e59c700 1 -- 172.21.15.44:0/1402648942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc630009d00 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc63e59c700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc64019bb20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc63000d010 tx=0x7fc630003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63000e040 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64019c260 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc6300044f0 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.124+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63001d920 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.125+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc640195b10 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.125+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fc63001dba0 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.125+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6400623c0 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.126+0000 7fc62f7fe700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc6280384b0 0x7fc62803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.126+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc63004c510 con 0x7fc640106820 2024-04-22T23:37:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.126+0000 7fc63dd9b700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc6280384b0 0x7fc62803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:55.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.126+0000 7fc63dd9b700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc6280384b0 0x7fc62803a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc634006fd0 tx=0x7fc634006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:55.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.131+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc630011570 con 0x7fc640106820 2024-04-22T23:37:55.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.427+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6401965a0 con 0x7fc640106820 2024-04-22T23:37:55.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.428+0000 7fc62f7fe700 1 -- 172.21.15.44:0/1402648942 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc630011390 con 0x7fc640106820 2024-04-22T23:37:55.422 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:55.422 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc6280384b0 msgr2=0x7fc62803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc6280384b0 0x7fc62803a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc634006fd0 tx=0x7fc634006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 msgr2=0x7fc64019bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc64019bb20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc63000d010 tx=0x7fc630003ad0 comp rx=0 tx=0).stop 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 shutdown_connections 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc6280384b0 0x7fc62803a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 --2- 172.21.15.44:0/1402648942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc640106820 0x7fc64019bb20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 >> 172.21.15.44:0/1402648942 conn(0x7fc640075290 msgr2=0x7fc6400fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 shutdown_connections 2024-04-22T23:37:55.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:55.430+0000 7fc64487c700 1 -- 172.21.15.44:0/1402648942 wait complete. 2024-04-22T23:37:55.425 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:37:56.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/1402648942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:37:56.999 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:37:56.999 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:37:57.253 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:57.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:57.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:37:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:37:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-22T23:37:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:57.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:37:57 smithi031 ceph-mon[26721]: Deploying daemon crash.smithi044 on smithi044 2024-04-22T23:37:59.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 -- 172.21.15.44:0/2503705106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 msgr2=0x7f7eac105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 --2- 172.21.15.44:0/2503705106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac105b00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009910 tx=0x7f7e9c009c20 comp rx=0 tx=0).stop 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 -- 172.21.15.44:0/2503705106 shutdown_connections 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 --2- 172.21.15.44:0/2503705106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac105b00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 -- 172.21.15.44:0/2503705106 >> 172.21.15.44:0/2503705106 conn(0x7f7eac0fb230 msgr2=0x7f7eac0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 -- 172.21.15.44:0/2503705106 shutdown_connections 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.438+0000 7f7eb11e5700 1 -- 172.21.15.44:0/2503705106 wait complete. 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eb11e5700 1 Processor -- start 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eb11e5700 1 -- start start 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eb11e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eb11e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac1984b0 con 0x7f7eac105720 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eaad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:59.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eaad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:57736/0 (socket says 172.21.15.44:57736) 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.439+0000 7f7eaad9d700 1 -- 172.21.15.44:0/1550494598 learned_addr learned my addr 172.21.15.44:0/1550494598 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7eaad9d700 1 -- 172.21.15.44:0/1550494598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0095f0 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7eaad9d700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac197f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005b40 tx=0x7f7e9c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e9c01d070 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac1986b0 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c0044d0 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7e9c003910 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.440+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac198b30 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.441+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f7e9c003b90 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.441+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac0623c0 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.441+0000 7f7ea8d99700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7e980384b0 0x7f7e9803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.441+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7e9c04c460 con 0x7f7eac105720 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.442+0000 7f7ea259c700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7e980384b0 0x7f7e9803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:59.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.442+0000 7f7ea259c700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7e980384b0 0x7f7e9803a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e94006fd0 tx=0x7f7e94006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:59.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.447+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7e9c02a540 con 0x7f7eac105720 2024-04-22T23:37:59.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.736+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7eac19ca20 con 0x7f7eac105720 2024-04-22T23:37:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.737+0000 7f7ea8d99700 1 -- 172.21.15.44:0/1550494598 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7e9c02a360 con 0x7f7eac105720 2024-04-22T23:37:59.731 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:37:59.731 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7e980384b0 msgr2=0x7f7e9803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7e980384b0 0x7f7e9803a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e94006fd0 tx=0x7f7e94006e40 comp rx=0 tx=0).stop 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 msgr2=0x7f7eac197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac197f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005b40 tx=0x7f7e9c004c80 comp rx=0 tx=0).stop 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 shutdown_connections 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7e980384b0 0x7f7e9803a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 --2- 172.21.15.44:0/1550494598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eac105720 0x7f7eac197f70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.739+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 >> 172.21.15.44:0/1550494598 conn(0x7f7eac0fb230 msgr2=0x7f7eac0688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.740+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 shutdown_connections 2024-04-22T23:37:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:37:59.740+0000 7f7eb11e5700 1 -- 172.21.15.44:0/1550494598 wait complete. 2024-04-22T23:37:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:38:00.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:00.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:00.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:00.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:00.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:00 smithi031 ceph-mon[26721]: Deploying daemon node-exporter.smithi044 on smithi044 2024-04-22T23:38:00.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:00 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/1550494598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:01.326 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:38:01.326 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:38:01.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:01 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:01.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:01 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:01.569 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 -- 172.21.15.44:0/2103939557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 msgr2=0x7faa2c106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:04.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 --2- 172.21.15.44:0/2103939557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c106bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faa28009910 tx=0x7faa28009c20 comp rx=0 tx=0).stop 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 -- 172.21.15.44:0/2103939557 shutdown_connections 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 --2- 172.21.15.44:0/2103939557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c106bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 -- 172.21.15.44:0/2103939557 >> 172.21.15.44:0/2103939557 conn(0x7faa2c0fc320 msgr2=0x7faa2c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 -- 172.21.15.44:0/2103939557 shutdown_connections 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.401+0000 7faa33728700 1 -- 172.21.15.44:0/2103939557 wait complete. 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa33728700 1 Processor -- start 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa33728700 1 -- start start 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa33728700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa33728700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c075010 con 0x7faa2c106810 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa314c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa314c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:43910/0 (socket says 172.21.15.44:43910) 2024-04-22T23:38:04.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa314c4700 1 -- 172.21.15.44:0/718655045 learned_addr learned my addr 172.21.15.44:0/718655045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.402+0000 7faa314c4700 1 -- 172.21.15.44:0/718655045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280095f0 con 0x7faa2c106810 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.403+0000 7faa314c4700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c073090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa28005b40 tx=0x7faa28004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.403+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa2801d070 con 0x7faa2c106810 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.403+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa280044d0 con 0x7faa2c106810 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.403+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c0735d0 con 0x7faa2c106810 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.403+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa28003930 con 0x7faa2c106810 2024-04-22T23:38:04.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.403+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c073a50 con 0x7faa2c106810 2024-04-22T23:38:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.404+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c104ac0 con 0x7faa2c106810 2024-04-22T23:38:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.404+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7faa28003bf0 con 0x7faa2c106810 2024-04-22T23:38:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.404+0000 7faa227fc700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa180384b0 0x7faa1803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.404+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7faa2804c4a0 con 0x7faa2c106810 2024-04-22T23:38:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.404+0000 7faa30cc3700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa180384b0 0x7faa1803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:04.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.405+0000 7faa30cc3700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa180384b0 0x7faa1803a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa1c006fd0 tx=0x7faa1c006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:04.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.409+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faa2802a540 con 0x7faa2c106810 2024-04-22T23:38:04.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.702+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faa2c0623c0 con 0x7faa2c106810 2024-04-22T23:38:04.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.702+0000 7faa227fc700 1 -- 172.21.15.44:0/718655045 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7faa2802a360 con 0x7faa2c106810 2024-04-22T23:38:04.697 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:38:04.697 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa180384b0 msgr2=0x7faa1803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa180384b0 0x7faa1803a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa1c006fd0 tx=0x7faa1c006e40 comp rx=0 tx=0).stop 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 msgr2=0x7faa2c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c073090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa28005b40 tx=0x7faa28004c80 comp rx=0 tx=0).stop 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 shutdown_connections 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa180384b0 0x7faa1803a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 --2- 172.21.15.44:0/718655045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa2c106810 0x7faa2c073090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 >> 172.21.15.44:0/718655045 conn(0x7faa2c0fc320 msgr2=0x7faa2c108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 shutdown_connections 2024-04-22T23:38:04.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:04.704+0000 7faa33728700 1 -- 172.21.15.44:0/718655045 wait complete. 2024-04-22T23:38:04.700 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:38:05.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:05.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:05.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:05.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: Deploying daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/718655045' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:05.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:06.376 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:38:06.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:38:06.598 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:07.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:06 smithi031 ceph-mon[26721]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:08.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4098100777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 msgr2=0x7faa68105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:08.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 --2- 172.21.15.44:0/4098100777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa68105b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faa58009910 tx=0x7faa58009c20 comp rx=0 tx=0).stop 2024-04-22T23:38:08.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4098100777 shutdown_connections 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 --2- 172.21.15.44:0/4098100777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa68105b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4098100777 >> 172.21.15.44:0/4098100777 conn(0x7faa680fb250 msgr2=0x7faa680fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4098100777 shutdown_connections 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4098100777 wait complete. 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 Processor -- start 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.098+0000 7faa6e1a8700 1 -- start start 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.099+0000 7faa6e1a8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa680fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.099+0000 7faa6e1a8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa680fd680 con 0x7faa68105720 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.099+0000 7faa677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa680fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.099+0000 7faa677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa680fb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:43936/0 (socket says 172.21.15.44:43936) 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.099+0000 7faa677fe700 1 -- 172.21.15.44:0/4183208902 learned_addr learned my addr 172.21.15.44:0/4183208902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:38:08.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.099+0000 7faa677fe700 1 -- 172.21.15.44:0/4183208902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580095f0 con 0x7faa68105720 2024-04-22T23:38:08.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.100+0000 7faa677fe700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa680fb700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa58005b40 tx=0x7faa58005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.100+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa5801d070 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.100+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa580044d0 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.100+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa680fbc40 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.100+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa58003a40 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.100+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa680fc0c0 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.101+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa680623c0 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.101+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7faa5800b960 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.101+0000 7faa657fa700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa540384b0 0x7faa5403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.101+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7faa5804c530 con 0x7faa68105720 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.101+0000 7faa5effd700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa540384b0 0x7faa5403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:08.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.102+0000 7faa5effd700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa540384b0 0x7faa5403a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa50006fd0 tx=0x7faa50006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:08.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.106+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faa5802a540 con 0x7faa68105720 2024-04-22T23:38:08.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.410+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faa680fc460 con 0x7faa68105720 2024-04-22T23:38:08.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.411+0000 7faa657fa700 1 -- 172.21.15.44:0/4183208902 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7faa5802a360 con 0x7faa68105720 2024-04-22T23:38:08.405 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:38:08.406 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:38:08.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa540384b0 msgr2=0x7faa5403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa540384b0 0x7faa5403a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa50006fd0 tx=0x7faa50006e40 comp rx=0 tx=0).stop 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 msgr2=0x7faa680fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa680fb700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faa58005b40 tx=0x7faa58005000 comp rx=0 tx=0).stop 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 shutdown_connections 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faa540384b0 0x7faa5403a970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 --2- 172.21.15.44:0/4183208902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa68105720 0x7faa680fb700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 >> 172.21.15.44:0/4183208902 conn(0x7faa680fb250 msgr2=0x7faa68107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 shutdown_connections 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:08.413+0000 7faa6e1a8700 1 -- 172.21.15.44:0/4183208902 wait complete. 2024-04-22T23:38:08.408 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:38:09.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:09.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/4183208902' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:09.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:09.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:09.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:09.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:09.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:09.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:10.073 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:38:10.074 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:38:10.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:09 smithi031 ceph-mon[26721]: Deploying daemon mon.smithi044 on smithi044 2024-04-22T23:38:10.808 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:11.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:10 smithi031 ceph-mon[26721]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:11.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:10 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:12.417 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:12 smithi044 systemd[1]: Starting Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 -- 172.21.15.44:0/2497477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 msgr2=0x7f5d5c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 --2- 172.21.15.44:0/2497477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c106be0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009910 tx=0x7f5d4c009c20 comp rx=0 tx=0).stop 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 -- 172.21.15.44:0/2497477935 shutdown_connections 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 --2- 172.21.15.44:0/2497477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c106be0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 -- 172.21.15.44:0/2497477935 >> 172.21.15.44:0/2497477935 conn(0x7f5d5c0fc2f0 msgr2=0x7f5d5c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 -- 172.21.15.44:0/2497477935 shutdown_connections 2024-04-22T23:38:12.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.792+0000 7f5d60cb2700 1 -- 172.21.15.44:0/2497477935 wait complete. 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.793+0000 7f5d60cb2700 1 Processor -- start 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.793+0000 7f5d60cb2700 1 -- start start 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.793+0000 7f5d60cb2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.793+0000 7f5d60cb2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c196250 con 0x7f5d5c106800 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d5a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d5a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.44:56456/0 (socket says 172.21.15.44:56456) 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d5a59c700 1 -- 172.21.15.44:0/990900491 learned_addr learned my addr 172.21.15.44:0/990900491 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d5a59c700 1 -- 172.21.15.44:0/990900491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0095f0 con 0x7f5d5c106800 2024-04-22T23:38:12.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d5a59c700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c195d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005b40 tx=0x7f5d4c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d4c01d070 con 0x7f5d5c106800 2024-04-22T23:38:12.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d4c0044d0 con 0x7f5d5c106800 2024-04-22T23:38:12.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.794+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c196450 con 0x7f5d5c106800 2024-04-22T23:38:12.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.795+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d4c003910 con 0x7f5d5c106800 2024-04-22T23:38:12.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.795+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1968d0 con 0x7f5d5c106800 2024-04-22T23:38:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.795+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c0623c0 con 0x7f5d5c106800 2024-04-22T23:38:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.796+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f5d4c003b90 con 0x7f5d5c106800 2024-04-22T23:38:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.796+0000 7f5d4b7fe700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5d440384b0 0x7f5d4403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.796+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5d4c04c420 con 0x7f5d5c106800 2024-04-22T23:38:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.796+0000 7f5d59d9b700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5d440384b0 0x7f5d4403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.797+0000 7f5d59d9b700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5d440384b0 0x7f5d4403a970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006fd0 tx=0x7f5d50006e40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:12.801+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5d4c020b20 con 0x7f5d5c106800 2024-04-22T23:38:13.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.101+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5d5c104610 con 0x7f5d5c106800 2024-04-22T23:38:13.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.101+0000 7f5d4b7fe700 1 -- 172.21.15.44:0/990900491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5d4c020b20 con 0x7f5d5c106800 2024-04-22T23:38:13.096 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:38:13.096 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:35:31.547688Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:38:13.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5d440384b0 msgr2=0x7f5d4403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5d440384b0 0x7f5d4403a970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006fd0 tx=0x7f5d50006e40 comp rx=0 tx=0).stop 2024-04-22T23:38:13.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 msgr2=0x7f5d5c195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c195d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005b40 tx=0x7f5d4c004c80 comp rx=0 tx=0).stop 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 shutdown_connections 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5d440384b0 0x7f5d4403a970 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 --2- 172.21.15.44:0/990900491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d5c106800 0x7f5d5c195d10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 >> 172.21.15.44:0/990900491 conn(0x7f5d5c0fc2f0 msgr2=0x7f5d5c108bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 shutdown_connections 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:13.104+0000 7f5d60cb2700 1 -- 172.21.15.44:0/990900491 wait complete. 2024-04-22T23:38:13.099 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 1 2024-04-22T23:38:13.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:12 smithi031 ceph-mon[26721]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:13.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 podman[34422]: 2024-04-22 23:38:12.931955648 +0000 UTC m=+0.026446514 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3 2024-04-22T23:38:13.437 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 podman[34422]: 2024-04-22 23:38:13.054479267 +0000 UTC m=+0.148970105 container create 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, ceph=True) 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 podman[34422]: 2024-04-22 23:38:13.727550079 +0000 UTC m=+0.822040928 container init 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, ceph=True, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 podman[34422]: 2024-04-22 23:38:13.730703387 +0000 UTC m=+0.825194233 container start 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222) 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pidfile_write: ignore empty --pid-file 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: load: jerasure load: lrc 2024-04-22T23:38:13.999 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: RocksDB version: 7.9.2 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Git sha 0 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Compile date 2023-12-11 22:07:34 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: DB SUMMARY 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: DB Session ID: NIK1Y5WXPVYZY6PYMRNH 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi044/store.db dir, Total Num: 0, files: 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi044/store.db: 000004.log size: 511 ; 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:38:14.000 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.env: 0x55559926e720 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.info_log: 0x55559c80f2e0 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:38:14.001 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.statistics: (nil) 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.use_fsync: 0 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:38:14.002 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.db_log_dir: 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.wal_dir: 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:38:14.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.write_buffer_manager: 0x55559ba80140 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.unordered_write: 0 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:38:14.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.row_cache: None 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.wal_filter: None 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.wal_compression: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:38:14.005 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:38:14.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_open_files: -1 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Compression algorithms supported: 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kZSTD supported: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:38:14.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000005 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.merge_operator: 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_filter: None 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55559c80f3e0) 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:38:14.008 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_type: 0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_index_type: 0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_shortening: 1 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: checksum: 4 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: no_block_cache: 0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache: 0x55559bb211f0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_options: 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: capacity : 536870912 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_shard_bits : 4 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: strict_capacity_limit : 0 2024-04-22T23:38:14.009 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_compressed: (nil) 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: persistent_cache: (nil) 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size: 4096 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size_deviation: 10 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_restart_interval: 16 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_block_restart_interval: 1 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: metadata_block_size: 4096 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: partition_filters: 0 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: use_delta_encoding: 1 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: filter_policy: bloomfilter 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: whole_key_filtering: 1 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: verify_compression: 0 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: format_version: 5 2024-04-22T23:38:14.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_index_compression: 1 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_align: 0 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_auto_readahead_size: 262144 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: prepopulate_block_cache: 0 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: initial_auto_readahead_size: 8192 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression: NoCompression 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.num_levels: 7 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:38:14.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:38:14.012 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:38:14.013 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:38:14.015 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:38:14.015 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:38:14.015 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:38:14.015 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:38:14.015 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:38:14.016 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.table_properties_collectors: 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.ttl: 2592000 2024-04-22T23:38:14.017 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-22T23:38:14.018 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 39138f31-451a-4300-beb3-f3ad6c38bf76 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829093760530, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829093761137, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713829093, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "39138f31-451a-4300-beb3-f3ad6c38bf76", "db_session_id": "NIK1Y5WXPVYZY6PYMRNH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829093761248, "job": 1, "event": "recovery_finished"} 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi044/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55559bbbe000 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: DB pointer 0x55559bbaa000 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** DB Stats ** 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:38:14.019 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T23:38:14.020 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache BinnedLRUCache@0x55559bb211f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044 does not exist in monmap, will attempt to join an existing cluster 2024-04-22T23:38:14.021 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: using public_addr v2:172.21.15.44:0/0 -> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: starting mon.smithi044 rank -1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi044 fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(???) e0 preinit fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).mds e1 new map 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).mds e1 print_map 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: e1 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: legacy client fscid: -1 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: No filesystems configured 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-22T23:38:14.022 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.023 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/250095437' entity='client.admin' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon crash.smithi031 on smithi031 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.024 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying cephadm binary to smithi044 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Added host smithi044 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.025 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/366366641' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/366366641' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-22T23:38:14.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon grafana.smithi031 on smithi031 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Saving service mon spec with placement smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044;count:2 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/3171010987' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/205051826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/2848238467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/3636670449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/3084548066' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.028 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/2967856242' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/1177221197' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14162 172.21.15.31:0/1543487688' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mgrmap e14: smithi031.fazdcv(active, since 58s) 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/1661022376' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/3079550682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Active manager daemon smithi031.fazdcv restarted 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:38:14.029 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mgrmap e15: smithi031.fazdcv(active, starting, since 0.0043203s) 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: [22/Apr/2024:23:37:45] ENGINE Bus STARTING 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: [22/Apr/2024:23:37:45] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:38:14.030 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: [22/Apr/2024:23:37:45] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: [22/Apr/2024:23:37:45] ENGINE Bus STARTED 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mgrmap e16: smithi031.fazdcv(active, since 1.00645s) 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/144794986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mgrmap e17: smithi031.fazdcv(active, since 2s) 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.031 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.032 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/37099818' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/1402648942' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon crash.smithi044 on smithi044 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon node-exporter.smithi044 on smithi044 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/1550494598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:14.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/718655045' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/4183208902' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: Deploying daemon mon.smithi044 on smithi044 2024-04-22T23:38:14.035 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: mon.smithi044@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expand map: {default=false} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta from 'false' to 'false' 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expanded map: {default=false} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expand map: {default=info} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta from 'info' to 'info' 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expanded map: {default=info} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expand map: {default=daemon} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta from 'daemon' to 'daemon' 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expanded map: {default=daemon} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expand map: {default=debug} 2024-04-22T23:38:14.036 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta from 'debug' to 'debug' 2024-04-22T23:38:14.037 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 ceph-mon[34463]: expand_channel_meta expanded map: {default=debug} 2024-04-22T23:38:14.037 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 bash[34422]: 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 2024-04-22T23:38:14.037 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:13 smithi044 systemd[1]: Started Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:38:14.968 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:38:14.969 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:38:15.176 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: mon.smithi031 calling monitor election 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:19.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: mon.smithi044 calling monitor election 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: mon.smithi031 is new leader, mons smithi031,smithi044 in quorum (ranks 0,1) 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: monmap e2: 2 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],smithi044=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: fsmap 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: mgrmap e17: smithi031.fazdcv(active, since 33s) 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:38:19.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: overall HEALTH_OK 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:18 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: mon.smithi031 calling monitor election 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: mon.smithi044 calling monitor election 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:19.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: mon.smithi031 is new leader, mons smithi031,smithi044 in quorum (ranks 0,1) 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: monmap e2: 2 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],smithi044=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: fsmap 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: mgrmap e17: smithi031.fazdcv(active, since 33s) 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: overall HEALTH_OK 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/85218492' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:18 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/2825127482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40035c0 msgr2=0x7fbad4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/2825127482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40035c0 0x7fbad4005a50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbae4076780 tx=0x7fbad800feb0 comp rx=0 tx=0).stop 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/2825127482 shutdown_connections 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/2825127482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40035c0 0x7fbad4005a50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/2825127482 >> 172.21.15.44:0/2825127482 conn(0x7fbae4075340 msgr2=0x7fbae4075750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/2825127482 shutdown_connections 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.732+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/2825127482 wait complete. 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbaeaf1e700 1 Processor -- start 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbaeaf1e700 1 -- start start 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbaeaf1e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:19.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbaeaf1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbae40fff00 0x7fbae4100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbaeaf1e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae41a20e0 con 0x7fbae40fff00 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbaeaf1e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae41a2220 con 0x7fbad40035c0 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43538/0 (socket says 172.21.15.44:43538) 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 -- 172.21.15.44:0/3795553426 learned_addr learned my addr 172.21.15.44:0/3795553426 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae3fff700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbae40fff00 0x7fbae4100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 -- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 msgr2=0x7fbae40ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 -- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 msgr2=0x7fbae40ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.733+0000 7fbae8cba700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:38:19.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae3fff700 1 -- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 msgr2=0x7fbae40ff9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae3fff700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae3fff700 1 -- 172.21.15.44:0/3795553426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad800f900 con 0x7fbae40fff00 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae3fff700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbae40fff00 0x7fbae4100380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbad000b500 tx=0x7fbad000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad0011800 con 0x7fbae40fff00 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbad0011e10 con 0x7fbae40fff00 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41a2360 con 0x7fbae40fff00 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad000f2e0 con 0x7fbae40fff00 2024-04-22T23:38:19.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.734+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae41a28b0 con 0x7fbae40fff00 2024-04-22T23:38:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.736+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae40623c0 con 0x7fbae40fff00 2024-04-22T23:38:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.736+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7fbad000f440 con 0x7fbae40fff00 2024-04-22T23:38:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.736+0000 7fbae1ffb700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbac806c5b0 0x7fbac806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.737+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fbad0089c30 con 0x7fbae40fff00 2024-04-22T23:38:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.737+0000 7fbae8cba700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbac806c5b0 0x7fbac806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:19.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.737+0000 7fbae8cba700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbac806c5b0 0x7fbac806ea70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005b40 tx=0x7fbad8005a90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:19.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:19.741+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbad00554d0 con 0x7fbae40fff00 2024-04-22T23:38:20.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.028+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbae41a2cf0 con 0x7fbae40fff00 2024-04-22T23:38:20.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.029+0000 7fbae1ffb700 1 -- 172.21.15.44:0/3795553426 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7fbad0058af0 con 0x7fbae40fff00 2024-04-22T23:38:20.023 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T23:38:20.023 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","modified":"2024-04-22T23:38:13.792510Z","created":"2024-04-22T23:35:31.547688Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi044","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T23:38:20.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbac806c5b0 msgr2=0x7fbac806ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbac806c5b0 0x7fbac806ea70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005b40 tx=0x7fbad8005a90 comp rx=0 tx=0).stop 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbae40fff00 msgr2=0x7fbae4100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbae40fff00 0x7fbae4100380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbad000b500 tx=0x7fbad000b8c0 comp rx=0 tx=0).stop 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 shutdown_connections 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbac806c5b0 0x7fbac806ea70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbad40035c0 0x7fbae40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 --2- 172.21.15.44:0/3795553426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbae40fff00 0x7fbae4100380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 >> 172.21.15.44:0/3795553426 conn(0x7fbae4075340 msgr2=0x7fbae418e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 shutdown_connections 2024-04-22T23:38:20.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:38:20.032+0000 7fbaeaf1e700 1 -- 172.21.15.44:0/3795553426 wait complete. 2024-04-22T23:38:20.028 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 2 2024-04-22T23:38:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:19 smithi031 ceph-mon[26721]: mgrmap e18: smithi031.fazdcv(active, since 33s), standbys: smithi044.iglcti 2024-04-22T23:38:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:19 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:38:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:19 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:19 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:20.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:19 smithi044 ceph-mon[34463]: mgrmap e18: smithi031.fazdcv(active, since 33s), standbys: smithi044.iglcti 2024-04-22T23:38:20.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:19 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:38:20.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:19 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:20.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:19 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:38:20.612 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-22T23:38:20.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-22T23:38:20.834 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:38:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:20 smithi031 ceph-mon[26721]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:20 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/3795553426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:20 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:21.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:20 smithi044 ceph-mon[34463]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:21.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:20 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/3795553426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:38:21.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:20 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 -- 172.21.15.31:0/1832831744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 msgr2=0x7fd3e8101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 --2- 172.21.15.31:0/1832831744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8101da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009910 tx=0x7fd3dc009c20 comp rx=0 tx=0).stop 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 -- 172.21.15.31:0/1832831744 shutdown_connections 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 --2- 172.21.15.31:0/1832831744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8101da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 -- 172.21.15.31:0/1832831744 >> 172.21.15.31:0/1832831744 conn(0x7fd3e80fb210 msgr2=0x7fd3e80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 -- 172.21.15.31:0/1832831744 shutdown_connections 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.612+0000 7fd3effce700 1 -- 172.21.15.31:0/1832831744 wait complete. 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3effce700 1 Processor -- start 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3effce700 1 -- start start 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3effce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:22.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3effce700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e8196460 0x7fd3e819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3effce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8196a80 con 0x7fd3e81019c0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3effce700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8196bf0 con 0x7fd3e8196460 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3edd6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3edd6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37724/0 (socket says 172.21.15.31:37724) 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.613+0000 7fd3edd6a700 1 -- 172.21.15.31:0/2405879252 learned_addr learned my addr 172.21.15.31:0/2405879252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3edd6a700 1 -- 172.21.15.31:0/2405879252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e8196460 msgr2=0x7fd3e819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3edd6a700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e8196460 0x7fd3e819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3edd6a700 1 -- 172.21.15.31:0/2405879252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0095f0 con 0x7fd3e81019c0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3edd6a700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8195f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005b40 tx=0x7fd3dc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3dc01d070 con 0x7fd3e81019c0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e819ae70 con 0x7fd3e81019c0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd3dc005140 con 0x7fd3e81019c0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3dc017440 con 0x7fd3e81019c0 2024-04-22T23:38:22.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.614+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e819b3c0 con 0x7fd3e81019c0 2024-04-22T23:38:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.615+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e8108010 con 0x7fd3e81019c0 2024-04-22T23:38:22.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.616+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7fd3dc00f6f0 con 0x7fd3e81019c0 2024-04-22T23:38:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.616+0000 7fd3daffd700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3d406c5c0 0x7fd3d406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.616+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fd3dc08bbd0 con 0x7fd3e81019c0 2024-04-22T23:38:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.616+0000 7fd3ed569700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3d406c5c0 0x7fd3d406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:22.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.616+0000 7fd3ed569700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3d406c5c0 0x7fd3d406ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098e0 tx=0x7fd3e4006dd0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:22.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.621+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd3dc057490 con 0x7fd3e81019c0 2024-04-22T23:38:22.686 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:22.686 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:22.687 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:22.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:22.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:22.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:22.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.824+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd3e80623c0 con 0x7fd3e81019c0 2024-04-22T23:38:22.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.824+0000 7fd3daffd700 1 -- 172.21.15.31:0/2405879252 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7fd3dc08b390 con 0x7fd3e81019c0 2024-04-22T23:38:22.819 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:38:22.819 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-04-22T23:38:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:38:22.819 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-04-22T23:38:22.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3d406c5c0 msgr2=0x7fd3d406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:22.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3d406c5c0 0x7fd3d406ea80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40098e0 tx=0x7fd3e4006dd0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 msgr2=0x7fd3e8195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8195f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005b40 tx=0x7fd3dc00bda0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 shutdown_connections 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3d406c5c0 0x7fd3d406ea80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e81019c0 0x7fd3e8195f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 --2- 172.21.15.31:0/2405879252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3e8196460 0x7fd3e819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 >> 172.21.15.31:0/2405879252 conn(0x7fd3e80fb210 msgr2=0x7fd3e8105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.826+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 shutdown_connections 2024-04-22T23:38:22.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:22.827+0000 7fd3effce700 1 -- 172.21.15.31:0/2405879252 wait complete. 2024-04-22T23:38:23.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:23 smithi031 ceph-mon[26721]: Reconfiguring mon.smithi031 (unknown last config time)... 2024-04-22T23:38:23.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:23 smithi031 ceph-mon[26721]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-22T23:38:23.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:23 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2405879252' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:23.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:23 smithi044 ceph-mon[34463]: Reconfiguring mon.smithi031 (unknown last config time)... 2024-04-22T23:38:23.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:23 smithi044 ceph-mon[34463]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-22T23:38:23.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:23 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2405879252' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:24.411 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-22T23:38:24.411 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:38:24.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T23:38:24.441 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:38:24.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:24.505 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:38:24.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T23:38:24.537 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:38:24.537 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:24.604 INFO:tasks.cephadm:Deploying OSDs... 2024-04-22T23:38:24.604 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:38:24.604 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T23:38:24.620 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T23:38:24.620 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-22 23:37:36.227248765 +0000 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-22 23:32:32.815848021 +0000 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-22 23:32:32.815848021 +0000 2024-04-22T23:38:24.677 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-22 23:32:32.815848021 +0000 2024-04-22T23:38:24.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T23:38:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-22T23:38:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-22T23:38:24.741 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00020266 s, 2.5 MB/s 2024-04-22T23:38:24.742 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T23:38:24.800 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-04-22T23:38:24.855 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-22 23:37:36.228248748 +0000 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-22 23:32:33.181841670 +0000 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-22 23:32:33.181841670 +0000 2024-04-22T23:38:24.856 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-22 23:32:33.181841670 +0000 2024-04-22T23:38:24.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T23:38:24.918 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-22T23:38:24.918 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-22T23:38:24.918 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000202731 s, 2.5 MB/s 2024-04-22T23:38:24.920 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T23:38:24.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:24 smithi031 ceph-mon[26721]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:24.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:24 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:24.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:24 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:24.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:24 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:24.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:24 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:24.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:24 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:24.961 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-04-22T23:38:24.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:24 smithi044 ceph-mon[34463]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:24.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:24 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:24.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:24 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:24.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:24 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:24.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:24 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:24.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:24 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-22 23:37:36.228248748 +0000 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-22 23:32:33.538835477 +0000 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-22 23:32:33.538835477 +0000 2024-04-22T23:38:25.017 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-22 23:32:33.538835477 +0000 2024-04-22T23:38:25.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T23:38:25.081 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-22T23:38:25.081 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-22T23:38:25.081 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000229183 s, 2.2 MB/s 2024-04-22T23:38:25.082 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T23:38:25.138 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-22 23:37:36.229248731 +0000 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-22 23:32:33.885829456 +0000 2024-04-22T23:38:25.194 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-22 23:32:33.885829456 +0000 2024-04-22T23:38:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-22 23:32:33.885829456 +0000 2024-04-22T23:38:25.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T23:38:25.257 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-22T23:38:25.257 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-22T23:38:25.257 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000206281 s, 2.5 MB/s 2024-04-22T23:38:25.258 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T23:38:25.315 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:38:25.315 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T23:38:25.334 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T23:38:25.334 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 23:38:12.286872781 +0000 2024-04-22T23:38:25.392 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 23:32:32.810700370 +0000 2024-04-22T23:38:25.393 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 23:32:32.810700370 +0000 2024-04-22T23:38:25.393 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 23:32:32.810700370 +0000 2024-04-22T23:38:25.393 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T23:38:25.459 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T23:38:25.459 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T23:38:25.459 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000233169 s, 2.2 MB/s 2024-04-22T23:38:25.461 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T23:38:25.519 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-22T23:38:25.577 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T23:38:25.577 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:25.577 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T23:38:25.577 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:25.577 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:25.578 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 23:38:12.286872781 +0000 2024-04-22T23:38:25.578 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 23:32:33.173694426 +0000 2024-04-22T23:38:25.578 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 23:32:33.173694426 +0000 2024-04-22T23:38:25.578 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 23:32:33.173694426 +0000 2024-04-22T23:38:25.578 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T23:38:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T23:38:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T23:38:25.644 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000326597 s, 1.6 MB/s 2024-04-22T23:38:25.646 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T23:38:25.704 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 23:38:12.286872781 +0000 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 23:32:33.525688676 +0000 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 23:32:33.525688676 +0000 2024-04-22T23:38:25.762 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 23:32:33.525688676 +0000 2024-04-22T23:38:25.763 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T23:38:25.827 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T23:38:25.828 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T23:38:25.828 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000247256 s, 2.1 MB/s 2024-04-22T23:38:25.829 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T23:38:25.886 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 23:38:12.287872765 +0000 2024-04-22T23:38:25.943 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 23:32:33.872683017 +0000 2024-04-22T23:38:25.944 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 23:32:33.872683017 +0000 2024-04-22T23:38:25.944 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 23:32:33.872683017 +0000 2024-04-22T23:38:25.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T23:38:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T23:38:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T23:38:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000227034 s, 2.3 MB/s 2024-04-22T23:38:26.008 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T23:38:26.065 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-04-22T23:38:26.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T23:38:26.136 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:25 smithi044 ceph-mon[34463]: Reconfiguring mgr.smithi031.fazdcv (unknown last config time)... 2024-04-22T23:38:26.136 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:25 smithi044 ceph-mon[34463]: Reconfiguring daemon mgr.smithi031.fazdcv on smithi031 2024-04-22T23:38:26.136 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:25 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:26.136 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:25 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:26.136 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:25 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:38:26.136 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:25 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:26.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:25 smithi031 ceph-mon[26721]: Reconfiguring mgr.smithi031.fazdcv (unknown last config time)... 2024-04-22T23:38:26.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:25 smithi031 ceph-mon[26721]: Reconfiguring daemon mgr.smithi031.fazdcv on smithi031 2024-04-22T23:38:26.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:25 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:26.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:25 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:26.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:25 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:38:26.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:25 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:26.911 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:38:27.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:26 smithi031 ceph-mon[26721]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:27.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:26 smithi031 ceph-mon[26721]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-04-22T23:38:27.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:26 smithi031 ceph-mon[26721]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:38:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:26 smithi044 ceph-mon[34463]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:26 smithi044 ceph-mon[34463]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-04-22T23:38:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:26 smithi044 ceph-mon[34463]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:38:27.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:27.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:27.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:27 smithi031 ceph-mon[26721]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-22T23:38:27.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:27.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:27.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:27 smithi031 ceph-mon[26721]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-22T23:38:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:27 smithi044 ceph-mon[34463]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-22T23:38:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:27 smithi044 ceph-mon[34463]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-22T23:38:29.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:28 smithi031 ceph-mon[26721]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:29.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:28 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:29.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:28 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:29.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:28 smithi044 ceph-mon[34463]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:29.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:28 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:29.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:28 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:30.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:29 smithi031 ceph-mon[26721]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-04-22T23:38:30.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:29 smithi031 ceph-mon[26721]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-04-22T23:38:30.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:29 smithi044 ceph-mon[34463]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-04-22T23:38:30.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:29 smithi044 ceph-mon[34463]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-04-22T23:38:31.163 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:30 smithi031 ceph-mon[26721]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:31.163 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:31.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:30 smithi044 ceph-mon[34463]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:31.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T23:38:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T23:38:31.496 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-22T23:38:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-22T23:38:31.496 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112742 s, 930 MB/s 2024-04-22T23:38:31.496 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 2024-04-22T23:38:31.497 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-22T23:38:32.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-04-22T23:38:33.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:32 smithi031 ceph-mon[26721]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:33.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:32 smithi044 ceph-mon[34463]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:33.401 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:38:35.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:34 smithi031 ceph-mon[26721]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:35.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:34 smithi044 ceph-mon[34463]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 -- 172.21.15.31:0/1862135392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 msgr2=0x7f5ce80fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 --2- 172.21.15.31:0/1862135392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce80fffd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009910 tx=0x7f5cd8009c20 comp rx=0 tx=0).stop 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 -- 172.21.15.31:0/1862135392 shutdown_connections 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 --2- 172.21.15.31:0/1862135392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce8100db0 0x7f5ce8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 --2- 172.21.15.31:0/1862135392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce80fffd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 -- 172.21.15.31:0/1862135392 >> 172.21.15.31:0/1862135392 conn(0x7f5ce80fb150 msgr2=0x7f5ce80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:35.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 -- 172.21.15.31:0/1862135392 shutdown_connections 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.308+0000 7f5ced352700 1 -- 172.21.15.31:0/1862135392 wait complete. 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ced352700 1 Processor -- start 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ced352700 1 -- start start 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ced352700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce8069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ced352700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce8100db0 0x7f5ce819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ced352700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81970b0 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ced352700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce819db80 con 0x7f5ce8100db0 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ce6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce8069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ce6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce8069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43142/0 (socket says 172.21.15.31:43142) 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ce6ffd700 1 -- 172.21.15.31:0/2763556504 learned_addr learned my addr 172.21.15.31:0/2763556504 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.309+0000 7f5ce67fc700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce8100db0 0x7f5ce819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5ce6ffd700 1 -- 172.21.15.31:0/2763556504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce8100db0 msgr2=0x7f5ce819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5ce6ffd700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce8100db0 0x7f5ce819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5ce6ffd700 1 -- 172.21.15.31:0/2763556504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80095f0 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5ce6ffd700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce8069970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800bdd0 tx=0x7f5cd800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd801d070 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5cd800f460 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce819ddd0 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd8017630 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.310+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce819e2c0 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.311+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce8108710 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.312+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f5cd8003a60 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.312+0000 7f5cdffff700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5cd406c460 0x7f5cd406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.312+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5cd808bc30 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.312+0000 7f5ce67fc700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5cd406c460 0x7f5cd406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.313+0000 7f5ce67fc700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5cd406c460 0x7f5cd406e920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8069ef0 tx=0x7f5cd000b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.318+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5cd805af80 con 0x7f5ce80ffbb0 2024-04-22T23:38:35.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:38:35.538+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ce8105700 con 0x7f5cd406c460 2024-04-22T23:38:36.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:36.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:38:36.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:36.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:38:36.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:38:36.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:38.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:38 smithi044 ceph-mon[34463]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:38.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:38 smithi031 ceph-mon[26721]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:41.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:40 smithi031 ceph-mon[26721]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:41.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:41.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:41.155 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:40 smithi044 ceph-mon[34463]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:41.155 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:41.155 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:42.145 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:41 smithi031 ceph-mon[26721]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-22T23:38:42.145 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:41 smithi031 ceph-mon[26721]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-22T23:38:42.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:41 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2313695144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a20f711-3635-473e-a87d-02f16913468e"}]: dispatch 2024-04-22T23:38:42.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:41 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2313695144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a20f711-3635-473e-a87d-02f16913468e"}]': finished 2024-04-22T23:38:42.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:41 smithi031 ceph-mon[26721]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T23:38:42.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:38:42.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:41 smithi044 ceph-mon[34463]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-22T23:38:42.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:41 smithi044 ceph-mon[34463]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-22T23:38:42.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:41 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2313695144' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a20f711-3635-473e-a87d-02f16913468e"}]: dispatch 2024-04-22T23:38:42.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:41 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2313695144' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a20f711-3635-473e-a87d-02f16913468e"}]': finished 2024-04-22T23:38:42.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:41 smithi044 ceph-mon[34463]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T23:38:42.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:38:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:42 smithi031 ceph-mon[26721]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:42 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2807055938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:38:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:42 smithi044 ceph-mon[34463]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:42 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2807055938' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:38:45.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:44 smithi031 ceph-mon[26721]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:45.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:44 smithi044 ceph-mon[34463]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:45.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:46.161 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:46.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:46 smithi031 ceph-mon[26721]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:46 smithi044 ceph-mon[34463]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:48.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:48 smithi031 ceph-mon[26721]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:49.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:48 smithi044 ceph-mon[34463]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:51.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:50 smithi031 ceph-mon[26721]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:51.167 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:50 smithi044 ceph-mon[34463]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:52.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:52 smithi044 ceph-mon[34463]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:38:52.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:52.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:52.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-04-22T23:38:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:38:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:52.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:52 smithi031 ceph-mon[26721]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:38:53.559 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:38:53.559 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: Deploying daemon osd.0 on smithi031 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: Reconfiguring crash.smithi044 (monmap changed)... 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:53.560 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:53 smithi044 ceph-mon[34463]: Reconfiguring daemon crash.smithi044 on smithi044 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: Deploying daemon osd.0 on smithi031 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: Reconfiguring crash.smithi044 (monmap changed)... 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:53.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:53 smithi031 ceph-mon[26721]: Reconfiguring daemon crash.smithi044 on smithi044 2024-04-22T23:38:55.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:54 smithi044 ceph-mon[34463]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:55.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:55.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:55.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:55.033 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:55.034 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:55.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:54 smithi031 ceph-mon[26721]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:55.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:55.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:55.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:55.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:55.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:56.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: Reconfiguring mgr.smithi044.iglcti (monmap changed)... 2024-04-22T23:38:56.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: Reconfiguring daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:38:56.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:56.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:56.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:56.130 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:56.131 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: Reconfiguring mgr.smithi044.iglcti (monmap changed)... 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: Reconfiguring daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:56.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:57.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:56 smithi031 ceph-mon[26721]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:57.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:56 smithi031 ceph-mon[26721]: Reconfiguring mon.smithi044 (monmap changed)... 2024-04-22T23:38:57.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:56 smithi031 ceph-mon[26721]: Reconfiguring daemon mon.smithi044 on smithi044 2024-04-22T23:38:57.132 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:56 smithi044 ceph-mon[34463]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:57.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:56 smithi044 ceph-mon[34463]: Reconfiguring mon.smithi044 (monmap changed)... 2024-04-22T23:38:57.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:56 smithi044 ceph-mon[34463]: Reconfiguring daemon mon.smithi044 on smithi044 2024-04-22T23:38:58.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:58.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:38:58.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:38:58.151 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:58.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:38:58.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:38:58.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:38:58.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:58.464 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:59.332 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.332 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:38:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:38:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:38:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:00.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:00 smithi031 ceph-mon[26721]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:39:00.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:01.180 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:00 smithi044 ceph-mon[34463]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:39:01.180 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:01.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.388+0000 7f5cdffff700 1 -- 172.21.15.31:0/2763556504 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5ce8105700 con 0x7f5cd406c460 2024-04-22T23:39:01.382 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5cd406c460 msgr2=0x7f5cd406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5cd406c460 0x7f5cd406e920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8069ef0 tx=0x7f5cd000b410 comp rx=0 tx=0).stop 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 msgr2=0x7f5ce8069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce8069970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800bdd0 tx=0x7f5cd800be00 comp rx=0 tx=0).stop 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 shutdown_connections 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5cd406c460 0x7f5cd406e920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce80ffbb0 0x7f5ce8069970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 --2- 172.21.15.31:0/2763556504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ce8100db0 0x7f5ce819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 >> 172.21.15.31:0/2763556504 conn(0x7f5ce80fb150 msgr2=0x7f5ce8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 shutdown_connections 2024-04-22T23:39:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:01.390+0000 7f5ced352700 1 -- 172.21.15.31:0/2763556504 wait complete. 2024-04-22T23:39:02.003 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.0.service 2024-04-22T23:39:02.006 INFO:tasks.cephadm:Deploying osd.1 on smithi031 with /dev/vg_nvme/lv_3... 2024-04-22T23:39:02.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T23:39:02.340 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:39:02 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[45209]: 2024-04-22T23:39:02.172+0000 7fec58f33640 -1 osd.0 0 log_to_monitors true 2024-04-22T23:39:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:39:02.620 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:02 smithi031 ceph-mon[26721]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:39:02.620 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:02.620 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:02.620 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:02 smithi031 ceph-mon[26721]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:39:02.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:02 smithi044 ceph-mon[34463]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:39:02.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:02.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:02.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:02 smithi044 ceph-mon[34463]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:39:04.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:03 smithi031 ceph-mon[26721]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:39:04.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:03 smithi031 ceph-mon[26721]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T23:39:04.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:03 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:04.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:03 smithi031 ceph-mon[26721]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:39:04.086 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:39:03 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[45209]: 2024-04-22T23:39:03.873+0000 7fec4dd96700 -1 osd.0 0 waiting for initial osdmap 2024-04-22T23:39:04.086 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:39:03 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[45209]: 2024-04-22T23:39:03.876+0000 7fec46384700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:39:04.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:03 smithi044 ceph-mon[34463]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:39:04.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:03 smithi044 ceph-mon[34463]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T23:39:04.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:03 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:04.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:03 smithi044 ceph-mon[34463]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:39:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T23:39:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T23:39:04.898 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-22T23:39:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-22T23:39:04.898 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124802 s, 840 MB/s 2024-04-22T23:39:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-22T23:39:05.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:04 smithi031 ceph-mon[26721]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:39:05.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:04 smithi031 ceph-mon[26721]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-22T23:39:05.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:04 smithi031 ceph-mon[26721]: osdmap e8: 1 total, 0 up, 1 in 2024-04-22T23:39:05.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:04 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:05.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:04 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:04 smithi044 ceph-mon[34463]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:39:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:04 smithi044 ceph-mon[34463]: from='osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-22T23:39:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:04 smithi044 ceph-mon[34463]: osdmap e8: 1 total, 0 up, 1 in 2024-04-22T23:39:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:04 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:04 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:06.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-04-22T23:39:06.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:05 smithi031 ceph-mon[26721]: purged_snaps scrub starts 2024-04-22T23:39:06.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:05 smithi031 ceph-mon[26721]: purged_snaps scrub ok 2024-04-22T23:39:06.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:05 smithi031 ceph-mon[26721]: osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] boot 2024-04-22T23:39:06.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:05 smithi031 ceph-mon[26721]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T23:39:06.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:06.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:05 smithi044 ceph-mon[34463]: purged_snaps scrub starts 2024-04-22T23:39:06.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:05 smithi044 ceph-mon[34463]: purged_snaps scrub ok 2024-04-22T23:39:06.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:05 smithi044 ceph-mon[34463]: osd.0 [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] boot 2024-04-22T23:39:06.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:05 smithi044 ceph-mon[34463]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T23:39:06.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:05 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:06.506 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:39:06.935 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:06 smithi031 ceph-mon[26721]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:06.935 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:06 smithi031 ceph-mon[26721]: osdmap e10: 1 total, 1 up, 1 in 2024-04-22T23:39:06.935 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:06.935 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:07.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:06 smithi044 ceph-mon[34463]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:07.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:06 smithi044 ceph-mon[34463]: osdmap e10: 1 total, 1 up, 1 in 2024-04-22T23:39:07.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:07.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:08.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 -- 172.21.15.31:0/1345788438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a0101f40 msgr2=0x7f87a01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:08.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 --2- 172.21.15.31:0/1345788438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a0101f40 0x7f87a01023a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8790009910 tx=0x7f8790009c20 comp rx=0 tx=0).stop 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 -- 172.21.15.31:0/1345788438 shutdown_connections 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 --2- 172.21.15.31:0/1345788438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a0101f40 0x7f87a01023a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 --2- 172.21.15.31:0/1345788438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a00fef60 0x7f87a00ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 -- 172.21.15.31:0/1345788438 >> 172.21.15.31:0/1345788438 conn(0x7f87a00fa8c0 msgr2=0x7f87a00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 -- 172.21.15.31:0/1345788438 shutdown_connections 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 -- 172.21.15.31:0/1345788438 wait complete. 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.159+0000 7f87a7313700 1 Processor -- start 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a7313700 1 -- start start 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a7313700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 0x7f87a01964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a7313700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a0101f40 0x7f87a0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a7313700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0197030 con 0x7f87a00fef60 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a7313700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a0197170 con 0x7f87a0101f40 2024-04-22T23:39:08.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a50af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 0x7f87a01964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a50af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 0x7f87a01964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49644/0 (socket says 172.21.15.31:49644) 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a50af700 1 -- 172.21.15.31:0/25991518 learned_addr learned my addr 172.21.15.31:0/25991518 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a48ae700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a0101f40 0x7f87a0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a50af700 1 -- 172.21.15.31:0/25991518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a0101f40 msgr2=0x7f87a0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a50af700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a0101f40 0x7f87a0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a50af700 1 -- 172.21.15.31:0/25991518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87900095f0 con 0x7f87a00fef60 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.160+0000 7f87a48ae700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a0101f40 0x7f87a0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.161+0000 7f87a50af700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 0x7f87a01964d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f879c00c8f0 tx=0x7f879c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.161+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879c007a70 con 0x7f87a00fef60 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.161+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a019bc20 con 0x7f87a00fef60 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.161+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a019c170 con 0x7f87a00fef60 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.161+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f879c007bd0 con 0x7f87a00fef60 2024-04-22T23:39:08.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.161+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f879c018630 con 0x7f87a00fef60 2024-04-22T23:39:08.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.162+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a0109800 con 0x7f87a00fef60 2024-04-22T23:39:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.162+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f879c018850 con 0x7f87a00fef60 2024-04-22T23:39:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.163+0000 7f87967fc700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f878c06c460 0x7f878c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.163+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f879c08a870 con 0x7f87a00fef60 2024-04-22T23:39:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.163+0000 7f87a48ae700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f878c06c460 0x7f878c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:08.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.163+0000 7f87a48ae700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f878c06c460 0x7f878c06e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87900059d0 tx=0x7f8790005920 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.171+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f879c059a50 con 0x7f87a00fef60 2024-04-22T23:39:08.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:08.380+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f87a0106890 con 0x7f878c06c460 2024-04-22T23:39:09.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:08 smithi031 ceph-mon[26721]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:09.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:39:09.143 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:39:09.143 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:09.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:08 smithi044 ceph-mon[34463]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:09.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:08 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:39:09.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:08 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:39:09.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:08 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:10.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:09 smithi031 ceph-mon[26721]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:10.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:09 smithi044 ceph-mon[34463]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:11.193 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:10 smithi044 ceph-mon[34463]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:10 smithi031 ceph-mon[26721]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi031 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi031 to 11845M 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:12.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: Detected new or changed devices on smithi031 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi031 to 11845M 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:12.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:13.308 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:12 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2033292910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b77430fc-df87-47da-a6c3-d237e3e0bc31"}]: dispatch 2024-04-22T23:39:13.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:12 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2033292910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b77430fc-df87-47da-a6c3-d237e3e0bc31"}]': finished 2024-04-22T23:39:13.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:12 smithi031 ceph-mon[26721]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T23:39:13.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:13.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:13.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:13.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:12 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2033292910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b77430fc-df87-47da-a6c3-d237e3e0bc31"}]: dispatch 2024-04-22T23:39:13.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:12 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2033292910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b77430fc-df87-47da-a6c3-d237e3e0bc31"}]': finished 2024-04-22T23:39:13.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:12 smithi044 ceph-mon[34463]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T23:39:13.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:13.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:13.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:14.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:13 smithi031 ceph-mon[26721]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:14.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:14 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1484184870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:39:14.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:14 smithi044 ceph-mon[34463]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:14.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:14 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1484184870' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:39:16.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:16.199 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:17.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:16 smithi031 ceph-mon[26721]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:17.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:17.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:17.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:17.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:17.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:17.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:16 smithi044 ceph-mon[34463]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:17.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:17.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:17.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:17.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:17.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:19.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:18 smithi031 ceph-mon[26721]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:19.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:18 smithi044 ceph-mon[34463]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:20 smithi031 ceph-mon[26721]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:20 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:39:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:20 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:20.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:20 smithi031 ceph-mon[26721]: Deploying daemon osd.1 on smithi031 2024-04-22T23:39:21.206 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:20 smithi044 ceph-mon[34463]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:21.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:20 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:39:21.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:20 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:21.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:20 smithi044 ceph-mon[34463]: Deploying daemon osd.1 on smithi031 2024-04-22T23:39:23.185 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:22 smithi031 ceph-mon[26721]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:22 smithi044 ceph-mon[34463]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:24.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:24 smithi031 ceph-mon[26721]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:25.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:24 smithi044 ceph-mon[34463]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:26.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:26 smithi031 ceph-mon[26721]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:26.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:26.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:26.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:26.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:26 smithi044 ceph-mon[34463]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:26.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:26.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:26.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:28.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:28 smithi031 ceph-mon[26721]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:28.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:28 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:28.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:28 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:28.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:28 smithi044 ceph-mon[34463]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:28.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:28 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:28.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:28 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:28.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.641+0000 7f87967fc700 1 -- 172.21.15.31:0/25991518 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f87a0106890 con 0x7f878c06c460 2024-04-22T23:39:28.635 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 1 on host 'smithi031' 2024-04-22T23:39:28.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f878c06c460 msgr2=0x7f878c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f878c06c460 0x7f878c06e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f87900059d0 tx=0x7f8790005920 comp rx=0 tx=0).stop 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 msgr2=0x7f87a01964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 0x7f87a01964d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f879c00c8f0 tx=0x7f879c00ccb0 comp rx=0 tx=0).stop 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 shutdown_connections 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f878c06c460 0x7f878c06e920 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87a00fef60 0x7f87a01964d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 --2- 172.21.15.31:0/25991518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87a0101f40 0x7f87a0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 >> 172.21.15.31:0/25991518 conn(0x7f87a00fa8c0 msgr2=0x7f87a0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 shutdown_connections 2024-04-22T23:39:28.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:28.644+0000 7f87a7313700 1 -- 172.21.15.31:0/25991518 wait complete. 2024-04-22T23:39:29.658 DEBUG:teuthology.orchestra.run.smithi031:osd.1> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.1.service 2024-04-22T23:39:29.701 INFO:tasks.cephadm:Deploying osd.2 on smithi031 with /dev/vg_nvme/lv_2... 2024-04-22T23:39:29.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T23:39:29.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:29 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:29.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:29 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:29.846 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:29 smithi031 ceph-mon[26721]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:39:29.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:29 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:29.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:29 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:29.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:29 smithi044 ceph-mon[34463]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:39:30.244 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:39:30.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-mon[26721]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:30.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-mon[26721]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:39:30.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-mon[26721]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T23:39:30.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:30.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-mon[26721]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:39:30.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:30.648 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[52945]: 2024-04-22T23:39:30.604+0000 7fb3b6cf1700 -1 osd.1 0 waiting for initial osdmap 2024-04-22T23:39:30.649 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:39:30 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[52945]: 2024-04-22T23:39:30.608+0000 7fb3b12e3700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:39:30.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:30 smithi044 ceph-mon[34463]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:30.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:30 smithi044 ceph-mon[34463]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:39:30.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:30 smithi044 ceph-mon[34463]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T23:39:30.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:30.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:30 smithi044 ceph-mon[34463]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:39:30.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:31.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:31 smithi031 ceph-mon[26721]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-22T23:39:31.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:31 smithi031 ceph-mon[26721]: osdmap e13: 2 total, 1 up, 2 in 2024-04-22T23:39:31.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:31 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:31.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:31 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:31.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:31 smithi044 ceph-mon[34463]: from='osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-22T23:39:31.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:31 smithi044 ceph-mon[34463]: osdmap e13: 2 total, 1 up, 2 in 2024-04-22T23:39:31.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:31 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:31.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:31 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:32.571 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T23:39:32.571 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T23:39:32.571 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-22T23:39:32.571 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-22T23:39:32.571 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125144 s, 838 MB/s 2024-04-22T23:39:32.571 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-22T23:39:32.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:32 smithi031 ceph-mon[26721]: purged_snaps scrub starts 2024-04-22T23:39:32.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:32 smithi031 ceph-mon[26721]: purged_snaps scrub ok 2024-04-22T23:39:32.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:32 smithi031 ceph-mon[26721]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:32.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:32 smithi031 ceph-mon[26721]: osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] boot 2024-04-22T23:39:32.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:32 smithi031 ceph-mon[26721]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T23:39:32.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:32.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:32 smithi044 ceph-mon[34463]: purged_snaps scrub starts 2024-04-22T23:39:32.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:32 smithi044 ceph-mon[34463]: purged_snaps scrub ok 2024-04-22T23:39:32.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:32 smithi044 ceph-mon[34463]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:39:32.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:32 smithi044 ceph-mon[34463]: osd.1 [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] boot 2024-04-22T23:39:32.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:32 smithi044 ceph-mon[34463]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T23:39:32.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:39:33.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-04-22T23:39:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:39:33.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:33 smithi031 ceph-mon[26721]: osdmap e15: 2 total, 2 up, 2 in 2024-04-22T23:39:33.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:33 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:33.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:33 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:33.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:33 smithi044 ceph-mon[34463]: osdmap e15: 2 total, 2 up, 2 in 2024-04-22T23:39:33.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:33 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:33.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:33 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:34.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:34 smithi031 ceph-mon[26721]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:34.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:34 smithi044 ceph-mon[34463]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 -- 172.21.15.31:0/1656378488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef200ffc10 msgr2=0x7fef20100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 --2- 172.21.15.31:0/1656378488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef200ffc10 0x7fef20100030 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b0a0 tx=0x7fef1000b3b0 comp rx=0 tx=0).stop 2024-04-22T23:39:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 -- 172.21.15.31:0/1656378488 shutdown_connections 2024-04-22T23:39:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 --2- 172.21.15.31:0/1656378488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef20100e10 0x7fef20101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 --2- 172.21.15.31:0/1656378488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef200ffc10 0x7fef20100030 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 -- 172.21.15.31:0/1656378488 >> 172.21.15.31:0/1656378488 conn(0x7fef200fb190 msgr2=0x7fef200fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 -- 172.21.15.31:0/1656378488 shutdown_connections 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.757+0000 7fef27dee700 1 -- 172.21.15.31:0/1656378488 wait complete. 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef27dee700 1 Processor -- start 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef27dee700 1 -- start start 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef27dee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef200ffc10 0x7fef20069140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef27dee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 0x7fef2019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef27dee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019b7f0 con 0x7fef20100e10 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef27dee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2019b930 con 0x7fef200ffc10 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 0x7fef2019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 0x7fef2019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55482/0 (socket says 172.21.15.31:55482) 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 -- 172.21.15.31:0/1172759630 learned_addr learned my addr 172.21.15.31:0/1172759630 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:39:35.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25b8a700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef200ffc10 0x7fef20069140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 -- 172.21.15.31:0/1172759630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef200ffc10 msgr2=0x7fef20069140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef200ffc10 0x7fef20069140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 -- 172.21.15.31:0/1172759630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10009d00 con 0x7fef20100e10 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.758+0000 7fef25389700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 0x7fef2019b1d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef1400d8d0 tx=0x7fef1400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.759+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef140098a0 con 0x7fef20100e10 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.759+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef14010460 con 0x7fef20100e10 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.759+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2019bc10 con 0x7fef20100e10 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.759+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef1400f5f0 con 0x7fef20100e10 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.759+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2019c160 con 0x7fef20100e10 2024-04-22T23:39:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.760+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef20108700 con 0x7fef20100e10 2024-04-22T23:39:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.760+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7fef14009a00 con 0x7fef20100e10 2024-04-22T23:39:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.761+0000 7fef1effd700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0806c460 0x7fef0806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.761+0000 7fef25b8a700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0806c460 0x7fef0806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.761+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fef1408a9b0 con 0x7fef20100e10 2024-04-22T23:39:35.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.761+0000 7fef25b8a700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0806c460 0x7fef0806e920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b830 tx=0x7fef100060c0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.765+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef140598f0 con 0x7fef20100e10 2024-04-22T23:39:35.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:35.975+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fef20105760 con 0x7fef0806c460 2024-04-22T23:39:37.037 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:36 smithi031 ceph-mon[26721]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:37.037 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:39:37.037 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:39:37.037 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:37.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:36 smithi044 ceph-mon[34463]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:37.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:39:37.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:39:37.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:38.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:37 smithi031 ceph-mon[26721]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:38.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:37 smithi044 ceph-mon[34463]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi031 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi031 to 5922M 2024-04-22T23:39:38.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:38.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:38.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:38.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:38.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:38 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:39.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:39.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: Detected new or changed devices on smithi031 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi031 to 5922M 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:39.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:38 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:40.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:39 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1183051486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27f0f9ea-73fe-4b71-84f8-31bd466c08e6"}]: dispatch 2024-04-22T23:39:40.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:39 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1183051486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27f0f9ea-73fe-4b71-84f8-31bd466c08e6"}]': finished 2024-04-22T23:39:40.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:39 smithi031 ceph-mon[26721]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T23:39:40.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:39 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1183051486' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27f0f9ea-73fe-4b71-84f8-31bd466c08e6"}]: dispatch 2024-04-22T23:39:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:39 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1183051486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27f0f9ea-73fe-4b71-84f8-31bd466c08e6"}]': finished 2024-04-22T23:39:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:39 smithi044 ceph-mon[34463]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T23:39:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:41.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:40 smithi031 ceph-mon[26721]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:41.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:41.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:41.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:40 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1117598415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:39:41.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:40 smithi044 ceph-mon[34463]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:41.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:41.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:41.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:40 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1117598415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:39:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:42 smithi031 ceph-mon[26721]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:43.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:42 smithi044 ceph-mon[34463]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:44.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:44 smithi031 ceph-mon[26721]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:45.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:44 smithi044 ceph-mon[34463]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:46.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:46.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:47.163 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:46 smithi031 ceph-mon[26721]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:47.164 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:46 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:39:47.164 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:46 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:46 smithi044 ceph-mon[34463]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:46 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:39:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:46 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:48.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:47 smithi031 ceph-mon[26721]: Deploying daemon osd.2 on smithi031 2024-04-22T23:39:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:47 smithi044 ceph-mon[34463]: Deploying daemon osd.2 on smithi031 2024-04-22T23:39:49.179 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:48 smithi031 ceph-mon[26721]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:49.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:48 smithi044 ceph-mon[34463]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:51.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:50 smithi031 ceph-mon[26721]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:50 smithi044 ceph-mon[34463]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:52 smithi031 ceph-mon[26721]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:53.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:52 smithi044 ceph-mon[34463]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:53.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:53.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:53.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:55.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:54 smithi031 ceph-mon[26721]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:55.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:54 smithi044 ceph-mon[34463]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.327+0000 7fef1effd700 1 -- 172.21.15.31:0/1172759630 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fef20105760 con 0x7fef0806c460 2024-04-22T23:39:55.322 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 2 on host 'smithi031' 2024-04-22T23:39:55.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0806c460 msgr2=0x7fef0806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0806c460 0x7fef0806e920 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b830 tx=0x7fef100060c0 comp rx=0 tx=0).stop 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 msgr2=0x7fef2019b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 0x7fef2019b1d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef1400d8d0 tx=0x7fef1400dbe0 comp rx=0 tx=0).stop 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 shutdown_connections 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fef0806c460 0x7fef0806e920 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef200ffc10 0x7fef20069140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 --2- 172.21.15.31:0/1172759630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef20100e10 0x7fef2019b1d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 >> 172.21.15.31:0/1172759630 conn(0x7fef200fb190 msgr2=0x7fef20104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 shutdown_connections 2024-04-22T23:39:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:39:55.329+0000 7fef27dee700 1 -- 172.21.15.31:0/1172759630 wait complete. 2024-04-22T23:39:56.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:56 smithi031 ceph-mon[26721]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:56.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:56 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:56.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:56 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:56.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:56 smithi031 ceph-mon[26721]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:39:56.387 DEBUG:teuthology.orchestra.run.smithi031:osd.2> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.2.service 2024-04-22T23:39:56.389 INFO:tasks.cephadm:Deploying osd.3 on smithi044 with /dev/vg_nvme/lv_4... 2024-04-22T23:39:56.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T23:39:56.608 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:39:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:56 smithi044 ceph-mon[34463]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:56 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:56 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:56 smithi044 ceph-mon[34463]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:39:57.581 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:57 smithi044 ceph-mon[34463]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:39:57.581 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:57 smithi044 ceph-mon[34463]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T23:39:57.581 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:57 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:57.581 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:57 smithi044 ceph-mon[34463]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:39:57.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:57 smithi031 ceph-mon[26721]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:39:57.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:57 smithi031 ceph-mon[26721]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T23:39:57.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:57 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:57.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:57 smithi031 ceph-mon[26721]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:39:57.696 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:39:57 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[60808]: 2024-04-22T23:39:57.340+0000 7f9fc5c1a700 -1 osd.2 0 waiting for initial osdmap 2024-04-22T23:39:57.696 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:39:57 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[60808]: 2024-04-22T23:39:57.345+0000 7f9fc1a0f700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:39:58.334 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T23:39:58.334 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T23:39:58.334 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-22T23:39:58.334 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-22T23:39:58.334 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118432 s, 885 MB/s 2024-04-22T23:39:58.334 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-22T23:39:58.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:58 smithi031 ceph-mon[26721]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:58.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:58 smithi031 ceph-mon[26721]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-22T23:39:58.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:58 smithi031 ceph-mon[26721]: osdmap e18: 3 total, 2 up, 3 in 2024-04-22T23:39:58.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:58.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:58 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:58.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:58 smithi044 ceph-mon[34463]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:39:58.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:58 smithi044 ceph-mon[34463]: from='osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-22T23:39:58.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:58 smithi044 ceph-mon[34463]: osdmap e18: 3 total, 2 up, 3 in 2024-04-22T23:39:58.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:58.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:58 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:58.970 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-04-22T23:39:59.181 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: purged_snaps scrub starts 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: purged_snaps scrub ok 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] boot 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:59.584 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:39:59 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T23:39:59.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: purged_snaps scrub starts 2024-04-22T23:39:59.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: purged_snaps scrub ok 2024-04-22T23:39:59.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:59.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: osd.2 [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] boot 2024-04-22T23:39:59.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T23:39:59.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:39:59.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:59.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:39:59.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:39:59 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T23:40:00.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 -- 172.21.15.44:0/2286335626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 msgr2=0x7f5734106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 --2- 172.21.15.44:0/2286335626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f5734106470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f571c009910 tx=0x7f571c009c20 comp rx=0 tx=0).stop 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 -- 172.21.15.44:0/2286335626 shutdown_connections 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 --2- 172.21.15.44:0/2286335626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f5734106470 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 --2- 172.21.15.44:0/2286335626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5734101750 0x7f5734103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 -- 172.21.15.44:0/2286335626 >> 172.21.15.44:0/2286335626 conn(0x7f57340fb160 msgr2=0x7f57340fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 -- 172.21.15.44:0/2286335626 shutdown_connections 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.664+0000 7f5738fc8700 1 -- 172.21.15.44:0/2286335626 wait complete. 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5738fc8700 1 Processor -- start 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5738fc8700 1 -- start start 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5738fc8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5734101750 0x7f57341010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5738fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f57340ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:00.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5738fc8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340ffc40 con 0x7f5734101750 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5738fc8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340ffd80 con 0x7f5734104080 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f57340ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f57340ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59186/0 (socket says 172.21.15.44:59186) 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 -- 172.21.15.44:0/419850369 learned_addr learned my addr 172.21.15.44:0/419850369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f573259c700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5734101750 0x7f57341010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 -- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5734101750 msgr2=0x7f57341010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5734101750 0x7f57341010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 -- 172.21.15.44:0/419850369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0095f0 con 0x7f5734104080 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.665+0000 7f5731d9b700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f57340ff700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f571c006010 tx=0x7f571c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.666+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571c01d070 con 0x7f5734104080 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.666+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f571c00f460 con 0x7f5734104080 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.666+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5734100000 con 0x7f5734104080 2024-04-22T23:40:00.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.666+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f571c021640 con 0x7f5734104080 2024-04-22T23:40:00.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.666+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57341004f0 con 0x7f5734104080 2024-04-22T23:40:00.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.667+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734190630 con 0x7f5734104080 2024-04-22T23:40:00.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.667+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f571c00fab0 con 0x7f5734104080 2024-04-22T23:40:00.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.668+0000 7f572b7fe700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f572006c460 0x7f572006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:00.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.668+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f571c08bfa0 con 0x7f5734104080 2024-04-22T23:40:00.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.668+0000 7f573259c700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f572006c460 0x7f572006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:00.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.668+0000 7f573259c700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f572006c460 0x7f572006e920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5724009550 tx=0x7f57240091b0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:00.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.673+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f571c05ac30 con 0x7f5734104080 2024-04-22T23:40:00.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:00 smithi044 ceph-mon[34463]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:40:00.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T23:40:00.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:00 smithi044 ceph-mon[34463]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T23:40:00.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T23:40:00.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:00 smithi044 ceph-mon[34463]: overall HEALTH_OK 2024-04-22T23:40:00.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:00.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:00.881+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f573402d010 con 0x7f572006c460 2024-04-22T23:40:00.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:00 smithi031 ceph-mon[26721]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T23:40:00.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T23:40:00.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:00 smithi031 ceph-mon[26721]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T23:40:00.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T23:40:00.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:00 smithi031 ceph-mon[26721]: overall HEALTH_OK 2024-04-22T23:40:00.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:01.519 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65267]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:40:01.519 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65267]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:01.519 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65267]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:01.519 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65267]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:01.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:01.602+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f571c057570 con 0x7f5734104080 2024-04-22T23:40:01.945 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:40:01.946 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:01.946 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65302]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:01.946 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:40:01 smithi031 sudo[65302]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:01.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:01 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T23:40:01.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:01 smithi031 ceph-mon[26721]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T23:40:01.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:01 smithi031 ceph-mon[26721]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:01.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:01 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:40:01.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:01 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:40:01.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:01 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:01.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:01 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T23:40:01.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:01 smithi044 ceph-mon[34463]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T23:40:01.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:01 smithi044 ceph-mon[34463]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:01.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:01 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:40:01.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:01 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:40:01.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:01 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:02.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65413]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:40:02.446 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65413]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:02.446 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65413]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:02.446 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65413]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:02.730 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:40:02.730 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:02.730 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:02.730 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:40:02 smithi031 sudo[65482]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: Detected new or changed devices on smithi031 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: mgrmap e19: smithi031.fazdcv(active, since 2m), standbys: smithi044.iglcti 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi031 to 3948M 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:02.731 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:02.732 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.732 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.732 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:02.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:02.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi031 2024-04-22T23:40:02.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: mgrmap e19: smithi031.fazdcv(active, since 2m), standbys: smithi044.iglcti 2024-04-22T23:40:02.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T23:40:02.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi031 to 3948M 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:02.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:03.445 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:40:03.446 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:03.446 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65548]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:03.446 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65548]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:03.902 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65613]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:40:03.902 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:03.902 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65613]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:03.902 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:40:03 smithi031 sudo[65613]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:04.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65685]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:40:04.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65685]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:04.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65685]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:04.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65685]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65733]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65733]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 sudo[65733]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/2021494167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b88b2ea-7239-40b7-ad29-5ed83a3c5635"}]: dispatch 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 ceph-mon[26721]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b88b2ea-7239-40b7-ad29-5ed83a3c5635"}]: dispatch 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 ceph-mon[26721]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b88b2ea-7239-40b7-ad29-5ed83a3c5635"}]': finished 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 ceph-mon[26721]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:04.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:04 smithi031 ceph-mon[26721]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:40:04.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/2021494167' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b88b2ea-7239-40b7-ad29-5ed83a3c5635"}]: dispatch 2024-04-22T23:40:04.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 ceph-mon[34463]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b88b2ea-7239-40b7-ad29-5ed83a3c5635"}]: dispatch 2024-04-22T23:40:04.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 ceph-mon[34463]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b88b2ea-7239-40b7-ad29-5ed83a3c5635"}]': finished 2024-04-22T23:40:04.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 ceph-mon[34463]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T23:40:04.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:04.921 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 ceph-mon[34463]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:40:05.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 sudo[38699]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:40:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 sudo[38699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 sudo[38699]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:05.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:04 smithi044 sudo[38699]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:05.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 sudo[38887]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 sudo[38887]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 sudo[38887]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 sudo[38887]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/2680436521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:40:05.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:40:05.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:40:05.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:05 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:05.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/2680436521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:40:05.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:05 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:06.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:06 smithi044 ceph-mon[34463]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:06.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:06.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:06.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:06.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:06.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:06 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:06.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:06 smithi031 ceph-mon[26721]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:06.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:06.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:06.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:06.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:06.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:06 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:08.559 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:08 smithi044 ceph-mon[34463]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:08.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:08 smithi031 ceph-mon[26721]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:11.189 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:10 smithi044 ceph-mon[34463]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-22T23:40:11.189 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:10 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:40:11.189 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:10 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:10 smithi031 ceph-mon[26721]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-22T23:40:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:10 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:40:11.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:10 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:12.127 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:11 smithi044 ceph-mon[34463]: Deploying daemon osd.3 on smithi044 2024-04-22T23:40:12.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:11 smithi031 ceph-mon[26721]: Deploying daemon osd.3 on smithi044 2024-04-22T23:40:13.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:12 smithi031 ceph-mon[26721]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-22T23:40:13.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:12 smithi044 ceph-mon[34463]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-22T23:40:15.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:14 smithi031 ceph-mon[26721]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-22T23:40:15.200 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:14 smithi044 ceph-mon[34463]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-22T23:40:16.112 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:16.112 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:16.112 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:16.112 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:16.197 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:16.197 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:16.197 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:16.198 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:16.881 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:16 smithi044 ceph-mon[34463]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-04-22T23:40:17.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:16 smithi031 ceph-mon[26721]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-04-22T23:40:19.104 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:18 smithi044 ceph-mon[34463]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:40:19.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:18 smithi031 ceph-mon[26721]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:40:19.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.797+0000 7f572b7fe700 1 -- 172.21.15.44:0/419850369 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f573402d010 con 0x7f572006c460 2024-04-22T23:40:19.792 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 3 on host 'smithi044' 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f572006c460 msgr2=0x7f572006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f572006c460 0x7f572006e920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5724009550 tx=0x7f57240091b0 comp rx=0 tx=0).stop 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 msgr2=0x7f57340ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f57340ff700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f571c006010 tx=0x7f571c00bd70 comp rx=0 tx=0).stop 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 shutdown_connections 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f572006c460 0x7f572006e920 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5734101750 0x7f57341010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 --2- 172.21.15.44:0/419850369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5734104080 0x7f57340ff700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 >> 172.21.15.44:0/419850369 conn(0x7f57340fb160 msgr2=0x7f57340fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:19.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 shutdown_connections 2024-04-22T23:40:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:19.800+0000 7f5738fc8700 1 -- 172.21.15.44:0/419850369 wait complete. 2024-04-22T23:40:20.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:19 smithi044 ceph-mon[34463]: from='osd.3 [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:40:20.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:19 smithi044 ceph-mon[34463]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:40:20.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:19 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:20.095 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:19 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:20.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:19 smithi031 ceph-mon[26721]: from='osd.3 [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:40:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:19 smithi031 ceph-mon[26721]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:40:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:19 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:20.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:19 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:20.384 DEBUG:teuthology.orchestra.run.smithi044:osd.3> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.3.service 2024-04-22T23:40:20.386 INFO:tasks.cephadm:Deploying osd.4 on smithi044 with /dev/vg_nvme/lv_3... 2024-04-22T23:40:20.386 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T23:40:20.641 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:40:21.056 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:40:21.056 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:40:21.056 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T23:40:21.056 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:21.056 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: from='osd.3 [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:21.057 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:21.057 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:21.057 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:21.057 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[40534]: 2024-04-22T23:40:20.907+0000 7fd4a216c700 -1 osd.3 0 waiting for initial osdmap 2024-04-22T23:40:21.057 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:40:20 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[40534]: 2024-04-22T23:40:20.914+0000 7fd49c75e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:40:21.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:40:21.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:40:21.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T23:40:21.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:21.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: from='osd.3 [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:21.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:21.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:21.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:20 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:22.171 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:21 smithi044 ceph-mon[34463]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-22T23:40:22.171 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:21 smithi044 ceph-mon[34463]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T23:40:22.171 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:21 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:22.171 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:21 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:22.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:21 smithi031 ceph-mon[26721]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-22T23:40:22.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:21 smithi031 ceph-mon[26721]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T23:40:22.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:21 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:22.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:21 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:23.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: purged_snaps scrub starts 2024-04-22T23:40:23.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: purged_snaps scrub ok 2024-04-22T23:40:23.093 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:23.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: osd.3 [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] boot 2024-04-22T23:40:23.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T23:40:23.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:23.094 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:22 smithi044 ceph-mon[34463]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T23:40:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: purged_snaps scrub starts 2024-04-22T23:40:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: purged_snaps scrub ok 2024-04-22T23:40:23.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-22T23:40:23.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: osd.3 [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] boot 2024-04-22T23:40:23.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T23:40:23.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:40:23.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:22 smithi031 ceph-mon[26721]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T23:40:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T23:40:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T23:40:23.893 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-22T23:40:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-22T23:40:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0160961 s, 651 MB/s 2024-04-22T23:40:23.893 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-22T23:40:24.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-04-22T23:40:25.156 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:40:25.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:24 smithi031 ceph-mon[26721]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:25.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:24 smithi031 ceph-mon[26721]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T23:40:25.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:24 smithi031 ceph-mon[26721]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T23:40:25.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:24 smithi044 ceph-mon[34463]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:25.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:24 smithi044 ceph-mon[34463]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T23:40:25.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:24 smithi044 ceph-mon[34463]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T23:40:26.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: Detected new or changed devices on smithi044 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi044 to 14917M 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:26.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:26.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:26.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:26 smithi031 ceph-mon[26721]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi044 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi044 to 14917M 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:26.371 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:26.372 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:26.372 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:26.372 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:26 smithi044 ceph-mon[34463]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T23:40:26.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.678+0000 7f865534e700 1 -- 172.21.15.44:0/1954549804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650104060 msgr2=0x7f8650106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.678+0000 7f865534e700 1 --2- 172.21.15.44:0/1954549804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650104060 0x7f8650106450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8644009910 tx=0x7f8644009c20 comp rx=0 tx=0).stop 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.678+0000 7f865534e700 1 -- 172.21.15.44:0/1954549804 shutdown_connections 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.678+0000 7f865534e700 1 --2- 172.21.15.44:0/1954549804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650104060 0x7f8650106450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.678+0000 7f865534e700 1 --2- 172.21.15.44:0/1954549804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8650101730 0x7f8650103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.678+0000 7f865534e700 1 -- 172.21.15.44:0/1954549804 >> 172.21.15.44:0/1954549804 conn(0x7f86500fb120 msgr2=0x7f86500fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.679+0000 7f865534e700 1 -- 172.21.15.44:0/1954549804 shutdown_connections 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.679+0000 7f865534e700 1 -- 172.21.15.44:0/1954549804 wait complete. 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.679+0000 7f865534e700 1 Processor -- start 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.679+0000 7f865534e700 1 -- start start 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f865534e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 0x7f8650196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:26.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f865534e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8650104060 0x7f8650196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f865534e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650196f80 con 0x7f8650104060 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f865534e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86501970c0 con 0x7f8650101730 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 0x7f8650196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 0x7f8650196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38606/0 (socket says 172.21.15.44:38606) 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 -- 172.21.15.44:0/1290932334 learned_addr learned my addr 172.21.15.44:0/1290932334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864e7fc700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8650104060 0x7f8650196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 -- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8650104060 msgr2=0x7f8650196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8650104060 0x7f8650196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 -- 172.21.15.44:0/1290932334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440095f0 con 0x7f8650101730 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f864effd700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 0x7f8650196420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f864000d8d0 tx=0x7f864000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86400098a0 con 0x7f8650101730 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.680+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8640010460 con 0x7f8650101730 2024-04-22T23:40:26.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.681+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865019bb70 con 0x7f8650101730 2024-04-22T23:40:26.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.681+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864000f5f0 con 0x7f8650101730 2024-04-22T23:40:26.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.681+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865019c0c0 con 0x7f8650101730 2024-04-22T23:40:26.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.682+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8650190610 con 0x7f8650101730 2024-04-22T23:40:26.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.682+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8640009a00 con 0x7f8650101730 2024-04-22T23:40:26.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.683+0000 7f8637fff700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f863806c4e0 0x7f863806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:26.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.683+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f864008b030 con 0x7f8650101730 2024-04-22T23:40:26.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.683+0000 7f864e7fc700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f863806c4e0 0x7f863806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:26.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.683+0000 7f864e7fc700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f863806c4e0 0x7f863806e9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8644006010 tx=0x7f8644005a90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:26.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.688+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8640059940 con 0x7f8650101730 2024-04-22T23:40:26.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:26.892+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f86500611d0 con 0x7f863806c4e0 2024-04-22T23:40:27.374 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:27 smithi044 ceph-mon[34463]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:27.374 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:40:27.374 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:40:27.374 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:27.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:27 smithi031 ceph-mon[26721]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:27.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:40:27.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:40:27.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:28.377 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:28 smithi044 ceph-mon[34463]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:28.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:28 smithi031 ceph-mon[26721]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:30.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/4030931761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5"}]: dispatch 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5"}]: dispatch 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5"}]': finished 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:30.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:30 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/278362160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:40:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/4030931761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5"}]: dispatch 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5"}]: dispatch 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5"}]': finished 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:30.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:30 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/278362160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:40:32.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:32 smithi031 ceph-mon[26721]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-22T23:40:32.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:32 smithi044 ceph-mon[34463]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-22T23:40:34.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:34 smithi031 ceph-mon[26721]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T23:40:34.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:34 smithi044 ceph-mon[34463]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T23:40:36.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:36 smithi044 ceph-mon[34463]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-22T23:40:36.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:40:36.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:37.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:36 smithi031 ceph-mon[26721]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-22T23:40:37.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:40:37.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:38.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:37 smithi031 ceph-mon[26721]: Deploying daemon osd.4 on smithi044 2024-04-22T23:40:38.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:37 smithi044 ceph-mon[34463]: Deploying daemon osd.4 on smithi044 2024-04-22T23:40:39.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:38 smithi031 ceph-mon[26721]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T23:40:39.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:38 smithi044 ceph-mon[34463]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T23:40:41.090 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:40 smithi044 ceph-mon[34463]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:41.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:40 smithi031 ceph-mon[26721]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:42 smithi044 ceph-mon[34463]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:43.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:43.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:42 smithi031 ceph-mon[26721]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:43.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:43.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:43.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:45.084 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:44 smithi044 ceph-mon[34463]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:45.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:44 smithi031 ceph-mon[26721]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:45.752 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 4 on host 'smithi044' 2024-04-22T23:40:45.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.758+0000 7f8637fff700 1 -- 172.21.15.44:0/1290932334 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86500611d0 con 0x7f863806c4e0 2024-04-22T23:40:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.760+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f863806c4e0 msgr2=0x7f863806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.760+0000 7f865534e700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f863806c4e0 0x7f863806e9a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8644006010 tx=0x7f8644005a90 comp rx=0 tx=0).stop 2024-04-22T23:40:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.760+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 msgr2=0x7f8650196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.760+0000 7f865534e700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 0x7f8650196420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f864000d8d0 tx=0x7f864000dbe0 comp rx=0 tx=0).stop 2024-04-22T23:40:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 shutdown_connections 2024-04-22T23:40:45.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f863806c4e0 0x7f863806e9a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8650101730 0x7f8650196420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 --2- 172.21.15.44:0/1290932334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8650104060 0x7f8650196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 >> 172.21.15.44:0/1290932334 conn(0x7f86500fb120 msgr2=0x7f86500fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 shutdown_connections 2024-04-22T23:40:45.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:45.761+0000 7f865534e700 1 -- 172.21.15.44:0/1290932334 wait complete. 2024-04-22T23:40:46.284 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:46.284 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:46.284 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:46.284 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:45 smithi044 ceph-mon[34463]: from='osd.4 [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:40:46.284 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:45 smithi044 ceph-mon[34463]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:40:46.306 DEBUG:teuthology.orchestra.run.smithi044:osd.4> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.4.service 2024-04-22T23:40:46.312 INFO:tasks.cephadm:Deploying osd.5 on smithi044 with /dev/vg_nvme/lv_2... 2024-04-22T23:40:46.312 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T23:40:46.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:46.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:46.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:46.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:45 smithi031 ceph-mon[26721]: from='osd.4 [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:40:46.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:45 smithi031 ceph-mon[26721]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:40:46.563 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: from='osd.4 [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:47.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:46 smithi044 ceph-mon[34463]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:40:47.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T23:40:47.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:47.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: from='osd.4 [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:47.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:46 smithi031 ceph-mon[26721]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:40:47.936 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:40:47 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[47021]: 2024-04-22T23:40:47.734+0000 7fe1d3ff4700 -1 osd.4 0 waiting for initial osdmap 2024-04-22T23:40:47.936 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:40:47 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[47021]: 2024-04-22T23:40:47.742+0000 7fe1cfde9700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:40:48.769 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T23:40:48.769 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T23:40:48.770 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-22T23:40:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-22T23:40:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119314 s, 879 MB/s 2024-04-22T23:40:48.770 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-22T23:40:48.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:48 smithi044 ceph-mon[34463]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:48.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:48 smithi044 ceph-mon[34463]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-22T23:40:48.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:48 smithi044 ceph-mon[34463]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T23:40:48.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:48 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:48.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:48 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:48 smithi031 ceph-mon[26721]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-22T23:40:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:48 smithi031 ceph-mon[26721]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-22T23:40:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:48 smithi031 ceph-mon[26721]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T23:40:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:48 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:48 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:49.384 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-04-22T23:40:50.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:49 smithi044 ceph-mon[34463]: purged_snaps scrub starts 2024-04-22T23:40:50.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:49 smithi044 ceph-mon[34463]: purged_snaps scrub ok 2024-04-22T23:40:50.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:49 smithi044 ceph-mon[34463]: osd.4 [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] boot 2024-04-22T23:40:50.006 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:49 smithi044 ceph-mon[34463]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T23:40:50.007 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:49 smithi031 ceph-mon[26721]: purged_snaps scrub starts 2024-04-22T23:40:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:49 smithi031 ceph-mon[26721]: purged_snaps scrub ok 2024-04-22T23:40:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:49 smithi031 ceph-mon[26721]: osd.4 [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] boot 2024-04-22T23:40:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:49 smithi031 ceph-mon[26721]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T23:40:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:40:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:40:50.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:50 smithi044 ceph-mon[34463]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:50.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:50 smithi044 ceph-mon[34463]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T23:40:51.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:50 smithi031 ceph-mon[26721]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:51.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:50 smithi031 ceph-mon[26721]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T23:40:52.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 -- 172.21.15.44:0/3932751446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0101eb0 msgr2=0x7fd6b0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:52.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/3932751446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0101eb0 0x7fd6b0102310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009910 tx=0x7fd6a4009c20 comp rx=0 tx=0).stop 2024-04-22T23:40:52.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 -- 172.21.15.44:0/3932751446 shutdown_connections 2024-04-22T23:40:52.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/3932751446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0101eb0 0x7fd6b0102310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/3932751446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6b0100cb0 0x7fd6b01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 -- 172.21.15.44:0/3932751446 >> 172.21.15.44:0/3932751446 conn(0x7fd6b00fc230 msgr2=0x7fd6b00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 -- 172.21.15.44:0/3932751446 shutdown_connections 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.503+0000 7fd6b62e1700 1 -- 172.21.15.44:0/3932751446 wait complete. 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6b62e1700 1 Processor -- start 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6b62e1700 1 -- start start 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6b62e1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 0x7fd6b0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6b62e1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6b0101eb0 0x7fd6b0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6b62e1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00736c0 con 0x7fd6b0101eb0 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6b62e1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b0073800 con 0x7fd6b0100cb0 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6affff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 0x7fd6b0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:52.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6affff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 0x7fd6b0074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50742/0 (socket says 172.21.15.44:50742) 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.504+0000 7fd6affff700 1 -- 172.21.15.44:0/908321688 learned_addr learned my addr 172.21.15.44:0/908321688 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6af7fe700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6b0101eb0 0x7fd6b0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6affff700 1 -- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6b0101eb0 msgr2=0x7fd6b0073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6affff700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6b0101eb0 0x7fd6b0073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6affff700 1 -- 172.21.15.44:0/908321688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a40095f0 con 0x7fd6b0100cb0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6affff700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 0x7fd6b0074b30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c910 tx=0x7fd6a000cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a00041f0 con 0x7fd6b0100cb0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0073ae0 con 0x7fd6b0100cb0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd6a000ce60 con 0x7fd6b0100cb0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a0003d20 con 0x7fd6b0100cb0 2024-04-22T23:40:52.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.505+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b019e540 con 0x7fd6b0100cb0 2024-04-22T23:40:52.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.506+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b0109810 con 0x7fd6b0100cb0 2024-04-22T23:40:52.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.507+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd6a0007900 con 0x7fd6b0100cb0 2024-04-22T23:40:52.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.507+0000 7fd6ad7fa700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd69806c4e0 0x7fd69806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:52.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.507+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fd6a008b000 con 0x7fd6b0100cb0 2024-04-22T23:40:52.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.507+0000 7fd6af7fe700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd69806c4e0 0x7fd69806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:52.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.508+0000 7fd6af7fe700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd69806c4e0 0x7fd69806e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4006010 tx=0x7fd6a400b540 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:52.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.512+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd6a0059760 con 0x7fd6b0100cb0 2024-04-22T23:40:52.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:40:52.723+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6b019eaf0 con 0x7fd69806c4e0 2024-04-22T23:40:52.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:52.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: Detected new or changed devices on smithi044 2024-04-22T23:40:52.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi044 to 7458M 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:52.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:52.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi044 2024-04-22T23:40:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi044 to 7458M 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:40:53.753 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:53 smithi044 ceph-mon[34463]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:53.753 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:40:53.753 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:40:53.753 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:53 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:53.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:53 smithi031 ceph-mon[26721]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:53.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:40:53.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:40:53.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:53 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:54 smithi031 ceph-mon[26721]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:54 smithi044 ceph-mon[34463]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:56.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:55 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/2082223773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ef19c59-4032-4263-a6e0-519ebe782468"}]: dispatch 2024-04-22T23:40:56.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:55 smithi044 ceph-mon[34463]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ef19c59-4032-4263-a6e0-519ebe782468"}]: dispatch 2024-04-22T23:40:56.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:55 smithi044 ceph-mon[34463]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ef19c59-4032-4263-a6e0-519ebe782468"}]': finished 2024-04-22T23:40:56.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:55 smithi044 ceph-mon[34463]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T23:40:56.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:40:56.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:55 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/4098530727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:40:56.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/2082223773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ef19c59-4032-4263-a6e0-519ebe782468"}]: dispatch 2024-04-22T23:40:56.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:55 smithi031 ceph-mon[26721]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ef19c59-4032-4263-a6e0-519ebe782468"}]: dispatch 2024-04-22T23:40:56.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:55 smithi031 ceph-mon[26721]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ef19c59-4032-4263-a6e0-519ebe782468"}]': finished 2024-04-22T23:40:56.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:55 smithi031 ceph-mon[26721]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T23:40:56.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:40:56.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/4098530727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:40:57.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:56 smithi044 ceph-mon[34463]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:57.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:56 smithi031 ceph-mon[26721]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:59.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:40:58 smithi044 ceph-mon[34463]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:40:59.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:40:58 smithi031 ceph-mon[26721]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:01.068 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:00 smithi044 ceph-mon[34463]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:01.068 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:01.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:00 smithi031 ceph-mon[26721]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:01.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:03.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:02 smithi044 ceph-mon[34463]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:03.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:41:03.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:02 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:02 smithi031 ceph-mon[26721]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:41:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:02 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:04.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:03 smithi044 ceph-mon[34463]: Deploying daemon osd.5 on smithi044 2024-04-22T23:41:04.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:03 smithi031 ceph-mon[26721]: Deploying daemon osd.5 on smithi044 2024-04-22T23:41:05.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:04 smithi044 ceph-mon[34463]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:05.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:04 smithi031 ceph-mon[26721]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:07.064 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:06 smithi044 ceph-mon[34463]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:07.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:06 smithi031 ceph-mon[26721]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:08.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:08 smithi031 ceph-mon[26721]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:08.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:08.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:08.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:08 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:08.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:08 smithi044 ceph-mon[34463]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:08.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:08 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:08.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:08 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:08.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:08 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:11.144 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:10 smithi044 ceph-mon[34463]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:10 smithi031 ceph-mon[26721]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:11.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.438+0000 7fd6ad7fa700 1 -- 172.21.15.44:0/908321688 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd6b019eaf0 con 0x7fd69806c4e0 2024-04-22T23:41:11.432 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 5 on host 'smithi044' 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd69806c4e0 msgr2=0x7fd69806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd69806c4e0 0x7fd69806e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4006010 tx=0x7fd6a400b540 comp rx=0 tx=0).stop 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 msgr2=0x7fd6b0074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 0x7fd6b0074b30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c910 tx=0x7fd6a000cc20 comp rx=0 tx=0).stop 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 shutdown_connections 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd69806c4e0 0x7fd69806e9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6b0100cb0 0x7fd6b0074b30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:11.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 --2- 172.21.15.44:0/908321688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6b0101eb0 0x7fd6b0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 >> 172.21.15.44:0/908321688 conn(0x7fd6b00fc230 msgr2=0x7fd6b01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 shutdown_connections 2024-04-22T23:41:11.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:11.440+0000 7fd6b62e1700 1 -- 172.21.15.44:0/908321688 wait complete. 2024-04-22T23:41:12.087 DEBUG:teuthology.orchestra.run.smithi044:osd.5> sudo journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.5.service 2024-04-22T23:41:12.090 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-22T23:41:12.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T23:41:12.320 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:12.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:12.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: from='osd.5 [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:41:12.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:41:12.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:12 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: from='osd.5 [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:12.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:12 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:13.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:13 smithi031 ceph-mon[26721]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:41:13.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:13 smithi031 ceph-mon[26721]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T23:41:13.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:13 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:13.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:13 smithi031 ceph-mon[26721]: from='osd.5 [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:41:13.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:13 smithi031 ceph-mon[26721]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:41:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-mon[34463]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:41:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-mon[34463]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T23:41:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-mon[34463]: from='osd.5 [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:41:13.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-mon[34463]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:41:13.712 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[53944]: 2024-04-22T23:41:13.442+0000 7fdae583f700 -1 osd.5 0 waiting for initial osdmap 2024-04-22T23:41:13.712 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:41:13 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[53944]: 2024-04-22T23:41:13.450+0000 7fdadde2d700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:41:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 -- 172.21.15.31:0/117694459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6500ffdf0 msgr2=0x7fe650100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 --2- 172.21.15.31:0/117694459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6500ffdf0 0x7fe650100270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe644009910 tx=0x7fe644009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 -- 172.21.15.31:0/117694459 shutdown_connections 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 --2- 172.21.15.31:0/117694459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6500ffdf0 0x7fe650100270 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 --2- 172.21.15.31:0/117694459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe650101220 0x7fe650101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 -- 172.21.15.31:0/117694459 >> 172.21.15.31:0/117694459 conn(0x7fe6500fa8c0 msgr2=0x7fe6500fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 -- 172.21.15.31:0/117694459 shutdown_connections 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 -- 172.21.15.31:0/117694459 wait complete. 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.127+0000 7fe655861700 1 Processor -- start 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe655861700 1 -- start start 2024-04-22T23:41:14.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe655861700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6500ffdf0 0x7fe650103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe655861700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 0x7fe6501021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe655861700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650102720 con 0x7fe650101220 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe655861700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe650102860 con 0x7fe6500ffdf0 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 0x7fe6501021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 0x7fe6501021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46742/0 (socket says 172.21.15.31:46742) 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 -- 172.21.15.31:0/2222195935 learned_addr learned my addr 172.21.15.31:0/2222195935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64effd700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6500ffdf0 0x7fe650103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 -- 172.21.15.31:0/2222195935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6500ffdf0 msgr2=0x7fe650103b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6500ffdf0 0x7fe650103b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 -- 172.21.15.31:0/2222195935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6440095f0 con 0x7fe650101220 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.128+0000 7fe64e7fc700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 0x7fe6501021e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe64400bd70 tx=0x7fe64400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.129+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe64401d070 con 0x7fe650101220 2024-04-22T23:41:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.129+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe650102ae0 con 0x7fe650101220 2024-04-22T23:41:14.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.129+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe644005dc0 con 0x7fe650101220 2024-04-22T23:41:14.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.129+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe644021410 con 0x7fe650101220 2024-04-22T23:41:14.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.129+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe650102fd0 con 0x7fe650101220 2024-04-22T23:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.129+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6501087c0 con 0x7fe650101220 2024-04-22T23:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.130+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe64400f7d0 con 0x7fe650101220 2024-04-22T23:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.131+0000 7fe65485f700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe63806c4e0 0x7fe63806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.131+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4358+0+0 (secure 0 0 0) 0x7fe64408c6b0 con 0x7fe650101220 2024-04-22T23:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.131+0000 7fe64effd700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe63806c4e0 0x7fe63806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.131+0000 7fe64effd700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe63806c4e0 0x7fe63806e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe640007be0 tx=0x7fe6400073d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:14.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.135+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe64405ad70 con 0x7fe650101220 2024-04-22T23:41:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.334+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe650108980 con 0x7fe650101220 2024-04-22T23:41:14.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.335+0000 7fe65485f700 1 -- 172.21.15.31:0/2222195935 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fe64400f460 con 0x7fe650101220 2024-04-22T23:41:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:41:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe63806c4e0 msgr2=0x7fe63806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe63806c4e0 0x7fe63806e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe640007be0 tx=0x7fe6400073d0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 msgr2=0x7fe6501021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 0x7fe6501021e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe64400bd70 tx=0x7fe64400bda0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 shutdown_connections 2024-04-22T23:41:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe63806c4e0 0x7fe63806e9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6500ffdf0 0x7fe650103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 --2- 172.21.15.31:0/2222195935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe650101220 0x7fe6501021e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 >> 172.21.15.31:0/2222195935 conn(0x7fe6500fa8c0 msgr2=0x7fe6500fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 shutdown_connections 2024-04-22T23:41:14.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:14.337+0000 7fe655861700 1 -- 172.21.15.31:0/2222195935 wait complete. 2024-04-22T23:41:14.627 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:14 smithi031 ceph-mon[26721]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:14.628 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:14 smithi031 ceph-mon[26721]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-22T23:41:14.628 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:14 smithi031 ceph-mon[26721]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T23:41:14.628 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:14 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:14.628 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:14 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:14.628 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:14 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2222195935' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T23:41:14.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:14 smithi044 ceph-mon[34463]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:14.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:14 smithi044 ceph-mon[34463]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-22T23:41:14.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:14 smithi044 ceph-mon[34463]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T23:41:14.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:14 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:14.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:14 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:14.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:14 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2222195935' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T23:41:14.982 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1713829248,"num_in_osds":6,"osd_in_since":1713829255,"num_remapped_pgs":0} 2024-04-22T23:41:15.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:15 smithi044 ceph-mon[34463]: purged_snaps scrub starts 2024-04-22T23:41:15.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:15 smithi044 ceph-mon[34463]: purged_snaps scrub ok 2024-04-22T23:41:15.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:15 smithi044 ceph-mon[34463]: osd.5 [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] boot 2024-04-22T23:41:15.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:15 smithi044 ceph-mon[34463]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T23:41:15.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:15.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:15 smithi031 ceph-mon[26721]: purged_snaps scrub starts 2024-04-22T23:41:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:15 smithi031 ceph-mon[26721]: purged_snaps scrub ok 2024-04-22T23:41:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:15 smithi031 ceph-mon[26721]: osd.5 [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] boot 2024-04-22T23:41:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:15 smithi031 ceph-mon[26721]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T23:41:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:41:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:15.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T23:41:16.201 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:16.639 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:16.639 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T23:41:16.639 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: Detected new or changed devices on smithi044 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi044 to 4972M 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:16.640 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:16 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-22T23:41:16.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T23:41:16.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi044 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi044 to 4972M 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:16.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:16 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:41:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 -- 172.21.15.31:0/1672185980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01017a0 msgr2=0x7fecc0103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 --2- 172.21.15.31:0/1672185980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01017a0 0x7fecc0103b90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009910 tx=0x7fecb0009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 -- 172.21.15.31:0/1672185980 shutdown_connections 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 --2- 172.21.15.31:0/1672185980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc01040d0 0x7fecc01064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 --2- 172.21.15.31:0/1672185980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01017a0 0x7fecc0103b90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 -- 172.21.15.31:0/1672185980 >> 172.21.15.31:0/1672185980 conn(0x7fecc00fb190 msgr2=0x7fecc00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 -- 172.21.15.31:0/1672185980 shutdown_connections 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.832+0000 7fecc592a700 1 -- 172.21.15.31:0/1672185980 wait complete. 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.833+0000 7fecc592a700 1 Processor -- start 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.833+0000 7fecc592a700 1 -- start start 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.833+0000 7fecc592a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc01017a0 0x7fecc0101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.833+0000 7fecc592a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 0x7fecc00ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.833+0000 7fecc592a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00ffc90 con 0x7fecc01040d0 2024-04-22T23:41:17.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.833+0000 7fecc592a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00ffdd0 con 0x7fecc01017a0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 0x7fecc00ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 0x7fecc00ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46766/0 (socket says 172.21.15.31:46766) 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 -- 172.21.15.31:0/2673494368 learned_addr learned my addr 172.21.15.31:0/2673494368 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbeffd700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc01017a0 0x7fecc0101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 -- 172.21.15.31:0/2673494368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc01017a0 msgr2=0x7fecc0101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc01017a0 0x7fecc0101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 -- 172.21.15.31:0/2673494368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00095f0 con 0x7fecc01040d0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecbe7fc700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 0x7fecc00ff750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feca800d8d0 tx=0x7feca800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca80098a0 con 0x7fecc01040d0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc01000b0 con 0x7fecc01040d0 2024-04-22T23:41:17.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feca8010460 con 0x7fecc01040d0 2024-04-22T23:41:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca800f5f0 con 0x7fecc01040d0 2024-04-22T23:41:17.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.834+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc019e4c0 con 0x7fecc01040d0 2024-04-22T23:41:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.835+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc0190610 con 0x7fecc01040d0 2024-04-22T23:41:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.836+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7feca8009a00 con 0x7fecc01040d0 2024-04-22T23:41:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.836+0000 7fecc4928700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fecac06c4e0 0x7fecac06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.836+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feca808b3d0 con 0x7fecc01040d0 2024-04-22T23:41:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.836+0000 7fecbeffd700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fecac06c4e0 0x7fecac06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.837+0000 7fecbeffd700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fecac06c4e0 0x7fecac06e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006010 tx=0x7fecb000b540 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:17.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:17.841+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feca8059980 con 0x7fecc01040d0 2024-04-22T23:41:18.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.040+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fecc00619a0 con 0x7fecc01040d0 2024-04-22T23:41:18.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.040+0000 7fecc4928700 1 -- 172.21.15.31:0/2673494368 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7feca8059510 con 0x7fecc01040d0 2024-04-22T23:41:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:41:18.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fecac06c4e0 msgr2=0x7fecac06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:18.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fecac06c4e0 0x7fecac06e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecb0006010 tx=0x7fecb000b540 comp rx=0 tx=0).stop 2024-04-22T23:41:18.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 msgr2=0x7fecc00ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:18.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 0x7fecc00ff750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feca800d8d0 tx=0x7feca800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:41:18.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 shutdown_connections 2024-04-22T23:41:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fecac06c4e0 0x7fecac06e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecc01017a0 0x7fecc0101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 --2- 172.21.15.31:0/2673494368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc01040d0 0x7fecc00ff750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 >> 172.21.15.31:0/2673494368 conn(0x7fecc00fb190 msgr2=0x7fecc00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 shutdown_connections 2024-04-22T23:41:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:18.042+0000 7fecc592a700 1 -- 172.21.15.31:0/2673494368 wait complete. 2024-04-22T23:41:18.594 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:18 smithi031 ceph-mon[26721]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:18.594 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:18 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2673494368' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T23:41:18.595 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1713829274,"num_in_osds":6,"osd_in_since":1713829255,"num_remapped_pgs":0} 2024-04-22T23:41:18.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T23:41:18.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:18 smithi044 ceph-mon[34463]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:18.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:18 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2673494368' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T23:41:18.810 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 -- 172.21.15.31:0/570114676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d0105bd0 msgr2=0x7fe2d0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/570114676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d0105bd0 0x7fe2d0105fb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009910 tx=0x7fe2c0009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 -- 172.21.15.31:0/570114676 shutdown_connections 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/570114676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/570114676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d0105bd0 0x7fe2d0105fb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 -- 172.21.15.31:0/570114676 >> 172.21.15.31:0/570114676 conn(0x7fe2d0075670 msgr2=0x7fe2d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 -- 172.21.15.31:0/570114676 shutdown_connections 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 -- 172.21.15.31:0/570114676 wait complete. 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.336+0000 7fe2d5faf700 1 Processor -- start 2024-04-22T23:41:20.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2d5faf700 1 -- start start 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2d5faf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2d5faf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2d0105bd0 0x7fe2d0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2d5faf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01022c0 con 0x7fe2d00ffc20 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2d5faf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d0102400 con 0x7fe2d0105bd0 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46782/0 (socket says 172.21.15.31:46782) 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 -- 172.21.15.31:0/1913501908 learned_addr learned my addr 172.21.15.31:0/1913501908 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2ceffd700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2d0105bd0 0x7fe2d0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 -- 172.21.15.31:0/1913501908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2d0105bd0 msgr2=0x7fe2d0101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2d0105bd0 0x7fe2d0101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 -- 172.21.15.31:0/1913501908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00095f0 con 0x7fe2d00ffc20 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.337+0000 7fe2cf7fe700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0101840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0006010 tx=0x7fe2c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.338+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c001d070 con 0x7fe2d00ffc20 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.338+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe2c0004fa0 con 0x7fe2d00ffc20 2024-04-22T23:41:20.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.338+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d0102680 con 0x7fe2d00ffc20 2024-04-22T23:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.338+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c000f690 con 0x7fe2d00ffc20 2024-04-22T23:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.338+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d01a2e10 con 0x7fe2d00ffc20 2024-04-22T23:41:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.339+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d004f2e0 con 0x7fe2d00ffc20 2024-04-22T23:41:20.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.339+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe2c000f7f0 con 0x7fe2d00ffc20 2024-04-22T23:41:20.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.339+0000 7fe2ccff9700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe2b806c4e0 0x7fe2b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:20.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.340+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe2c008cd70 con 0x7fe2d00ffc20 2024-04-22T23:41:20.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.340+0000 7fe2ceffd700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe2b806c4e0 0x7fe2b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:20.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.340+0000 7fe2ceffd700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe2b806c4e0 0x7fe2b806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0103030 tx=0x7fe2c4009190 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:20.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.344+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe2c0057a20 con 0x7fe2d00ffc20 2024-04-22T23:41:20.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.543+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2d01a30f0 con 0x7fe2d00ffc20 2024-04-22T23:41:20.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.544+0000 7fe2ccff9700 1 -- 172.21.15.31:0/1913501908 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11300 (secure 0 0 0) 0x7fe2c0027090 con 0x7fe2d00ffc20 2024-04-22T23:41:20.538 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:41:20.538 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:41:15.447593+0000","last_up_change":"2024-04-22T23:41:14.440871+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6803","nonce":1941835757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6805","nonce":1941835757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6809","nonce":1941835757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6807","nonce":1941835757}]},"public_addr":"172.21.15.31:6803/1941835757","cluster_addr":"172.21.15.31:6805/1941835757","heartbeat_back_addr":"172.21.15.31:6809/1941835757","heartbeat_front_addr":"172.21.15.31:6807/1941835757","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6811","nonce":3091658850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6813","nonce":3091658850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6817","nonce":3091658850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6815","nonce":3091658850}]},"public_addr":"172.21.15.31:6811/3091658850","cluster_addr":"172.21.15.31:6813/3091658850","heartbeat_back_addr":"172.21.15.31:6817/3091658850","heartbeat_front_addr":"172.21.15.31:6815/3091658850","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6819","nonce":3807947725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6821","nonce":3807947725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6825","nonce":3807947725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6823","nonce":3807947725}]},"public_addr":"172.21.15.31:6819/3807947725","cluster_addr":"172.21.15.31:6821/3807947725","heartbeat_back_addr":"172.21.15.31:6825/3807947725","heartbeat_front_addr":"172.21.15.31:6823/3807947725","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6801","nonce":3279105995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6803","nonce":3279105995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6807","nonce":3279105995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6805","nonce":3279105995}]},"public_addr":"172.21.15.44:6801/3279105995","cluster_addr":"172.21.15.44:6803/3279105995","heartbeat_back_addr":"172.21.15.44:6807/3279105995","heartbeat_front_addr":"172.21.15.44:6805/3279105995","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6809","nonce":3049083294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6811","nonce":3049083294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6815","nonce":3049083294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6813","nonce":3049083294}]},"public_addr":"172.21.15.44:6809/3049083294","cluster_addr":"172.21.15.44:6811/3049083294","heartbeat_back_addr":"172.21.15.44:6815/3049083294","heartbeat_front_addr":"172.21.15.44:6813/3049083294","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6817","nonce":924200036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6819","nonce":924200036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6823","nonce":924200036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6821","nonce":924200036}]},"public_addr":"172.21.15.44:6817/924200036","cluster_addr":"172.21.15.44:6819/924200036","heartbeat_back_addr":"172.21.15.44:6823/924200036","heartbeat_front_addr":"172.21.15.44:6821/924200036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.546+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe2b806c4e0 msgr2=0x7fe2b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.546+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe2b806c4e0 0x7fe2b806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0103030 tx=0x7fe2c4009190 comp rx=0 tx=0).stop 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.546+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 msgr2=0x7fe2d0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.546+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0101840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0006010 tx=0x7fe2c000bd70 comp rx=0 tx=0).stop 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 shutdown_connections 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fe2b806c4e0 0x7fe2b806e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d00ffc20 0x7fe2d0101840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 --2- 172.21.15.31:0/1913501908 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2d0105bd0 0x7fe2d0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 >> 172.21.15.31:0/1913501908 conn(0x7fe2d0075670 msgr2=0x7fe2d00685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 shutdown_connections 2024-04-22T23:41:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:20.547+0000 7fe2d5faf700 1 -- 172.21.15.31:0/1913501908 wait complete. 2024-04-22T23:41:21.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:20 smithi031 ceph-mon[26721]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:21.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:20 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1913501908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:41:21.118 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:39:59.535490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:41:21.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-22T23:41:21.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:20 smithi044 ceph-mon[34463]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:21.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:20 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1913501908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:41:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 -- 172.21.15.31:0/89203361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 msgr2=0x7f7dc8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/89203361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc8100080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009910 tx=0x7f7dbc009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 -- 172.21.15.31:0/89203361 shutdown_connections 2024-04-22T23:41:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/89203361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc8100080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/89203361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc8105bd0 0x7f7dc8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:22.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 -- 172.21.15.31:0/89203361 >> 172.21.15.31:0/89203361 conn(0x7f7dc8075670 msgr2=0x7f7dc8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 -- 172.21.15.31:0/89203361 shutdown_connections 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.818+0000 7f7dcd78d700 1 -- 172.21.15.31:0/89203361 wait complete. 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dcd78d700 1 Processor -- start 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dcd78d700 1 -- start start 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dcd78d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dcd78d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc8105bd0 0x7f7dc819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dcd78d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc819d1f0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dcd78d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc81966a0 con 0x7f7dc8105bd0 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41164/0 (socket says 172.21.15.31:41164) 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc6ffd700 1 -- 172.21.15.31:0/3807985952 learned_addr learned my addr 172.21.15.31:0/3807985952 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:22.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc67fc700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc8105bd0 0x7f7dc819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc6ffd700 1 -- 172.21.15.31:0/3807985952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc8105bd0 msgr2=0x7f7dc819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc6ffd700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc8105bd0 0x7f7dc819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.819+0000 7f7dc6ffd700 1 -- 172.21.15.31:0/3807985952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0095f0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.820+0000 7f7dc6ffd700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc819c620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7db800d8d0 tx=0x7f7db800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.820+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db80098a0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.820+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7db8010460 con 0x7f7dc80ffc20 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.820+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc8196980 con 0x7f7dc80ffc20 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.820+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db800f5f0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.820+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc8196ea0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.821+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc804f2e0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.821+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f7db8009a00 con 0x7f7dc80ffc20 2024-04-22T23:41:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.822+0000 7f7daffff700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7db006c4e0 0x7f7db006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.822+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7db808b3d0 con 0x7f7dc80ffc20 2024-04-22T23:41:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.822+0000 7f7dc67fc700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7db006c4e0 0x7f7db006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:22.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.822+0000 7f7dc67fc700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7db006c4e0 0x7f7db006e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b5c0 tx=0x7f7dbc01a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:22.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:22.826+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7db8059980 con 0x7f7dc80ffc20 2024-04-22T23:41:23.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.024+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7dc8197180 con 0x7f7dc80ffc20 2024-04-22T23:41:23.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.024+0000 7f7daffff700 1 -- 172.21.15.31:0/3807985952 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7db8059510 con 0x7f7dc80ffc20 2024-04-22T23:41:23.018 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.026+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7db006c4e0 msgr2=0x7f7db006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.026+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7db006c4e0 0x7f7db006e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00b5c0 tx=0x7f7dbc01a040 comp rx=0 tx=0).stop 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 msgr2=0x7f7dc819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc819c620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7db800d8d0 tx=0x7f7db800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 shutdown_connections 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7db006c4e0 0x7f7db006e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dc80ffc20 0x7f7dc819c620 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 --2- 172.21.15.31:0/3807985952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dc8105bd0 0x7f7dc819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 >> 172.21.15.31:0/3807985952 conn(0x7f7dc8075670 msgr2=0x7f7dc8068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 shutdown_connections 2024-04-22T23:41:23.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:23.027+0000 7f7dcd78d700 1 -- 172.21.15.31:0/3807985952 wait complete. 2024-04-22T23:41:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:22 smithi031 ceph-mon[26721]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:22 smithi044 ceph-mon[34463]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:23.515 INFO:tasks.cephadm:Setting up client nodes... 2024-04-22T23:41:23.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T23:41:23.729 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:24.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:23 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3807985952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:41:24.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:23 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3807985952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:41:25.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:24 smithi031 ceph-mon[26721]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:25.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:24 smithi044 ceph-mon[34463]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 -- 172.21.15.31:0/2906514616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 msgr2=0x7fc1300ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 --2- 172.21.15.31:0/2906514616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc1300ff160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009910 tx=0x7fc11c009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 -- 172.21.15.31:0/2906514616 shutdown_connections 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 --2- 172.21.15.31:0/2906514616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc1300ff160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 --2- 172.21.15.31:0/2906514616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 0x7fc1301026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 -- 172.21.15.31:0/2906514616 >> 172.21.15.31:0/2906514616 conn(0x7fc1300fa8c0 msgr2=0x7fc1300fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 -- 172.21.15.31:0/2906514616 shutdown_connections 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 -- 172.21.15.31:0/2906514616 wait complete. 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 Processor -- start 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.324+0000 7fc136f56700 1 -- start start 2024-04-22T23:41:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc136f56700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc130196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc136f56700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 0x7fc130196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc136f56700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301970b0 con 0x7fc1300fed00 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc136f56700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1301971f0 con 0x7fc130102280 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc130196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc130196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41188/0 (socket says 172.21.15.31:41188) 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 -- 172.21.15.31:0/3908397513 learned_addr learned my addr 172.21.15.31:0/3908397513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc12ffff700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 0x7fc130196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 -- 172.21.15.31:0/3908397513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 msgr2=0x7fc130196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 0x7fc130196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 -- 172.21.15.31:0/3908397513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0095f0 con 0x7fc1300fed00 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc12ffff700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 0x7fc130196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:41:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc134cf2700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc130196550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c8f0 tx=0x7fc12400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:25.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc124007a70 con 0x7fc1300fed00 2024-04-22T23:41:25.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc124007bd0 con 0x7fc1300fed00 2024-04-22T23:41:25.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13019bca0 con 0x7fc1300fed00 2024-04-22T23:41:25.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.325+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc124018630 con 0x7fc1300fed00 2024-04-22T23:41:25.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.326+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13019c1a0 con 0x7fc1300fed00 2024-04-22T23:41:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.326+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc130109810 con 0x7fc1300fed00 2024-04-22T23:41:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.327+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc12400f450 con 0x7fc1300fed00 2024-04-22T23:41:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.327+0000 7fc12dffb700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc12006c4e0 0x7fc12006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.327+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc12408b4f0 con 0x7fc1300fed00 2024-04-22T23:41:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.327+0000 7fc12ffff700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc12006c4e0 0x7fc12006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:25.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.328+0000 7fc12ffff700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc12006c4e0 0x7fc12006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0059d0 tx=0x7fc11c005940 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:25.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.332+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc124059b50 con 0x7fc1300fed00 2024-04-22T23:41:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.617+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc1301099d0 con 0x7fc1300fed00 2024-04-22T23:41:25.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.620+0000 7fc12dffb700 1 -- 172.21.15.31:0/3908397513 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc1301099d0 con 0x7fc1300fed00 2024-04-22T23:41:25.614 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-04-22T23:41:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCl9SZmV6TbJBAA2Li3tzHoGd08OFELc/eDIw== 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc12006c4e0 msgr2=0x7fc12006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc12006c4e0 0x7fc12006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0059d0 tx=0x7fc11c005940 comp rx=0 tx=0).stop 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 msgr2=0x7fc130196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc130196550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c8f0 tx=0x7fc12400ccb0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 shutdown_connections 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fc12006c4e0 0x7fc12006e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1300fed00 0x7fc130196550 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 --2- 172.21.15.31:0/3908397513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc130102280 0x7fc130196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:25.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 >> 172.21.15.31:0/3908397513 conn(0x7fc1300fa8c0 msgr2=0x7fc130105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 shutdown_connections 2024-04-22T23:41:25.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:25.623+0000 7fc136f56700 1 -- 172.21.15.31:0/3908397513 wait complete. 2024-04-22T23:41:26.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:25 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3908397513' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:41:26.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:25 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3908397513' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:41:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:25 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3908397513' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:41:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:25 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3908397513' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:41:26.230 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:41:26.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-22T23:41:26.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T23:41:26.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T23:41:26.493 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi044/config 2024-04-22T23:41:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:26 smithi031 ceph-mon[26721]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:26 smithi044 ceph-mon[34463]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:27.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 -- 172.21.15.44:0/2462194870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241017a0 msgr2=0x7f5a24103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/2462194870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241017a0 0x7f5a24103b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00b0a0 tx=0x7f5a1c00b3b0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 -- 172.21.15.44:0/2462194870 shutdown_connections 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/2462194870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241040d0 0x7f5a241064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/2462194870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241017a0 0x7f5a24103b90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 -- 172.21.15.44:0/2462194870 >> 172.21.15.44:0/2462194870 conn(0x7f5a240fb190 msgr2=0x7f5a240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 -- 172.21.15.44:0/2462194870 shutdown_connections 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.974+0000 7f5a2adc5700 1 -- 172.21.15.44:0/2462194870 wait complete. 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a2adc5700 1 Processor -- start 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a2adc5700 1 -- start start 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a2adc5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241017a0 0x7f5a24196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a2adc5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 0x7f5a24196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a2adc5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a241970f0 con 0x7f5a241017a0 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a2adc5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24197230 con 0x7f5a241040d0 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 0x7f5a24196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 0x7f5a24196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42588/0 (socket says 172.21.15.44:42588) 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 -- 172.21.15.44:0/3167180706 learned_addr learned my addr 172.21.15.44:0/3167180706 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a28b61700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241017a0 0x7f5a24196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 -- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241017a0 msgr2=0x7f5a24196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241017a0 0x7f5a24196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 -- 172.21.15.44:0/3167180706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c009d00 con 0x7f5a241040d0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.975+0000 7f5a23fff700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 0x7f5a24196ad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800b4b0 tx=0x7f5a1800b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.976+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a18011800 con 0x7f5a241040d0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.976+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a18011e10 con 0x7f5a241040d0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.976+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2419bce0 con 0x7f5a241040d0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.976+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1800f2e0 con 0x7f5a241040d0 2024-04-22T23:41:27.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.976+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2419c230 con 0x7f5a241040d0 2024-04-22T23:41:27.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.977+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24109880 con 0x7f5a241040d0 2024-04-22T23:41:27.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.977+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5a18010380 con 0x7f5a241040d0 2024-04-22T23:41:27.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.978+0000 7f5a21ffb700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.978+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5a1808b090 con 0x7f5a241040d0 2024-04-22T23:41:27.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.978+0000 7f5a28b61700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.978+0000 7f5a28b61700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c000f80 tx=0x7f5a1c009340 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:27.982+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5a18059640 con 0x7f5a241040d0 2024-04-22T23:41:28.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.261+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5a2419c8a0 con 0x7f5a241040d0 2024-04-22T23:41:28.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.267+0000 7f5a21ffb700 1 -- 172.21.15.44:0/3167180706 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5a180591d0 con 0x7f5a241040d0 2024-04-22T23:41:28.261 INFO:teuthology.orchestra.run.smithi044.stdout:[client.1] 2024-04-22T23:41:28.261 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCo9SZmkE2wDxAAikDiFjdlO5WrH04tTJ45qA== 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.269+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5a0c06c4e0 msgr2=0x7f5a0c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.269+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c000f80 tx=0x7f5a1c009340 comp rx=0 tx=0).stop 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.269+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 msgr2=0x7f5a24196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.269+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 0x7f5a24196ad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800b4b0 tx=0x7f5a1800b7c0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 shutdown_connections 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241017a0 0x7f5a24196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 --2- 172.21.15.44:0/3167180706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a241040d0 0x7f5a24196ad0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 >> 172.21.15.44:0/3167180706 conn(0x7f5a240fb190 msgr2=0x7f5a240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:28.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 shutdown_connections 2024-04-22T23:41:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:41:28.270+0000 7f5a2adc5700 1 -- 172.21.15.44:0/3167180706 wait complete. 2024-04-22T23:41:28.901 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:41:28.901 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-22T23:41:28.901 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-22T23:41:28.926 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:28 smithi044 ceph-mon[34463]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:28.926 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:28 smithi044 ceph-mon[34463]: from='client.? 172.21.15.44:0/3167180706' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:41:28.926 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:28 smithi044 ceph-mon[34463]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:41:28.926 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:28 smithi044 ceph-mon[34463]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:41:28.941 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T23:41:28.941 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-22T23:41:28.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-22T23:41:29.168 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:29.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:28 smithi031 ceph-mon[26721]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:29.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:28 smithi031 ceph-mon[26721]: from='client.? 172.21.15.44:0/3167180706' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:41:29.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:28 smithi031 ceph-mon[26721]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:41:29.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:28 smithi031 ceph-mon[26721]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:41:30.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 -- 172.21.15.31:0/2599668318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 msgr2=0x7fd640101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:30.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 --2- 172.21.15.31:0/2599668318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640101280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd63c009910 tx=0x7fd63c009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 -- 172.21.15.31:0/2599668318 shutdown_connections 2024-04-22T23:41:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 --2- 172.21.15.31:0/2599668318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd640101850 0x7fd6401058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 --2- 172.21.15.31:0/2599668318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640101280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 -- 172.21.15.31:0/2599668318 >> 172.21.15.31:0/2599668318 conn(0x7fd6400fc750 msgr2=0x7fd6400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 -- 172.21.15.31:0/2599668318 shutdown_connections 2024-04-22T23:41:30.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.730+0000 7fd6476ae700 1 -- 172.21.15.31:0/2599668318 wait complete. 2024-04-22T23:41:30.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd6476ae700 1 Processor -- start 2024-04-22T23:41:30.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd6476ae700 1 -- start start 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd6476ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd6476ae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd640101850 0x7fd640196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd6476ae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd640197560 con 0x7fd640100ea0 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd6476ae700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64019b370 con 0x7fd640101850 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd64544a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd64544a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39732/0 (socket says 172.21.15.31:39732) 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.731+0000 7fd64544a700 1 -- 172.21.15.31:0/4087232957 learned_addr learned my addr 172.21.15.31:0/4087232957 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd644c49700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd640101850 0x7fd640196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd64544a700 1 -- 172.21.15.31:0/4087232957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd640101850 msgr2=0x7fd640196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd64544a700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd640101850 0x7fd640196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd64544a700 1 -- 172.21.15.31:0/4087232957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0095f0 con 0x7fd640100ea0 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd64544a700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640196990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006010 tx=0x7fd63c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:30.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63c01d070 con 0x7fd640100ea0 2024-04-22T23:41:30.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd63c004fa0 con 0x7fd640100ea0 2024-04-22T23:41:30.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64019b5f0 con 0x7fd640100ea0 2024-04-22T23:41:30.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63c00f690 con 0x7fd640100ea0 2024-04-22T23:41:30.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.732+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64019bae0 con 0x7fd640100ea0 2024-04-22T23:41:30.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.733+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd640109240 con 0x7fd640100ea0 2024-04-22T23:41:30.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.734+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd63c022470 con 0x7fd640100ea0 2024-04-22T23:41:30.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.734+0000 7fd6367fc700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd62c06c4e0 0x7fd62c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:30.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.734+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd63c08cbb0 con 0x7fd640100ea0 2024-04-22T23:41:30.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.734+0000 7fd644c49700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd62c06c4e0 0x7fd62c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:30.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.735+0000 7fd644c49700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd62c06c4e0 0x7fd62c06e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd640197fb0 tx=0x7fd630009190 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:30.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.739+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd63c093080 con 0x7fd640100ea0 2024-04-22T23:41:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.996+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd640197cf0 con 0x7fd640100ea0 2024-04-22T23:41:30.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:30.999+0000 7fd6367fc700 1 -- 172.21.15.31:0/4087232957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173038 (secure 0 0 0) 0x7fd63c027090 con 0x7fd640100ea0 2024-04-22T23:41:30.995 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:41:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd62c06c4e0 msgr2=0x7fd62c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd62c06c4e0 0x7fd62c06e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd640197fb0 tx=0x7fd630009190 comp rx=0 tx=0).stop 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 msgr2=0x7fd640196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640196990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006010 tx=0x7fd63c00bd70 comp rx=0 tx=0).stop 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 shutdown_connections 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd62c06c4e0 0x7fd62c06e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd640100ea0 0x7fd640196990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 --2- 172.21.15.31:0/4087232957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd640101850 0x7fd640196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 >> 172.21.15.31:0/4087232957 conn(0x7fd6400fc750 msgr2=0x7fd6400fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 shutdown_connections 2024-04-22T23:41:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:31.002+0000 7fd6476ae700 1 -- 172.21.15.31:0/4087232957 wait complete. 2024-04-22T23:41:31.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:30 smithi031 ceph-mon[26721]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:31.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:30 smithi044 ceph-mon[34463]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:31.587 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"active_gid":14211,"active_name":"smithi031.fazdcv","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2402370856},{"type":"v1","addr":"172.21.15.31:6801","nonce":2402370856}]},"active_addr":"172.21.15.31:6801/2402370856","active_change":"2024-04-22T23:37:45.021632+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14236,"name":"smithi044.iglcti","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/","prometheus":"http://172.21.15.31:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1857354965}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1029776087}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":127255297}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3574495240}]}]} 2024-04-22T23:41:31.590 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-22T23:41:31.591 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-22T23:41:31.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T23:41:31.800 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:31.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:31 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/4087232957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T23:41:32.168 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:31 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4087232957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T23:41:33.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:32 smithi031 ceph-mon[26721]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:33.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:32 smithi044 ceph-mon[34463]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 -- 172.21.15.31:0/2958754824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 msgr2=0x7f6900101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 --2- 172.21.15.31:0/2958754824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f6900101270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f68f800b0a0 tx=0x7f68f800b3b0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 -- 172.21.15.31:0/2958754824 shutdown_connections 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 --2- 172.21.15.31:0/2958754824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6900101840 0x7f6900105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 --2- 172.21.15.31:0/2958754824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f6900101270 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 -- 172.21.15.31:0/2958754824 >> 172.21.15.31:0/2958754824 conn(0x7f69000fc740 msgr2=0x7f69000feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 -- 172.21.15.31:0/2958754824 shutdown_connections 2024-04-22T23:41:33.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.355+0000 7f69052d7700 1 -- 172.21.15.31:0/2958754824 wait complete. 2024-04-22T23:41:33.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f69052d7700 1 Processor -- start 2024-04-22T23:41:33.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f69052d7700 1 -- start start 2024-04-22T23:41:33.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f69052d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f690019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:33.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f69052d7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6900101840 0x7f690019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f69052d7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690019f440 con 0x7f6900100e90 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f69052d7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69001988f0 con 0x7f6900101840 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f690019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f690019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39754/0 (socket says 172.21.15.31:39754) 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68feffd700 1 -- 172.21.15.31:0/2184774632 learned_addr learned my addr 172.21.15.31:0/2184774632 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68fe7fc700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6900101840 0x7f690019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68feffd700 1 -- 172.21.15.31:0/2184774632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6900101840 msgr2=0x7f690019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68feffd700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6900101840 0x7f690019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.356+0000 7f68feffd700 1 -- 172.21.15.31:0/2184774632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f8009d00 con 0x7f6900100e90 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.357+0000 7f68feffd700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f690019e870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000f80 tx=0x7f68f800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.357+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f800e040 con 0x7f6900100e90 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.357+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6900198b70 con 0x7f6900100e90 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.357+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f68f8003de0 con 0x7f6900100e90 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.357+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f801d840 con 0x7f6900100e90 2024-04-22T23:41:33.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.357+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6900199060 con 0x7f6900100e90 2024-04-22T23:41:33.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.358+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6900105010 con 0x7f6900100e90 2024-04-22T23:41:33.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.358+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f68f8004040 con 0x7f6900100e90 2024-04-22T23:41:33.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.359+0000 7f68f7fff700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f68ec06c4e0 0x7f68ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:33.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.359+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f68f808cd30 con 0x7f6900100e90 2024-04-22T23:41:33.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.359+0000 7f68fe7fc700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f68ec06c4e0 0x7f68ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:33.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.359+0000 7f68fe7fc700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f68ec06c4e0 0x7f68ec06e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69000fcf20 tx=0x7f68e800b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:33.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.363+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f68f805b2e0 con 0x7f6900100e90 2024-04-22T23:41:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.563+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6900199340 con 0x7f6900100e90 2024-04-22T23:41:33.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.564+0000 7f68f7fff700 1 -- 172.21.15.31:0/2184774632 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11300 (secure 0 0 0) 0x7f68f805ae70 con 0x7f6900100e90 2024-04-22T23:41:33.557 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:41:33.558 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:41:15.447593+0000","last_up_change":"2024-04-22T23:41:14.440871+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6803","nonce":1941835757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6805","nonce":1941835757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6809","nonce":1941835757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6807","nonce":1941835757}]},"public_addr":"172.21.15.31:6803/1941835757","cluster_addr":"172.21.15.31:6805/1941835757","heartbeat_back_addr":"172.21.15.31:6809/1941835757","heartbeat_front_addr":"172.21.15.31:6807/1941835757","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6811","nonce":3091658850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6813","nonce":3091658850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6817","nonce":3091658850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6815","nonce":3091658850}]},"public_addr":"172.21.15.31:6811/3091658850","cluster_addr":"172.21.15.31:6813/3091658850","heartbeat_back_addr":"172.21.15.31:6817/3091658850","heartbeat_front_addr":"172.21.15.31:6815/3091658850","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6819","nonce":3807947725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6821","nonce":3807947725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6825","nonce":3807947725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6823","nonce":3807947725}]},"public_addr":"172.21.15.31:6819/3807947725","cluster_addr":"172.21.15.31:6821/3807947725","heartbeat_back_addr":"172.21.15.31:6825/3807947725","heartbeat_front_addr":"172.21.15.31:6823/3807947725","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6801","nonce":3279105995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6803","nonce":3279105995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6807","nonce":3279105995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6805","nonce":3279105995}]},"public_addr":"172.21.15.44:6801/3279105995","cluster_addr":"172.21.15.44:6803/3279105995","heartbeat_back_addr":"172.21.15.44:6807/3279105995","heartbeat_front_addr":"172.21.15.44:6805/3279105995","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6809","nonce":3049083294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6811","nonce":3049083294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6815","nonce":3049083294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6813","nonce":3049083294}]},"public_addr":"172.21.15.44:6809/3049083294","cluster_addr":"172.21.15.44:6811/3049083294","heartbeat_back_addr":"172.21.15.44:6815/3049083294","heartbeat_front_addr":"172.21.15.44:6813/3049083294","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6817","nonce":924200036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6819","nonce":924200036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6823","nonce":924200036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6821","nonce":924200036}]},"public_addr":"172.21.15.44:6817/924200036","cluster_addr":"172.21.15.44:6819/924200036","heartbeat_back_addr":"172.21.15.44:6823/924200036","heartbeat_front_addr":"172.21.15.44:6821/924200036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.566+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f68ec06c4e0 msgr2=0x7f68ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.566+0000 7f69052d7700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f68ec06c4e0 0x7f68ec06e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69000fcf20 tx=0x7f68e800b410 comp rx=0 tx=0).stop 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.566+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 msgr2=0x7f690019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.566+0000 7f69052d7700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f690019e870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f68f8000f80 tx=0x7f68f800cd20 comp rx=0 tx=0).stop 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 shutdown_connections 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f68ec06c4e0 0x7f68ec06e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6900100e90 0x7f690019e870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 --2- 172.21.15.31:0/2184774632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6900101840 0x7f690019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 >> 172.21.15.31:0/2184774632 conn(0x7f69000fc740 msgr2=0x7f69000fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 shutdown_connections 2024-04-22T23:41:33.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:33.567+0000 7f69052d7700 1 -- 172.21.15.31:0/2184774632 wait complete. 2024-04-22T23:41:34.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:33 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2184774632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:41:34.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:33 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2184774632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:41:34.245 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-22T23:41:34.245 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T23:41:34.460 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:35.145 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:34 smithi031 ceph-mon[26721]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:35.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:34 smithi044 ceph-mon[34463]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 -- 172.21.15.31:0/1745766709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 msgr2=0x7f47c010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 --2- 172.21.15.31:0/1745766709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c010abf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009910 tx=0x7f47b0009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 -- 172.21.15.31:0/1745766709 shutdown_connections 2024-04-22T23:41:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 --2- 172.21.15.31:0/1745766709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c010abf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 --2- 172.21.15.31:0/1745766709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c0101d50 0x7f47c0102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 -- 172.21.15.31:0/1745766709 >> 172.21.15.31:0/1745766709 conn(0x7f47c00fb610 msgr2=0x7f47c00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 -- 172.21.15.31:0/1745766709 shutdown_connections 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.901+0000 7f47c788e700 1 -- 172.21.15.31:0/1745766709 wait complete. 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c788e700 1 Processor -- start 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c788e700 1 -- start start 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c788e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c0101d50 0x7f47c00ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c788e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c00fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c788e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01004c0 con 0x7f47c0102700 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c788e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c0100600 con 0x7f47c0101d50 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c4e29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c00fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c4e29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c00fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39776/0 (socket says 172.21.15.31:39776) 2024-04-22T23:41:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.902+0000 7f47c4e29700 1 -- 172.21.15.31:0/2785775880 learned_addr learned my addr 172.21.15.31:0/2785775880 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c562a700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c0101d50 0x7f47c00ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c4e29700 1 -- 172.21.15.31:0/2785775880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c0101d50 msgr2=0x7f47c00ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c4e29700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c0101d50 0x7f47c00ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c4e29700 1 -- 172.21.15.31:0/2785775880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00095f0 con 0x7f47c0102700 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c4e29700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c00fff80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f47b00092e0 tx=0x7f47b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b001d070 con 0x7f47c0102700 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c0100880 con 0x7f47c0102700 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f47b0022470 con 0x7f47c0102700 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b000f690 con 0x7f47c0102700 2024-04-22T23:41:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.903+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c01a2e30 con 0x7f47c0102700 2024-04-22T23:41:35.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.904+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c01082f0 con 0x7f47c0102700 2024-04-22T23:41:35.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.905+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f47b00225e0 con 0x7f47c0102700 2024-04-22T23:41:35.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.905+0000 7f47b67fc700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f47ac06c4e0 0x7f47ac06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:35.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.905+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f47b008cb00 con 0x7f47c0102700 2024-04-22T23:41:35.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.905+0000 7f47c562a700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f47ac06c4e0 0x7f47ac06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:35.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.906+0000 7f47c562a700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f47ac06c4e0 0x7f47ac06e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47bc006fd0 tx=0x7f47bc008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:35.910+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f47b00576d0 con 0x7f47c0102700 2024-04-22T23:41:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.113+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47c004f2e0 con 0x7f47c0102700 2024-04-22T23:41:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.114+0000 7f47b67fc700 1 -- 172.21.15.31:0/2785775880 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11300 (secure 0 0 0) 0x7f47b005acf0 con 0x7f47c0102700 2024-04-22T23:41:36.107 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:41:36.107 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:41:15.447593+0000","last_up_change":"2024-04-22T23:41:14.440871+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6803","nonce":1941835757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6805","nonce":1941835757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6809","nonce":1941835757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6807","nonce":1941835757}]},"public_addr":"172.21.15.31:6803/1941835757","cluster_addr":"172.21.15.31:6805/1941835757","heartbeat_back_addr":"172.21.15.31:6809/1941835757","heartbeat_front_addr":"172.21.15.31:6807/1941835757","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6811","nonce":3091658850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6813","nonce":3091658850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6817","nonce":3091658850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6815","nonce":3091658850}]},"public_addr":"172.21.15.31:6811/3091658850","cluster_addr":"172.21.15.31:6813/3091658850","heartbeat_back_addr":"172.21.15.31:6817/3091658850","heartbeat_front_addr":"172.21.15.31:6815/3091658850","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6819","nonce":3807947725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6821","nonce":3807947725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6825","nonce":3807947725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6823","nonce":3807947725}]},"public_addr":"172.21.15.31:6819/3807947725","cluster_addr":"172.21.15.31:6821/3807947725","heartbeat_back_addr":"172.21.15.31:6825/3807947725","heartbeat_front_addr":"172.21.15.31:6823/3807947725","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6801","nonce":3279105995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6803","nonce":3279105995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6807","nonce":3279105995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6805","nonce":3279105995}]},"public_addr":"172.21.15.44:6801/3279105995","cluster_addr":"172.21.15.44:6803/3279105995","heartbeat_back_addr":"172.21.15.44:6807/3279105995","heartbeat_front_addr":"172.21.15.44:6805/3279105995","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6809","nonce":3049083294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6811","nonce":3049083294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6815","nonce":3049083294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6813","nonce":3049083294}]},"public_addr":"172.21.15.44:6809/3049083294","cluster_addr":"172.21.15.44:6811/3049083294","heartbeat_back_addr":"172.21.15.44:6815/3049083294","heartbeat_front_addr":"172.21.15.44:6813/3049083294","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6817","nonce":924200036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6819","nonce":924200036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6823","nonce":924200036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6821","nonce":924200036}]},"public_addr":"172.21.15.44:6817/924200036","cluster_addr":"172.21.15.44:6819/924200036","heartbeat_back_addr":"172.21.15.44:6823/924200036","heartbeat_front_addr":"172.21.15.44:6821/924200036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f47ac06c4e0 msgr2=0x7f47ac06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f47ac06c4e0 0x7f47ac06e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47bc006fd0 tx=0x7f47bc008040 comp rx=0 tx=0).stop 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 msgr2=0x7f47c00fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c00fff80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f47b00092e0 tx=0x7f47b000be20 comp rx=0 tx=0).stop 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 shutdown_connections 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f47ac06c4e0 0x7f47ac06e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c0101d50 0x7f47c00ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 --2- 172.21.15.31:0/2785775880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47c0102700 0x7f47c00fff80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:36.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 >> 172.21.15.31:0/2785775880 conn(0x7f47c00fb610 msgr2=0x7f47c0105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:36.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 shutdown_connections 2024-04-22T23:41:36.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:36.116+0000 7f47c788e700 1 -- 172.21.15.31:0/2785775880 wait complete. 2024-04-22T23:41:36.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-22T23:41:36.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-22T23:41:36.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-22T23:41:36.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-22T23:41:36.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-22T23:41:36.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-22T23:41:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:37.113 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:37.186 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:37.188 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:37.188 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:37.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:36 smithi031 ceph-mon[26721]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:37.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:36 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2785775880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:41:37.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:36 smithi044 ceph-mon[34463]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:37.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:36 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2785775880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:41:39.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:38 smithi031 ceph-mon[26721]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:39.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:38 smithi044 ceph-mon[34463]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 -- 172.21.15.31:0/3394286213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8100ea0 msgr2=0x7f5ba8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/3394286213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8100ea0 0x7f5ba8101280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009910 tx=0x7f5b98009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 -- 172.21.15.31:0/3394286213 shutdown_connections 2024-04-22T23:41:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/3394286213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba8101850 0x7f5ba81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/3394286213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8100ea0 0x7f5ba8101280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 -- 172.21.15.31:0/3394286213 >> 172.21.15.31:0/3394286213 conn(0x7f5ba80fc750 msgr2=0x7f5ba80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:40.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 -- 172.21.15.31:0/3394286213 shutdown_connections 2024-04-22T23:41:40.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.847+0000 7f5bad0b0700 1 -- 172.21.15.31:0/3394286213 wait complete. 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5bad0b0700 1 Processor -- start 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5bad0b0700 1 -- start start 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5bad0b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba8100ea0 0x7f5ba819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5bad0b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 0x7f5ba819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5bad0b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba819d2e0 con 0x7f5ba8101850 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5bad0b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba8196790 con 0x7f5ba8100ea0 2024-04-22T23:41:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5ba659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 0x7f5ba819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5ba659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 0x7f5ba819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47080/0 (socket says 172.21.15.31:47080) 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5ba659c700 1 -- 172.21.15.31:0/2329619335 learned_addr learned my addr 172.21.15.31:0/2329619335 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.848+0000 7f5ba6d9d700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba8100ea0 0x7f5ba819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5ba659c700 1 -- 172.21.15.31:0/2329619335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba8100ea0 msgr2=0x7f5ba819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5ba659c700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba8100ea0 0x7f5ba819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5ba659c700 1 -- 172.21.15.31:0/2329619335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980095f0 con 0x7f5ba8101850 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5ba659c700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 0x7f5ba819cc50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00d900 tx=0x7f5b9c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9c009900 con 0x7f5ba8101850 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b9c010460 con 0x7f5ba8101850 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b9c00f5f0 con 0x7f5ba8101850 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba8196a70 con 0x7f5ba8101850 2024-04-22T23:41:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.849+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba8196fc0 con 0x7f5ba8101850 2024-04-22T23:41:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.850+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ba810e0b0 con 0x7f5ba8101850 2024-04-22T23:41:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5b9c0105d0 con 0x7f5ba8101850 2024-04-22T23:41:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5b8ffff700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5b9006c4e0 0x7f5b9006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5b9c08b080 con 0x7f5ba8101850 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5ba6d9d700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5b9006c4e0 0x7f5b9006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5b8ffff700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] conn(0x7f5b900721c0 0x7f5b900745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 --> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b90074c90 con 0x7f5b900721c0 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b9c08d750 con 0x7f5ba8101850 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.851+0000 7f5ba759e700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] conn(0x7f5b900721c0 0x7f5b900745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.852+0000 7f5ba6d9d700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5b9006c4e0 0x7f5b9006e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006010 tx=0x7f5b9801a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.852+0000 7f5ba759e700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] conn(0x7f5b900721c0 0x7f5b900745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.852+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== osd.0 v2:172.21.15.31:6802/1941835757 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5b90074c90 con 0x7f5b900721c0 2024-04-22T23:41:40.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.868+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 --> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ba804f2e0 con 0x7f5b900721c0 2024-04-22T23:41:40.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.868+0000 7f5b8ffff700 1 -- 172.21.15.31:0/2329619335 <== osd.0 v2:172.21.15.31:6802/1941835757 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ba804f2e0 con 0x7f5b900721c0 2024-04-22T23:41:40.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] conn(0x7f5b900721c0 msgr2=0x7f5b900745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:40.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] conn(0x7f5b900721c0 0x7f5b900745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5b9006c4e0 msgr2=0x7f5b9006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:40.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5b9006c4e0 0x7f5b9006e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006010 tx=0x7f5b9801a040 comp rx=0 tx=0).stop 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 msgr2=0x7f5ba819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 0x7f5ba819cc50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00d900 tx=0x7f5b9c00dc10 comp rx=0 tx=0).stop 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 shutdown_connections 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5b9006c4e0 0x7f5b9006e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ba8100ea0 0x7f5ba819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:6802/1941835757,v1:172.21.15.31:6803/1941835757] conn(0x7f5b900721c0 0x7f5b900745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 --2- 172.21.15.31:0/2329619335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ba8101850 0x7f5ba819cc50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 >> 172.21.15.31:0/2329619335 conn(0x7f5ba80fc750 msgr2=0x7f5ba80fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 shutdown_connections 2024-04-22T23:41:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:40.869+0000 7f5bad0b0700 1 -- 172.21.15.31:0/2329619335 wait complete. 2024-04-22T23:41:41.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:40 smithi031 ceph-mon[26721]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:41.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:40 smithi044 ceph-mon[34463]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:43.186 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:42 smithi031 ceph-mon[26721]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:42 smithi044 ceph-mon[34463]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 -- 172.21.15.31:0/75533489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730101850 msgr2=0x7f07301058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 --2- 172.21.15.31:0/75533489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730101850 0x7f07301058a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0720009910 tx=0x7f0720009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 -- 172.21.15.31:0/75533489 shutdown_connections 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 --2- 172.21.15.31:0/75533489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730101850 0x7f07301058a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 --2- 172.21.15.31:0/75533489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0730100ea0 0x7f0730101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 -- 172.21.15.31:0/75533489 >> 172.21.15.31:0/75533489 conn(0x7f07300fc750 msgr2=0x7f07300feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 -- 172.21.15.31:0/75533489 shutdown_connections 2024-04-22T23:41:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 -- 172.21.15.31:0/75533489 wait complete. 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 Processor -- start 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.221+0000 7f0737e94700 1 -- start start 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0737e94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 0x7f0730196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0737e94700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0730101850 0x7f0730196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0737e94700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0730197540 con 0x7f0730100ea0 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0737e94700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073019b350 con 0x7f0730101850 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 0x7f0730196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 0x7f0730196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47098/0 (socket says 172.21.15.31:47098) 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 -- 172.21.15.31:0/3252006188 learned_addr learned my addr 172.21.15.31:0/3252006188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f073542f700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0730101850 0x7f0730196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 -- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0730101850 msgr2=0x7f0730196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0730101850 0x7f0730196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 -- 172.21.15.31:0/3252006188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07200095f0 con 0x7f0730100ea0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0735c30700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 0x7f0730196970 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b4b0 tx=0x7f072c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.222+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072c011800 con 0x7f0730100ea0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.223+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f072c011e10 con 0x7f0730100ea0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.223+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073019b630 con 0x7f0730100ea0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.223+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072c00f2e0 con 0x7f0730100ea0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.223+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073019bb80 con 0x7f0730100ea0 2024-04-22T23:41:43.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.223+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f073010e140 con 0x7f0730100ea0 2024-04-22T23:41:43.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.224+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f072c010380 con 0x7f0730100ea0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.224+0000 7f0726ffd700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f071c06c4e0 0x7f071c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f073542f700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f071c06c4e0 0x7f071c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f072c08b130 con 0x7f0730100ea0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f0726ffd700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] conn(0x7f071c0721c0 0x7f071c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 --> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f071c074c90 con 0x7f071c0721c0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f072c08d800 con 0x7f0730100ea0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f073542f700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f071c06c4e0 0x7f071c06e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0720006010 tx=0x7f0720005960 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f0736431700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] conn(0x7f071c0721c0 0x7f071c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.225+0000 7f0736431700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] conn(0x7f071c0721c0 0x7f071c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:43.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.226+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== osd.5 v2:172.21.15.44:6816/924200036 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f071c074c90 con 0x7f071c0721c0 2024-04-22T23:41:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.242+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 --> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f073004f2e0 con 0x7f071c0721c0 2024-04-22T23:41:43.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.242+0000 7f0726ffd700 1 -- 172.21.15.31:0/3252006188 <== osd.5 v2:172.21.15.44:6816/924200036 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f073004f2e0 con 0x7f071c0721c0 2024-04-22T23:41:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] conn(0x7f071c0721c0 msgr2=0x7f071c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] conn(0x7f071c0721c0 0x7f071c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f071c06c4e0 msgr2=0x7f071c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f071c06c4e0 0x7f071c06e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0720006010 tx=0x7f0720005960 comp rx=0 tx=0).stop 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 msgr2=0x7f0730196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 0x7f0730196970 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f072c00b4b0 tx=0x7f072c00b870 comp rx=0 tx=0).stop 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 shutdown_connections 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:6816/924200036,v1:172.21.15.44:6817/924200036] conn(0x7f071c0721c0 0x7f071c0745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f071c06c4e0 0x7f071c06e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0730100ea0 0x7f0730196970 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 --2- 172.21.15.31:0/3252006188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0730101850 0x7f0730196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 >> 172.21.15.31:0/3252006188 conn(0x7f07300fc750 msgr2=0x7f07300fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 shutdown_connections 2024-04-22T23:41:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.243+0000 7f0737e94700 1 -- 172.21.15.31:0/3252006188 wait complete. 2024-04-22T23:41:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.734+0000 7fbced36d700 1 -- 172.21.15.31:0/467524217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8101850 msgr2=0x7fbce81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.734+0000 7fbced36d700 1 --2- 172.21.15.31:0/467524217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8101850 0x7fbce81058a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009910 tx=0x7fbcdc009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.734+0000 7fbced36d700 1 -- 172.21.15.31:0/467524217 shutdown_connections 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.734+0000 7fbced36d700 1 --2- 172.21.15.31:0/467524217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8101850 0x7fbce81058a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.734+0000 7fbced36d700 1 --2- 172.21.15.31:0/467524217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce8100ea0 0x7fbce8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.734+0000 7fbced36d700 1 -- 172.21.15.31:0/467524217 >> 172.21.15.31:0/467524217 conn(0x7fbce80fc750 msgr2=0x7fbce80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 -- 172.21.15.31:0/467524217 shutdown_connections 2024-04-22T23:41:43.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 -- 172.21.15.31:0/467524217 wait complete. 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 Processor -- start 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 -- start start 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 0x7fbce819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce8101850 0x7fbce819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.735+0000 7fbced36d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce819f4a0 con 0x7fbce8100ea0 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbced36d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce8198950 con 0x7fbce8101850 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 0x7fbce819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 0x7fbce819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47132/0 (socket says 172.21.15.31:47132) 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 -- 172.21.15.31:0/2285378752 learned_addr learned my addr 172.21.15.31:0/2285378752 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce67fc700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce8101850 0x7fbce819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 -- 172.21.15.31:0/2285378752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce8101850 msgr2=0x7fbce819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce8101850 0x7fbce819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 -- 172.21.15.31:0/2285378752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0095f0 con 0x7fbce8100ea0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbce6ffd700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 0x7fbce819e8d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800d8d0 tx=0x7fbcd800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd80098a0 con 0x7fbce8100ea0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8010460 con 0x7fbce8100ea0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce8198c30 con 0x7fbce8100ea0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd800f5f0 con 0x7fbce8100ea0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.736+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce8199150 con 0x7fbce8100ea0 2024-04-22T23:41:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.737+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbce810e0c0 con 0x7fbce8100ea0 2024-04-22T23:41:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbcd8009a00 con 0x7fbce8100ea0 2024-04-22T23:41:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbccffff700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbcd006c4e0 0x7fbcd006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbcd808b8b0 con 0x7fbce8100ea0 2024-04-22T23:41:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbce67fc700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbcd006c4e0 0x7fbcd006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbccffff700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] conn(0x7fbcd00721c0 0x7fbcd00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 --> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbcd0074c90 con 0x7fbcd00721c0 2024-04-22T23:41:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.738+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd808bd10 con 0x7fbce8100ea0 2024-04-22T23:41:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.739+0000 7fbce77fe700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] conn(0x7fbcd00721c0 0x7fbcd00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.739+0000 7fbce67fc700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbcd006c4e0 0x7fbcd006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006010 tx=0x7fbcdc01a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.739+0000 7fbce77fe700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] conn(0x7fbcd00721c0 0x7fbcd00745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.740+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== osd.1 v2:172.21.15.31:6810/3091658850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbcd0074c90 con 0x7fbcd00721c0 2024-04-22T23:41:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.755+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 --> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbce81995b0 con 0x7fbcd00721c0 2024-04-22T23:41:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.755+0000 7fbccffff700 1 -- 172.21.15.31:0/2285378752 <== osd.1 v2:172.21.15.31:6810/3091658850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbce81995b0 con 0x7fbcd00721c0 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] conn(0x7fbcd00721c0 msgr2=0x7fbcd00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] conn(0x7fbcd00721c0 0x7fbcd00745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbcd006c4e0 msgr2=0x7fbcd006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbcd006c4e0 0x7fbcd006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006010 tx=0x7fbcdc01a040 comp rx=0 tx=0).stop 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 msgr2=0x7fbce819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 0x7fbce819e8d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800d8d0 tx=0x7fbcd800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 shutdown_connections 2024-04-22T23:41:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbcd006c4e0 0x7fbcd006e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbce8100ea0 0x7fbce819e8d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.31:6810/3091658850,v1:172.21.15.31:6811/3091658850] conn(0x7fbcd00721c0 0x7fbcd00745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 --2- 172.21.15.31:0/2285378752 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbce8101850 0x7fbce819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 >> 172.21.15.31:0/2285378752 conn(0x7fbce80fc750 msgr2=0x7fbce80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 shutdown_connections 2024-04-22T23:41:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.756+0000 7fbced36d700 1 -- 172.21.15.31:0/2285378752 wait complete. 2024-04-22T23:41:44.081 INFO:teuthology.orchestra.run.smithi031.stdout:38654705698 2024-04-22T23:41:44.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-22T23:41:44.702 INFO:teuthology.orchestra.run.smithi031.stdout:167503724553 2024-04-22T23:41:44.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-22T23:41:45.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:44 smithi031 ceph-mon[26721]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:45.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:44 smithi044 ceph-mon[34463]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:45.347 INFO:teuthology.orchestra.run.smithi031.stdout:60129542173 2024-04-22T23:41:45.347 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-22T23:41:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.412+0000 7f77fb9f0700 1 -- 172.21.15.31:0/2088059522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f4101ea0 msgr2=0x7f77f4102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/2088059522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f4101ea0 0x7f77f4102280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009910 tx=0x7f77f0009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- 172.21.15.31:0/2088059522 shutdown_connections 2024-04-22T23:41:45.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/2088059522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f41027c0 0x7f77f410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/2088059522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f4101ea0 0x7f77f4102280 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- 172.21.15.31:0/2088059522 >> 172.21.15.31:0/2088059522 conn(0x7f77f40fb650 msgr2=0x7f77f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- 172.21.15.31:0/2088059522 shutdown_connections 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- 172.21.15.31:0/2088059522 wait complete. 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 Processor -- start 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- start start 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4101ea0 0x7f77f40ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 0x7f77f4100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4100580 con 0x7f77f41027c0 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.413+0000 7f77fb9f0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f41006c0 con 0x7f77f4101ea0 2024-04-22T23:41:45.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 0x7f77f4100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 0x7f77f4100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47116/0 (socket says 172.21.15.31:47116) 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 -- 172.21.15.31:0/798877723 learned_addr learned my addr 172.21.15.31:0/798877723 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f978c700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4101ea0 0x7f77f40ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 -- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4101ea0 msgr2=0x7f77f40ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4101ea0 0x7f77f40ffb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 -- 172.21.15.31:0/798877723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00095f0 con 0x7f77f41027c0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77f8f8b700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 0x7f77f4100040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77e400b480 tx=0x7f77e400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e40118c0 con 0x7f77f41027c0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f41a2b10 con 0x7f77f41027c0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f77e4010380 con 0x7f77f41027c0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e400f4f0 con 0x7f77f41027c0 2024-04-22T23:41:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.414+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f41a2f20 con 0x7f77f41027c0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.415+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77f410e160 con 0x7f77f41027c0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f77e40104f0 con 0x7f77f41027c0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77ea7fc700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f77e006c4e0 0x7f77e006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f77e408b1d0 con 0x7f77f41027c0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77f978c700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f77e006c4e0 0x7f77e006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77ea7fc700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] conn(0x7f77e00721c0 0x7f77e00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 --> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77e0074c90 con 0x7f77e00721c0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.416+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77e408d800 con 0x7f77f41027c0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.417+0000 7f77f978c700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f77e006c4e0 0x7f77e006e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b5c0 tx=0x7f77f0005960 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.417+0000 7f77f9f8d700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] conn(0x7f77e00721c0 0x7f77e00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.417+0000 7f77f9f8d700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] conn(0x7f77e00721c0 0x7f77e00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.418+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== osd.4 v2:172.21.15.44:6808/3049083294 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f77e0074c90 con 0x7f77e00721c0 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.434+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 --> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f77f404f2e0 con 0x7f77e00721c0 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.434+0000 7f77ea7fc700 1 -- 172.21.15.31:0/798877723 <== osd.4 v2:172.21.15.44:6808/3049083294 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f77f404f2e0 con 0x7f77e00721c0 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] conn(0x7f77e00721c0 msgr2=0x7f77e00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] conn(0x7f77e00721c0 0x7f77e00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f77e006c4e0 msgr2=0x7f77e006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f77e006c4e0 0x7f77e006e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77f000b5c0 tx=0x7f77f0005960 comp rx=0 tx=0).stop 2024-04-22T23:41:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 msgr2=0x7f77f4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 0x7f77f4100040 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77e400b480 tx=0x7f77e400b840 comp rx=0 tx=0).stop 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 shutdown_connections 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:6808/3049083294,v1:172.21.15.44:6809/3049083294] conn(0x7f77e00721c0 0x7f77e00745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f77e006c4e0 0x7f77e006e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4101ea0 0x7f77f40ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 --2- 172.21.15.31:0/798877723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77f41027c0 0x7f77f4100040 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 >> 172.21.15.31:0/798877723 conn(0x7f77f40fb650 msgr2=0x7f77f41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 shutdown_connections 2024-04-22T23:41:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:43.435+0000 7f77fb9f0700 1 -- 172.21.15.31:0/798877723 wait complete. 2024-04-22T23:41:45.985 INFO:teuthology.orchestra.run.smithi031.stdout:146028888076 2024-04-22T23:41:45.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-22T23:41:46.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:46.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:46.790 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:47.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:46 smithi031 ceph-mon[26721]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:46 smithi044 ceph-mon[34463]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:47.271 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:47.272 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 -- 172.21.15.31:0/1256937211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32740ffb40 msgr2=0x7f32740fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 --2- 172.21.15.31:0/1256937211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32740ffb40 0x7f32740fffc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3260009910 tx=0x7f3260009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 -- 172.21.15.31:0/1256937211 shutdown_connections 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 --2- 172.21.15.31:0/1256937211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32740ffb40 0x7f32740fffc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 --2- 172.21.15.31:0/1256937211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3274105bf0 0x7f3274105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 -- 172.21.15.31:0/1256937211 >> 172.21.15.31:0/1256937211 conn(0x7f32740fb610 msgr2=0x7f32740fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 -- 172.21.15.31:0/1256937211 shutdown_connections 2024-04-22T23:41:47.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 -- 172.21.15.31:0/1256937211 wait complete. 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 Processor -- start 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.311+0000 7f327addf700 1 -- start start 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f327addf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32740ffb40 0x7f327419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f327addf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 0x7f327419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f327addf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327419f3d0 con 0x7f3274105bf0 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 0x7f327419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f327addf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274198880 con 0x7f32740ffb40 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 0x7f327419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47160/0 (socket says 172.21.15.31:47160) 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 -- 172.21.15.31:0/3212922376 learned_addr learned my addr 172.21.15.31:0/3212922376 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3278b7b700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32740ffb40 0x7f327419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:47.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 -- 172.21.15.31:0/3212922376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32740ffb40 msgr2=0x7f327419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32740ffb40 0x7f327419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 -- 172.21.15.31:0/3212922376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600095f0 con 0x7f3274105bf0 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3278b7b700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32740ffb40 0x7f327419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3273fff700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 0x7f327419ed40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3260006010 tx=0x7f326000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326001d070 con 0x7f3274105bf0 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3260004f50 con 0x7f3274105bf0 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3274198b00 con 0x7f3274105bf0 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.312+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3260021640 con 0x7f3274105bf0 2024-04-22T23:41:47.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.313+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3274198ff0 con 0x7f3274105bf0 2024-04-22T23:41:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.313+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f327410bf30 con 0x7f3274105bf0 2024-04-22T23:41:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.314+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f326000f460 con 0x7f3274105bf0 2024-04-22T23:41:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.314+0000 7f3271ffb700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f326406c4e0 0x7f326406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f326008c740 con 0x7f3274105bf0 2024-04-22T23:41:47.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f3278b7b700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f326406c4e0 0x7f326406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f3271ffb700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] conn(0x7f32640721c0 0x7f32640745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 --> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3264074c90 con 0x7f32640721c0 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f326008ee10 con 0x7f3274105bf0 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f327937c700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] conn(0x7f32640721c0 0x7f32640745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f3278b7b700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f326406c4e0 0x7f326406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3268005950 tx=0x7f32680058e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.315+0000 7f327937c700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] conn(0x7f32640721c0 0x7f32640745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:47.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.316+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== osd.2 v2:172.21.15.31:6818/3807947725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3264074c90 con 0x7f32640721c0 2024-04-22T23:41:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.331+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 --> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32741994d0 con 0x7f32640721c0 2024-04-22T23:41:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f3271ffb700 1 -- 172.21.15.31:0/3212922376 <== osd.2 v2:172.21.15.31:6818/3807947725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f32741994d0 con 0x7f32640721c0 2024-04-22T23:41:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] conn(0x7f32640721c0 msgr2=0x7f32640745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] conn(0x7f32640721c0 0x7f32640745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f326406c4e0 msgr2=0x7f326406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f326406c4e0 0x7f326406e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3268005950 tx=0x7f32680058e0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 msgr2=0x7f327419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:47.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 0x7f327419ed40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3260006010 tx=0x7f326000be20 comp rx=0 tx=0).stop 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 shutdown_connections 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6818/3807947725,v1:172.21.15.31:6819/3807947725] conn(0x7f32640721c0 0x7f32640745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f326406c4e0 0x7f326406e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32740ffb40 0x7f327419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 --2- 172.21.15.31:0/3212922376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3274105bf0 0x7f327419ed40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 >> 172.21.15.31:0/3212922376 conn(0x7f32740fb610 msgr2=0x7f32740fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 shutdown_connections 2024-04-22T23:41:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:47.332+0000 7f327addf700 1 -- 172.21.15.31:0/3212922376 wait complete. 2024-04-22T23:41:48.065 INFO:teuthology.orchestra.run.smithi031.stdout:81604378649 2024-04-22T23:41:48.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-22T23:41:48.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 -- 172.21.15.31:0/2545704039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 msgr2=0x7f845c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2545704039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c105890 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8448009910 tx=0x7f8448009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 -- 172.21.15.31:0/2545704039 shutdown_connections 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2545704039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c105890 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2545704039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f845c100e90 0x7f845c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 -- 172.21.15.31:0/2545704039 >> 172.21.15.31:0/2545704039 conn(0x7f845c0fc740 msgr2=0x7f845c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 -- 172.21.15.31:0/2545704039 shutdown_connections 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 -- 172.21.15.31:0/2545704039 wait complete. 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 Processor -- start 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.822+0000 7f8462e3a700 1 -- start start 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8462e3a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f845c100e90 0x7f845c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:48.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8462e3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8462e3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c19d2e0 con 0x7f845c101840 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8462e3a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c196790 con 0x7f845c100e90 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47176/0 (socket says 172.21.15.31:47176) 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 -- 172.21.15.31:0/2293857215 learned_addr learned my addr 172.21.15.31:0/2293857215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8460bd6700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f845c100e90 0x7f845c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 -- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f845c100e90 msgr2=0x7f845c19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f845c100e90 0x7f845c19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 -- 172.21.15.31:0/2293857215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480095f0 con 0x7f845c101840 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f845bfff700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c19cc50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f84480092e0 tx=0x7f844800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f844801d070 con 0x7f845c101840 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c196a10 con 0x7f845c101840 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8448005dc0 con 0x7f845c101840 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.823+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8448017400 con 0x7f845c101840 2024-04-22T23:41:48.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.824+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c196f00 con 0x7f845c101840 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.825+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f845c10e0b0 con 0x7f845c101840 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.825+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8448017640 con 0x7f845c101840 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.825+0000 7f8459ffb700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f844c06c4e0 0x7f844c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f844808c940 con 0x7f845c101840 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f8460bd6700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f844c06c4e0 0x7f844c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f8459ffb700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] conn(0x7f844c0721c0 0x7f844c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 --> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f844c074c90 con 0x7f844c0721c0 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84480912a0 con 0x7f845c101840 2024-04-22T23:41:48.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f8460bd6700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f844c06c4e0 0x7f844c06e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8450005950 tx=0x7f845000b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.826+0000 7f84613d7700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] conn(0x7f844c0721c0 0x7f844c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.827+0000 7f84613d7700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] conn(0x7f844c0721c0 0x7f844c0745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:48.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.827+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== osd.3 v2:172.21.15.44:6800/3279105995 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f844c074c90 con 0x7f844c0721c0 2024-04-22T23:41:48.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.843+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 --> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f845c04f2e0 con 0x7f844c0721c0 2024-04-22T23:41:48.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8459ffb700 1 -- 172.21.15.31:0/2293857215 <== osd.3 v2:172.21.15.44:6800/3279105995 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f845c04f2e0 con 0x7f844c0721c0 2024-04-22T23:41:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] conn(0x7f844c0721c0 msgr2=0x7f844c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] conn(0x7f844c0721c0 0x7f844c0745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f844c06c4e0 msgr2=0x7f844c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f844c06c4e0 0x7f844c06e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8450005950 tx=0x7f845000b410 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 msgr2=0x7f845c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c19cc50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f84480092e0 tx=0x7f844800bd70 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 shutdown_connections 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:6800/3279105995,v1:172.21.15.44:6801/3279105995] conn(0x7f844c0721c0 0x7f844c0745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f844c06c4e0 0x7f844c06e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f845c100e90 0x7f845c19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 --2- 172.21.15.31:0/2293857215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f845c101840 0x7f845c19cc50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 >> 172.21.15.31:0/2293857215 conn(0x7f845c0fc740 msgr2=0x7f845c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 shutdown_connections 2024-04-22T23:41:48.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:48.844+0000 7f8462e3a700 1 -- 172.21.15.31:0/2293857215 wait complete. 2024-04-22T23:41:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:48 smithi031 ceph-mon[26721]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:49.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:48 smithi044 ceph-mon[34463]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.882+0000 7f4afe58b700 1 -- 172.21.15.31:0/245086780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8068580 msgr2=0x7f4af80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.882+0000 7f4afe58b700 1 --2- 172.21.15.31:0/245086780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8068580 0x7f4af80689e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009910 tx=0x7f4ae0009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 -- 172.21.15.31:0/245086780 shutdown_connections 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 --2- 172.21.15.31:0/245086780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8068580 0x7f4af80689e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 --2- 172.21.15.31:0/245086780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af8106cd0 0x7f4af81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 -- 172.21.15.31:0/245086780 >> 172.21.15.31:0/245086780 conn(0x7f4af8075670 msgr2=0x7f4af8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 -- 172.21.15.31:0/245086780 shutdown_connections 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 -- 172.21.15.31:0/245086780 wait complete. 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 Processor -- start 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.883+0000 7f4afe58b700 1 -- start start 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4afe58b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af8068580 0x7f4af819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4afe58b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 0x7f4af819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4afe58b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af819d170 con 0x7f4af8106cd0 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4afe58b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81966b0 con 0x7f4af8068580 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 0x7f4af819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 0x7f4af819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42170/0 (socket says 172.21.15.31:42170) 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 -- 172.21.15.31:0/3938509270 learned_addr learned my addr 172.21.15.31:0/3938509270 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af7fff700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af8068580 0x7f4af819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 -- 172.21.15.31:0/3938509270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af8068580 msgr2=0x7f4af819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af8068580 0x7f4af819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 -- 172.21.15.31:0/3938509270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00095f0 con 0x7f4af8106cd0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af77fe700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 0x7f4af819cb70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae001d070 con 0x7f4af8106cd0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ae0004fa0 con 0x7f4af8106cd0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af8196930 con 0x7f4af8106cd0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.884+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae000f690 con 0x7f4af8106cd0 2024-04-22T23:41:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.885+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af8196e20 con 0x7f4af8106cd0 2024-04-22T23:41:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.886+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af81091d0 con 0x7f4af8106cd0 2024-04-22T23:41:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.886+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4ae0022470 con 0x7f4af8106cd0 2024-04-22T23:41:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.886+0000 7f4af57fa700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f4ae406c4e0 0x7f4ae406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.887+0000 7f4af7fff700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f4ae406c4e0 0x7f4ae406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.887+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4ae008cc10 con 0x7f4af8106cd0 2024-04-22T23:41:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.887+0000 7f4af7fff700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f4ae406c4e0 0x7f4ae406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800ac50 tx=0x7f4ae800a380 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:50.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:50.891+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4ae005b1c0 con 0x7f4af8106cd0 2024-04-22T23:41:51.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:50 smithi031 ceph-mon[26721]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.097+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4af8197bc0 con 0x7f4af8106cd0 2024-04-22T23:41:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.098+0000 7f4af57fa700 1 -- 172.21.15.31:0/3938509270 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ae005ad50 con 0x7f4af8106cd0 2024-04-22T23:41:51.092 INFO:teuthology.orchestra.run.smithi031.stdout:167503724554 2024-04-22T23:41:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f4ae406c4e0 msgr2=0x7f4ae406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f4ae406c4e0 0x7f4ae406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800ac50 tx=0x7f4ae800a380 comp rx=0 tx=0).stop 2024-04-22T23:41:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 msgr2=0x7f4af819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 0x7f4af819cb70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae000bd70 comp rx=0 tx=0).stop 2024-04-22T23:41:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 shutdown_connections 2024-04-22T23:41:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f4ae406c4e0 0x7f4ae406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4af8068580 0x7f4af819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 --2- 172.21.15.31:0/3938509270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4af8106cd0 0x7f4af819cb70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 >> 172.21.15.31:0/3938509270 conn(0x7f4af8075670 msgr2=0x7f4af80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 shutdown_connections 2024-04-22T23:41:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:51.100+0000 7f4afe58b700 1 -- 172.21.15.31:0/3938509270 wait complete. 2024-04-22T23:41:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:50 smithi044 ceph-mon[34463]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:52.033 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:51 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3938509270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T23:41:52.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:51 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3938509270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T23:41:52.841 INFO:teuthology.orchestra.run.smithi031.stdout:111669149716 2024-04-22T23:41:52.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-22T23:41:53.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:52 smithi031 ceph-mon[26721]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:53.123 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:53.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:52 smithi044 ceph-mon[34463]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:53.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 -- 172.21.15.31:0/820544446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 msgr2=0x7f014410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:53.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 --2- 172.21.15.31:0/820544446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f014410ac10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0138009910 tx=0x7f0138009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:53.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 -- 172.21.15.31:0/820544446 shutdown_connections 2024-04-22T23:41:53.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 --2- 172.21.15.31:0/820544446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f014410ac10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:53.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 --2- 172.21.15.31:0/820544446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0144101e00 0x7f01441021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:53.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 -- 172.21.15.31:0/820544446 >> 172.21.15.31:0/820544446 conn(0x7f01440fb650 msgr2=0x7f01440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:53.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 -- 172.21.15.31:0/820544446 shutdown_connections 2024-04-22T23:41:53.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 -- 172.21.15.31:0/820544446 wait complete. 2024-04-22T23:41:53.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.366+0000 7f014a0b1700 1 Processor -- start 2024-04-22T23:41:53.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f014a0b1700 1 -- start start 2024-04-22T23:41:53.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f014a0b1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0144101e00 0x7f01440ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:53.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f014a0b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f01440fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f014a0b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441004e0 con 0x7f0144102720 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f014a0b1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144100620 con 0x7f0144101e00 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f01440fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f01440fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42192/0 (socket says 172.21.15.31:42192) 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 -- 172.21.15.31:0/2637881042 learned_addr learned my addr 172.21.15.31:0/2637881042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f01437fe700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0144101e00 0x7f01440ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 -- 172.21.15.31:0/2637881042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0144101e00 msgr2=0x7f01440ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0144101e00 0x7f01440ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 -- 172.21.15.31:0/2637881042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380095f0 con 0x7f0144102720 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.367+0000 7f0142ffd700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f01440fffa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0138000c00 tx=0x7f013800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.368+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013801d070 con 0x7f0144102720 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.368+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f013800f460 con 0x7f0144102720 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.368+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0138017630 con 0x7f0144102720 2024-04-22T23:41:53.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.368+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01441008a0 con 0x7f0144102720 2024-04-22T23:41:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.368+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01441a2cc0 con 0x7f0144102720 2024-04-22T23:41:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.369+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144108310 con 0x7f0144102720 2024-04-22T23:41:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.369+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f013800fa00 con 0x7f0144102720 2024-04-22T23:41:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.370+0000 7f0140ff9700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f012c06c4e0 0x7f012c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.370+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f013808c920 con 0x7f0144102720 2024-04-22T23:41:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.370+0000 7f01437fe700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f012c06c4e0 0x7f012c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:53.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.370+0000 7f01437fe700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f012c06c4e0 0x7f012c06e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f013400ac50 tx=0x7f013400a380 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:53.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.374+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0138057440 con 0x7f0144102720 2024-04-22T23:41:53.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.579+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0144101030 con 0x7f0144102720 2024-04-22T23:41:53.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.579+0000 7f0140ff9700 1 -- 172.21.15.31:0/2637881042 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f013805aa60 con 0x7f0144102720 2024-04-22T23:41:53.573 INFO:teuthology.orchestra.run.smithi031.stdout:60129542174 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f012c06c4e0 msgr2=0x7f012c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f012c06c4e0 0x7f012c06e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f013400ac50 tx=0x7f013400a380 comp rx=0 tx=0).stop 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 msgr2=0x7f01440fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f01440fffa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0138000c00 tx=0x7f013800bd70 comp rx=0 tx=0).stop 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 shutdown_connections 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f012c06c4e0 0x7f012c06e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0144101e00 0x7f01440ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 --2- 172.21.15.31:0/2637881042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144102720 0x7f01440fffa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 >> 172.21.15.31:0/2637881042 conn(0x7f01440fb650 msgr2=0x7f0144105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 shutdown_connections 2024-04-22T23:41:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:53.582+0000 7f014a0b1700 1 -- 172.21.15.31:0/2637881042 wait complete. 2024-04-22T23:41:54.183 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:53 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2637881042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T23:41:54.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:53 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2637881042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 -- 172.21.15.31:0/499089743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30101840 msgr2=0x7f1b30105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/499089743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30101840 0x7f1b30105890 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009910 tx=0x7f1b18009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 -- 172.21.15.31:0/499089743 shutdown_connections 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/499089743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30101840 0x7f1b30105890 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/499089743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b30100e90 0x7f1b30101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 -- 172.21.15.31:0/499089743 >> 172.21.15.31:0/499089743 conn(0x7f1b300fc740 msgr2=0x7f1b300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 -- 172.21.15.31:0/499089743 shutdown_connections 2024-04-22T23:41:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.293+0000 7f1b34f5a700 1 -- 172.21.15.31:0/499089743 wait complete. 2024-04-22T23:41:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b34f5a700 1 Processor -- start 2024-04-22T23:41:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b34f5a700 1 -- start start 2024-04-22T23:41:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b34f5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 0x7f1b3019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b34f5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b30101840 0x7f1b3019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b34f5a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3019d2e0 con 0x7f1b30100e90 2024-04-22T23:41:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b34f5a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30196790 con 0x7f1b30101840 2024-04-22T23:41:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.294+0000 7f1b2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 0x7f1b3019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 0x7f1b3019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42208/0 (socket says 172.21.15.31:42208) 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2e59c700 1 -- 172.21.15.31:0/1138422735 learned_addr learned my addr 172.21.15.31:0/1138422735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2dd9b700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b30101840 0x7f1b3019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2e59c700 1 -- 172.21.15.31:0/1138422735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b30101840 msgr2=0x7f1b3019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2e59c700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b30101840 0x7f1b3019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2e59c700 1 -- 172.21.15.31:0/1138422735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180095f0 con 0x7f1b30100e90 2024-04-22T23:41:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b2e59c700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 0x7f1b3019c710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800ba50 tx=0x7f1b2800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b2800c900 con 0x7f1b30100e90 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1b28014440 con 0x7f1b30100e90 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b30196a70 con 0x7f1b30100e90 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b280075a0 con 0x7f1b30100e90 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.295+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b30196fc0 con 0x7f1b30100e90 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.296+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b301091e0 con 0x7f1b30100e90 2024-04-22T23:41:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.297+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1b280145b0 con 0x7f1b30100e90 2024-04-22T23:41:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.297+0000 7f1b277fe700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1b1c06c4e0 0x7f1b1c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:54.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.297+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1b28006e40 con 0x7f1b30100e90 2024-04-22T23:41:54.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.297+0000 7f1b2dd9b700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1b1c06c4e0 0x7f1b1c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:54.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.298+0000 7f1b2dd9b700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1b1c06c4e0 0x7f1b1c06e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b30197c00 tx=0x7f1b1801a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:54.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.302+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1b280591c0 con 0x7f1b30100e90 2024-04-22T23:41:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.500+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1b30197ca0 con 0x7f1b30100e90 2024-04-22T23:41:54.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.501+0000 7f1b277fe700 1 -- 172.21.15.31:0/1138422735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b28058d50 con 0x7f1b30100e90 2024-04-22T23:41:54.495 INFO:teuthology.orchestra.run.smithi031.stdout:146028888078 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1b1c06c4e0 msgr2=0x7f1b1c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1b1c06c4e0 0x7f1b1c06e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b30197c00 tx=0x7f1b1801a040 comp rx=0 tx=0).stop 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 msgr2=0x7f1b3019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 0x7f1b3019c710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800ba50 tx=0x7f1b2800bd60 comp rx=0 tx=0).stop 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 shutdown_connections 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1b1c06c4e0 0x7f1b1c06e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b30100e90 0x7f1b3019c710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 --2- 172.21.15.31:0/1138422735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b30101840 0x7f1b3019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 >> 172.21.15.31:0/1138422735 conn(0x7f1b300fc740 msgr2=0x7f1b300fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 shutdown_connections 2024-04-22T23:41:54.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:54.503+0000 7f1b34f5a700 1 -- 172.21.15.31:0/1138422735 wait complete. 2024-04-22T23:41:54.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724554 for osd.5 2024-04-22T23:41:54.810 DEBUG:teuthology.parallel:result is None 2024-04-22T23:41:55.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:54 smithi031 ceph-mon[26721]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:55.081 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:54 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1138422735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T23:41:55.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:54 smithi044 ceph-mon[34463]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:55.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:54 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1138422735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T23:41:56.153 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888076 got 146028888078 for osd.4 2024-04-22T23:41:56.153 DEBUG:teuthology.parallel:result is None 2024-04-22T23:41:56.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 -- 172.21.15.31:0/2660717972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c1006b0 msgr2=0x7f017c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 --2- 172.21.15.31:0/2660717972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c1006b0 0x7f017c100a90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0178009910 tx=0x7f0178009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 -- 172.21.15.31:0/2660717972 shutdown_connections 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 --2- 172.21.15.31:0/2660717972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f017c101060 0x7f017c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 --2- 172.21.15.31:0/2660717972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c1006b0 0x7f017c100a90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 -- 172.21.15.31:0/2660717972 >> 172.21.15.31:0/2660717972 conn(0x7f017c074e30 msgr2=0x7f017c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 -- 172.21.15.31:0/2660717972 shutdown_connections 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.865+0000 7f0184248700 1 -- 172.21.15.31:0/2660717972 wait complete. 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0184248700 1 Processor -- start 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0184248700 1 -- start start 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0184248700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f017c1006b0 0x7f017c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0184248700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 0x7f017c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0184248700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19d290 con 0x7f017c101060 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0184248700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c196740 con 0x7f017c1006b0 2024-04-22T23:41:56.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f01817e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 0x7f017c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:56.668 INFO:teuthology.orchestra.run.smithi031.stdout:81604378651 2024-04-22T23:41:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f01817e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 0x7f017c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42230/0 (socket says 172.21.15.31:42230) 2024-04-22T23:41:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f01817e3700 1 -- 172.21.15.31:0/3027554548 learned_addr learned my addr 172.21.15.31:0/3027554548 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f0181fe4700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f017c1006b0 0x7f017c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f01817e3700 1 -- 172.21.15.31:0/3027554548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f017c1006b0 msgr2=0x7f017c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f01817e3700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f017c1006b0 0x7f017c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.866+0000 7f01817e3700 1 -- 172.21.15.31:0/3027554548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780095f0 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.867+0000 7f01817e3700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 0x7f017c19cc00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f016c00b480 tx=0x7f016c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.867+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f016c011800 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.867+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f016c011e10 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.867+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c196a20 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.867+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f016c010ea0 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.867+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c196f70 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.868+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c109130 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.868+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f016c010380 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.869+0000 7f0172ffd700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f016806c4e0 0x7f016806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.869+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f016c08b060 con 0x7f017c101060 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.869+0000 7f0181fe4700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f016806c4e0 0x7f016806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.869+0000 7f0181fe4700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f016806c4e0 0x7f016806e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0178006010 tx=0x7f017801a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:55.874+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f016c059610 con 0x7f017c101060 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.077+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f017c04f2e0 con 0x7f017c101060 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.077+0000 7f0172ffd700 1 -- 172.21.15.31:0/3027554548 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f016c0591a0 con 0x7f017c101060 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f016806c4e0 msgr2=0x7f016806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f016806c4e0 0x7f016806e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0178006010 tx=0x7f017801a040 comp rx=0 tx=0).stop 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 msgr2=0x7f017c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 0x7f017c19cc00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f016c00b480 tx=0x7f016c00b790 comp rx=0 tx=0).stop 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 shutdown_connections 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f016806c4e0 0x7f016806e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f017c1006b0 0x7f017c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 --2- 172.21.15.31:0/3027554548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f017c101060 0x7f017c19cc00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 >> 172.21.15.31:0/3027554548 conn(0x7f017c074e30 msgr2=0x7f017c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 shutdown_connections 2024-04-22T23:41:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:56.079+0000 7f0184248700 1 -- 172.21.15.31:0/3027554548 wait complete. 2024-04-22T23:41:57.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:56 smithi031 ceph-mon[26721]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:57.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:56 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3027554548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T23:41:57.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:56 smithi044 ceph-mon[34463]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:57.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:56 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3027554548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T23:41:57.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-04-22T23:41:57.331 DEBUG:teuthology.parallel:result is None 2024-04-22T23:41:57.978 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542174 for osd.1 2024-04-22T23:41:57.978 DEBUG:teuthology.parallel:result is None 2024-04-22T23:41:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:41:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 -- 172.21.15.31:0/514539302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3900100750 msgr2=0x7f39001047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 --2- 172.21.15.31:0/514539302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3900100750 0x7f39001047a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009910 tx=0x7f38e8009c20 comp rx=0 tx=0).stop 2024-04-22T23:41:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 -- 172.21.15.31:0/514539302 shutdown_connections 2024-04-22T23:41:58.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 --2- 172.21.15.31:0/514539302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3900100750 0x7f39001047a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 --2- 172.21.15.31:0/514539302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39000ffda0 0x7f3900100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 -- 172.21.15.31:0/514539302 >> 172.21.15.31:0/514539302 conn(0x7f39000fb650 msgr2=0x7f39000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 -- 172.21.15.31:0/514539302 shutdown_connections 2024-04-22T23:41:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 -- 172.21.15.31:0/514539302 wait complete. 2024-04-22T23:41:58.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.177+0000 7f3905e22700 1 Processor -- start 2024-04-22T23:41:58.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f3905e22700 1 -- start start 2024-04-22T23:41:58.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f3905e22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 0x7f390019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:58.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f3905e22700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3900100750 0x7f390019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f3905e22700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390019d240 con 0x7f39000ffda0 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f3905e22700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39001966f0 con 0x7f3900100750 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 0x7f390019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38ff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 0x7f390019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42250/0 (socket says 172.21.15.31:42250) 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38ff7fe700 1 -- 172.21.15.31:0/3250569305 learned_addr learned my addr 172.21.15.31:0/3250569305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38feffd700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3900100750 0x7f390019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38ff7fe700 1 -- 172.21.15.31:0/3250569305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3900100750 msgr2=0x7f390019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38ff7fe700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3900100750 0x7f390019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.178+0000 7f38ff7fe700 1 -- 172.21.15.31:0/3250569305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80095f0 con 0x7f39000ffda0 2024-04-22T23:41:58.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.179+0000 7f38ff7fe700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 0x7f390019c670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38f000ef10 tx=0x7f38f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.179+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f000c9f0 con 0x7f39000ffda0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.179+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38f000f040 con 0x7f39000ffda0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.179+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3900196970 con 0x7f39000ffda0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.179+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f00146b0 con 0x7f39000ffda0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.179+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3900196e90 con 0x7f39000ffda0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.180+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390004f2e0 con 0x7f39000ffda0 2024-04-22T23:41:58.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.180+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f38f00078b0 con 0x7f39000ffda0 2024-04-22T23:41:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.181+0000 7f38fcff9700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f38ec06c4e0 0x7f38ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.181+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f38f008b780 con 0x7f39000ffda0 2024-04-22T23:41:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.181+0000 7f38feffd700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f38ec06c4e0 0x7f38ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:58.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.181+0000 7f38feffd700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f38ec06c4e0 0x7f38ec06e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f38e80059d0 tx=0x7f38e8005940 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:58.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.185+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f38f0059db0 con 0x7f39000ffda0 2024-04-22T23:41:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.393+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3900197130 con 0x7f39000ffda0 2024-04-22T23:41:58.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.393+0000 7f38fcff9700 1 -- 172.21.15.31:0/3250569305 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38f0059940 con 0x7f39000ffda0 2024-04-22T23:41:58.386 INFO:teuthology.orchestra.run.smithi031.stdout:38654705701 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f38ec06c4e0 msgr2=0x7f38ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f38ec06c4e0 0x7f38ec06e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f38e80059d0 tx=0x7f38e8005940 comp rx=0 tx=0).stop 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 msgr2=0x7f390019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 0x7f390019c670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38f000ef10 tx=0x7f38f000c3a0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 shutdown_connections 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f38ec06c4e0 0x7f38ec06e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39000ffda0 0x7f390019c670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 --2- 172.21.15.31:0/3250569305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3900100750 0x7f390019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 >> 172.21.15.31:0/3250569305 conn(0x7f39000fb650 msgr2=0x7f3900103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:58.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 shutdown_connections 2024-04-22T23:41:58.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:41:58.395+0000 7f3905e22700 1 -- 172.21.15.31:0/3250569305 wait complete. 2024-04-22T23:41:59.327 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:58 smithi031 ceph-mon[26721]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:59.327 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:41:58 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3250569305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T23:41:59.350 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705701 for osd.0 2024-04-22T23:41:59.350 DEBUG:teuthology.parallel:result is None 2024-04-22T23:41:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:58 smithi044 ceph-mon[34463]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:41:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:41:58 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3250569305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T23:42:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 -- 172.21.15.31:0/4018817375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 msgr2=0x7f3f6c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 --2- 172.21.15.31:0/4018817375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c1070b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009910 tx=0x7f3f5c009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:00.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 -- 172.21.15.31:0/4018817375 shutdown_connections 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 --2- 172.21.15.31:0/4018817375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 0x7f3f6c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 --2- 172.21.15.31:0/4018817375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c1070b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 -- 172.21.15.31:0/4018817375 >> 172.21.15.31:0/4018817375 conn(0x7f3f6c075670 msgr2=0x7f3f6c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 -- 172.21.15.31:0/4018817375 shutdown_connections 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.087+0000 7f3f725ba700 1 -- 172.21.15.31:0/4018817375 wait complete. 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f725ba700 1 Processor -- start 2024-04-22T23:42:00.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f725ba700 1 -- start start 2024-04-22T23:42:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f725ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 0x7f3f6c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f725ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f725ba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c19d230 con 0x7f3f6c106cd0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f725ba700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1966e0 con 0x7f3f6c068580 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f6b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42266/0 (socket says 172.21.15.31:42266) 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.088+0000 7f3f6b7fe700 1 -- 172.21.15.31:0/3081703669 learned_addr learned my addr 172.21.15.31:0/3081703669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f6bfff700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 0x7f3f6c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f6b7fe700 1 -- 172.21.15.31:0/3081703669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 msgr2=0x7f3f6c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f6b7fe700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 0x7f3f6c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f6b7fe700 1 -- 172.21.15.31:0/3081703669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0095f0 con 0x7f3f6c106cd0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f6bfff700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 0x7f3f6c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f6b7fe700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c19cba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000d8d0 tx=0x7f3f6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f60009940 con 0x7f3f6c106cd0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c1969c0 con 0x7f3f6c106cd0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f60010460 con 0x7f3f6c106cd0 2024-04-22T23:42:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6000f5f0 con 0x7f3f6c106cd0 2024-04-22T23:42:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.089+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c196f10 con 0x7f3f6c106cd0 2024-04-22T23:42:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.090+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c1091d0 con 0x7f3f6c106cd0 2024-04-22T23:42:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.091+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3f600105d0 con 0x7f3f6c106cd0 2024-04-22T23:42:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.091+0000 7f3f697fa700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.091+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3f6008b4e0 con 0x7f3f6c106cd0 2024-04-22T23:42:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.091+0000 7f3f6bfff700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.092+0000 7f3f6bfff700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00b5c0 tx=0x7f3f5c0053a0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.096+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3f60059a90 con 0x7f3f6c106cd0 2024-04-22T23:42:00.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.296+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3f6c197130 con 0x7f3f6c106cd0 2024-04-22T23:42:00.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.296+0000 7f3f697fa700 1 -- 172.21.15.31:0/3081703669 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f60059620 con 0x7f3f6c106cd0 2024-04-22T23:42:00.289 INFO:teuthology.orchestra.run.smithi031.stdout:111669149718 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3f5406c4e0 msgr2=0x7f3f5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c00b5c0 tx=0x7f3f5c0053a0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 msgr2=0x7f3f6c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c19cba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000d8d0 tx=0x7f3f6000dc90 comp rx=0 tx=0).stop 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 shutdown_connections 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f6c068580 0x7f3f6c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 --2- 172.21.15.31:0/3081703669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f6c106cd0 0x7f3f6c19cba0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.299+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 >> 172.21.15.31:0/3081703669 conn(0x7f3f6c075670 msgr2=0x7f3f6c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.300+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 shutdown_connections 2024-04-22T23:42:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:00.300+0000 7f3f725ba700 1 -- 172.21.15.31:0/3081703669 wait complete. 2024-04-22T23:42:00.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149718 for osd.3 2024-04-22T23:42:00.906 DEBUG:teuthology.parallel:result is None 2024-04-22T23:42:00.906 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-22T23:42:00.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T23:42:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:00 smithi031 ceph-mon[26721]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:00 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3081703669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T23:42:01.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:00 smithi044 ceph-mon[34463]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:01.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:01.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:00 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3081703669' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.707+0000 7eff8b43f700 1 -- 172.21.15.31:0/2619047230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff840ffb40 msgr2=0x7eff840fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.707+0000 7eff8b43f700 1 --2- 172.21.15.31:0/2619047230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff840ffb40 0x7eff840fffc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff74009910 tx=0x7eff74009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 -- 172.21.15.31:0/2619047230 shutdown_connections 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 --2- 172.21.15.31:0/2619047230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff840ffb40 0x7eff840fffc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 --2- 172.21.15.31:0/2619047230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff84105bf0 0x7eff84105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 -- 172.21.15.31:0/2619047230 >> 172.21.15.31:0/2619047230 conn(0x7eff840fb650 msgr2=0x7eff840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 -- 172.21.15.31:0/2619047230 shutdown_connections 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 -- 172.21.15.31:0/2619047230 wait complete. 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 Processor -- start 2024-04-22T23:42:02.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.708+0000 7eff8b43f700 1 -- start start 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff8b43f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff840ffb40 0x7eff84106a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff8b43f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 0x7eff84106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff8b43f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff841074e0 con 0x7eff84105bf0 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff8b43f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff84107620 con 0x7eff840ffb40 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 0x7eff84106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 0x7eff84106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51190/0 (socket says 172.21.15.31:51190) 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 -- 172.21.15.31:0/532630098 learned_addr learned my addr 172.21.15.31:0/532630098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff891db700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff840ffb40 0x7eff84106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 -- 172.21.15.31:0/532630098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff840ffb40 msgr2=0x7eff84106a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff840ffb40 0x7eff84106a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 -- 172.21.15.31:0/532630098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff740095f0 con 0x7eff84105bf0 2024-04-22T23:42:02.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff889da700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 0x7eff84106fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff74006010 tx=0x7eff7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:02.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7401d070 con 0x7eff84105bf0 2024-04-22T23:42:02.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7eff74004fa0 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff841078a0 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7400f690 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.709+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff841a2d00 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.710+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff84103e90 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.711+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7eff74022470 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.711+0000 7eff7a7fc700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7eff7006c4e0 0x7eff7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.711+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7eff7408cbf0 con 0x7eff84105bf0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.711+0000 7eff891db700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7eff7006c4e0 0x7eff7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:02.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.712+0000 7eff891db700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7eff7006c4e0 0x7eff7006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff80005950 tx=0x7eff8000a300 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:02.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.716+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7eff74057830 con 0x7eff84105bf0 2024-04-22T23:42:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.907+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff841080a0 con 0x7eff7006c4e0 2024-04-22T23:42:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.909+0000 7eff7a7fc700 1 -- 172.21.15.31:0/532630098 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28131 (secure 0 0 0) 0x7eff841080a0 con 0x7eff7006c4e0 2024-04-22T23:42:02.902 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7eff7006c4e0 msgr2=0x7eff7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7eff7006c4e0 0x7eff7006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff80005950 tx=0x7eff8000a300 comp rx=0 tx=0).stop 2024-04-22T23:42:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 msgr2=0x7eff84106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 0x7eff84106fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff74006010 tx=0x7eff7400bd70 comp rx=0 tx=0).stop 2024-04-22T23:42:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 shutdown_connections 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7eff7006c4e0 0x7eff7006e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff840ffb40 0x7eff84106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 --2- 172.21.15.31:0/532630098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff84105bf0 0x7eff84106fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 >> 172.21.15.31:0/532630098 conn(0x7eff840fb650 msgr2=0x7eff840fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 shutdown_connections 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:02.911+0000 7eff8b43f700 1 -- 172.21.15.31:0/532630098 wait complete. 2024-04-22T23:42:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-22T23:42:03.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:02 smithi031 ceph-mon[26721]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:02 smithi044 ceph-mon[34463]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:03.585 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-22T23:42:01.069532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001984"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-22T23:41:14.444952+0000","last_change":"2024-04-22T23:40:48.736627+0000","last_active":"2024-04-22T23:41:14.444952+0000","last_peered":"2024-04-22T23:41:14.444952+0000","last_clean":"2024-04-22T23:41:14.444952+0000","last_became_active":"2024-04-22T23:40:48.736180+0000","last_became_peered":"2024-04-22T23:40:48.736180+0000","last_unstale":"2024-04-22T23:41:14.444952+0000","last_undegraded":"2024-04-22T23:41:14.444952+0000","last_fullsized":"2024-04-22T23:41:14.444952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T23:39:59.576732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T23:39:59.576732+0000","last_clean_scrub_stamp":"2024-04-22T23:39:59.576732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:08.963866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]}]},{"osd":4,"up_from":34,"seq":146028888080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.9489999999999998,"5min":3.9489999999999998,"15min":3.9489999999999998},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.441}]},{"osd":2,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":16.792000000000002,"5min":16.792000000000002,"15min":16.792000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Apr 22 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.505}]},{"osd":2,"last update":"Mon Apr 22 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Apr 22 23:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717848,"statfs":{"total":95995035648,"available":95967076352,"internally_reserved":0,"allocated":360448,"data_stored":139008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.312},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.36699999999999999}]},{"osd":1,"last update":"Mon Apr 22 23:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34100000000000003}]},{"osd":3,"last update":"Mon Apr 22 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.53300000000000003}]},{"osd":4,"last update":"Mon Apr 22 23:41:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.432}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 23:41:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36199999999999999},"min":{"1min":0.27300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.32900000000000001}]},{"osd":2,"last update":"Mon Apr 22 23:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.31},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.28199999999999997}]},{"osd":3,"last update":"Mon Apr 22 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Apr 22 23:41:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:33 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.33900000000000002},"min":{"1min":0.28499999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.317}]},{"osd":2,"last update":"Mon Apr 22 23:41:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38}]},{"osd":3,"last update":"Mon Apr 22 23:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.65200000000000002}]},{"osd":4,"last update":"Mon Apr 22 23:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T23:42:03.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T23:42:03.800 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:04.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:03 smithi031 ceph-mon[26721]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:42:04.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:03 smithi044 ceph-mon[34463]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:42:05.127 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:04 smithi031 ceph-mon[26721]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 -- 172.21.15.31:0/3853004816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 msgr2=0x7f13a80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 --2- 172.21.15.31:0/3853004816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a80689e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1398009910 tx=0x7f1398009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 -- 172.21.15.31:0/3853004816 shutdown_connections 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 --2- 172.21.15.31:0/3853004816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a80689e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 --2- 172.21.15.31:0/3853004816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a8106cd0 0x7f13a81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 -- 172.21.15.31:0/3853004816 >> 172.21.15.31:0/3853004816 conn(0x7f13a8075670 msgr2=0x7f13a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 -- 172.21.15.31:0/3853004816 shutdown_connections 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.339+0000 7f13ae67b700 1 -- 172.21.15.31:0/3853004816 wait complete. 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13ae67b700 1 Processor -- start 2024-04-22T23:42:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13ae67b700 1 -- start start 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13ae67b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13ae67b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a8106cd0 0x7f13a819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13ae67b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a819d260 con 0x7f13a8068580 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13ae67b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a8196760 con 0x7f13a8106cd0 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51222/0 (socket says 172.21.15.31:51222) 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13a7fff700 1 -- 172.21.15.31:0/2985589463 learned_addr learned my addr 172.21.15.31:0/2985589463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.340+0000 7f13a77fe700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a8106cd0 0x7f13a819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a7fff700 1 -- 172.21.15.31:0/2985589463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a8106cd0 msgr2=0x7f13a819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a7fff700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a8106cd0 0x7f13a819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a7fff700 1 -- 172.21.15.31:0/2985589463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13980095f0 con 0x7f13a8068580 2024-04-22T23:42:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a7fff700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a819c690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b4b0 tx=0x7f139c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f139c011800 con 0x7f13a8068580 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f139c011e10 con 0x7f13a8068580 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a8196a40 con 0x7f13a8068580 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f139c00f2e0 con 0x7f13a8068580 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.341+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a8196f90 con 0x7f13a8068580 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.342+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a81091d0 con 0x7f13a8068580 2024-04-22T23:42:05.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.343+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f139c010380 con 0x7f13a8068580 2024-04-22T23:42:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.343+0000 7f13a57fa700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f139006c4e0 0x7f139006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.343+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f139c08b090 con 0x7f13a8068580 2024-04-22T23:42:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.343+0000 7f13a77fe700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f139006c4e0 0x7f139006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.343+0000 7f13a77fe700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f139006c4e0 0x7f139006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13a8197b80 tx=0x7f139801a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:05.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.348+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f139c059640 con 0x7f13a8068580 2024-04-22T23:42:05.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:04 smithi044 ceph-mon[34463]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:05.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.542+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13a8197210 con 0x7f139006c4e0 2024-04-22T23:42:05.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.543+0000 7f13a57fa700 1 -- 172.21.15.31:0/2985589463 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28204 (secure 0 0 0) 0x7f13a8197210 con 0x7f139006c4e0 2024-04-22T23:42:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.545+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f139006c4e0 msgr2=0x7f139006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.545+0000 7f13ae67b700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f139006c4e0 0x7f139006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13a8197b80 tx=0x7f139801a040 comp rx=0 tx=0).stop 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.545+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 msgr2=0x7f13a819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.545+0000 7f13ae67b700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a819c690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b4b0 tx=0x7f139c00b7c0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 shutdown_connections 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f139006c4e0 0x7f139006e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13a8068580 0x7f13a819c690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 --2- 172.21.15.31:0/2985589463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a8106cd0 0x7f13a819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 >> 172.21.15.31:0/2985589463 conn(0x7f13a8075670 msgr2=0x7f13a80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 shutdown_connections 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:05.546+0000 7f13ae67b700 1 -- 172.21.15.31:0/2985589463 wait complete. 2024-04-22T23:42:05.539 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-22T23:42:06.046 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-04-22T23:42:05.070241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002140"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-22T23:41:14.444952+0000","last_change":"2024-04-22T23:40:48.736627+0000","last_active":"2024-04-22T23:41:14.444952+0000","last_peered":"2024-04-22T23:41:14.444952+0000","last_clean":"2024-04-22T23:41:14.444952+0000","last_became_active":"2024-04-22T23:40:48.736180+0000","last_became_peered":"2024-04-22T23:40:48.736180+0000","last_unstale":"2024-04-22T23:41:14.444952+0000","last_undegraded":"2024-04-22T23:41:14.444952+0000","last_fullsized":"2024-04-22T23:41:14.444952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T23:39:59.576732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T23:39:59.576732+0000","last_clean_scrub_stamp":"2024-04-22T23:39:59.576732+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:08.963866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":3.9489999999999998,"5min":3.9489999999999998,"15min":3.9489999999999998},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47699999999999998}]},{"osd":2,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":16.792000000000002,"5min":16.792000000000002,"15min":16.792000000000002},"last":0.443},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.45700000000000002}]},{"osd":3,"last update":"Mon Apr 22 23:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.41699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.42099999999999999}]},{"osd":1,"last update":"Mon Apr 22 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Apr 22 23:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.495},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44900000000000001}]},{"osd":4,"last update":"Mon Apr 22 23:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.48599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717848,"statfs":{"total":95995035648,"available":95967076352,"internally_reserved":0,"allocated":360448,"data_stored":139008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.25800000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.45200000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36099999999999999,"15min":0.36499999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.46600000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.35799999999999998}]},{"osd":1,"last update":"Mon Apr 22 23:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.378},"min":{"1min":0.23100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.49099999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.26000000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.47499999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Apr 22 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Mon Apr 22 23:41:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 23:41:33 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35499999999999998,"15min":0.35299999999999998},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.48499999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36199999999999999},"min":{"1min":0.27300000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.23799999999999999}]},{"osd":2,"last update":"Mon Apr 22 23:42:01 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.51300000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.31}]},{"osd":3,"last update":"Mon Apr 22 23:41:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Mon Apr 22 23:41:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.33800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:41:33 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34499999999999997,"15min":0.33900000000000002},"min":{"1min":0.28499999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.28199999999999997}]},{"osd":2,"last update":"Mon Apr 22 23:41:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Apr 22 23:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54200000000000004}]},{"osd":4,"last update":"Mon Apr 22 23:41:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.46700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T23:42:06.047 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-22T23:42:06.047 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T23:42:06.047 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-22T23:42:06.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-22T23:42:06.256 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:06.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:05 smithi031 ceph-mon[26721]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:06.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:05 smithi031 ceph-mon[26721]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:42:06.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:05 smithi044 ceph-mon[34463]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:06.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:05 smithi044 ceph-mon[34463]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/1682139428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04100e90 msgr2=0x7f7e04101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/1682139428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04100e90 0x7f7e04101270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009910 tx=0x7f7df4009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/1682139428 shutdown_connections 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/1682139428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04101840 0x7f7e04105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/1682139428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04100e90 0x7f7e04101270 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/1682139428 >> 172.21.15.31:0/1682139428 conn(0x7f7e040fc740 msgr2=0x7f7e040feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/1682139428 shutdown_connections 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.707+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/1682139428 wait complete. 2024-04-22T23:42:07.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.708+0000 7f7e0a1c7700 1 Processor -- start 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.708+0000 7f7e0a1c7700 1 -- start start 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e0a1c7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04100e90 0x7f7e0419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e0a1c7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 0x7f7e0419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e0a1c7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0419d2a0 con 0x7f7e04101840 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e0a1c7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04196750 con 0x7f7e04100e90 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 0x7f7e0419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 0x7f7e0419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51232/0 (socket says 172.21.15.31:51232) 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 -- 172.21.15.31:0/2489471183 learned_addr learned my addr 172.21.15.31:0/2489471183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e037fe700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04100e90 0x7f7e0419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 -- 172.21.15.31:0/2489471183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04100e90 msgr2=0x7f7e0419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04100e90 0x7f7e0419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 -- 172.21.15.31:0/2489471183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df40095f0 con 0x7f7e04101840 2024-04-22T23:42:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e02ffd700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 0x7f7e0419cc10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00b480 tx=0x7f7dec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.709+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dec0118c0 con 0x7f7e04101840 2024-04-22T23:42:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.710+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e04196a30 con 0x7f7e04101840 2024-04-22T23:42:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.710+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7dec010380 con 0x7f7e04101840 2024-04-22T23:42:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.710+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dec00f4f0 con 0x7f7e04101840 2024-04-22T23:42:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.710+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e04196f80 con 0x7f7e04101840 2024-04-22T23:42:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.711+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e04109230 con 0x7f7e04101840 2024-04-22T23:42:07.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.711+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f7dec00f7b0 con 0x7f7e04101840 2024-04-22T23:42:07.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.712+0000 7f7e00ff9700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7df006c4e0 0x7f7df006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:07.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.712+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7dec08b1b0 con 0x7f7e04101840 2024-04-22T23:42:07.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.712+0000 7f7e037fe700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7df006c4e0 0x7f7df006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:07.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.712+0000 7f7e037fe700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7df006c4e0 0x7f7df006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7df4006010 tx=0x7f7df401a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.716+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7dec0596c0 con 0x7f7e04101840 2024-04-22T23:42:07.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.963+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7e0404f2e0 con 0x7f7e04101840 2024-04-22T23:42:07.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.964+0000 7f7e00ff9700 1 -- 172.21.15.31:0/2489471183 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7dec059250 con 0x7f7e04101840 2024-04-22T23:42:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:07.957 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T23:42:07.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7df006c4e0 msgr2=0x7f7df006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7df006c4e0 0x7f7df006e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7df4006010 tx=0x7f7df401a040 comp rx=0 tx=0).stop 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 msgr2=0x7f7e0419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 0x7f7e0419cc10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00b480 tx=0x7f7dec00b840 comp rx=0 tx=0).stop 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 shutdown_connections 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f7df006c4e0 0x7f7df006e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e04100e90 0x7f7e0419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 --2- 172.21.15.31:0/2489471183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e04101840 0x7f7e0419cc10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 >> 172.21.15.31:0/2489471183 conn(0x7f7e040fc740 msgr2=0x7f7e040fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 shutdown_connections 2024-04-22T23:42:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:07.966+0000 7f7e0a1c7700 1 -- 172.21.15.31:0/2489471183 wait complete. 2024-04-22T23:42:08.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:08 smithi031 ceph-mon[26721]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:08.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:08 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2489471183' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T23:42:08.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:08 smithi044 ceph-mon[34463]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:08.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:08 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2489471183' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T23:42:08.570 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-22T23:42:08.570 INFO:tasks.cephadm:Setup complete, yielding 2024-04-22T23:42:08.571 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:42:08.579 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-04-22T23:42:08.579 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:08.588 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:08.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-22T23:42:08.806 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 -- 172.21.15.31:0/136718289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c0ff540 msgr2=0x7f878c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 --2- 172.21.15.31:0/136718289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c0ff540 0x7f878c101970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f877c00b0a0 tx=0x7f877c00b3b0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 -- 172.21.15.31:0/136718289 shutdown_connections 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 --2- 172.21.15.31:0/136718289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c101eb0 0x7f878c104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 --2- 172.21.15.31:0/136718289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c0ff540 0x7f878c101970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 -- 172.21.15.31:0/136718289 >> 172.21.15.31:0/136718289 conn(0x7f878c0fb160 msgr2=0x7f878c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 -- 172.21.15.31:0/136718289 shutdown_connections 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.091+0000 7f879388a700 1 -- 172.21.15.31:0/136718289 wait complete. 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f879388a700 1 Processor -- start 2024-04-22T23:42:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f879388a700 1 -- start start 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f879388a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c0ff540 0x7f878c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f879388a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 0x7f878c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f879388a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c19d9f0 con 0x7f878c101eb0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f879388a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c19db30 con 0x7f878c0ff540 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8790e25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 0x7f878c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8790e25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 0x7f878c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51248/0 (socket says 172.21.15.31:51248) 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8790e25700 1 -- 172.21.15.31:0/68229976 learned_addr learned my addr 172.21.15.31:0/68229976 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8791626700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c0ff540 0x7f878c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8790e25700 1 -- 172.21.15.31:0/68229976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c0ff540 msgr2=0x7f878c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8790e25700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c0ff540 0x7f878c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.092+0000 7f8790e25700 1 -- 172.21.15.31:0/68229976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c009d00 con 0x7f878c101eb0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.093+0000 7f8790e25700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 0x7f878c19d3d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f878000b4b0 tx=0x7f878000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.093+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8780011800 con 0x7f878c101eb0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.093+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8780011e10 con 0x7f878c101eb0 2024-04-22T23:42:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.093+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c1a25e0 con 0x7f878c101eb0 2024-04-22T23:42:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.093+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878000f2e0 con 0x7f878c101eb0 2024-04-22T23:42:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.093+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c1a2b30 con 0x7f878c101eb0 2024-04-22T23:42:10.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.094+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c196ca0 con 0x7f878c101eb0 2024-04-22T23:42:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.094+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8780010380 con 0x7f878c101eb0 2024-04-22T23:42:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.095+0000 7f878a7fc700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f877406c4e0 0x7f877406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.095+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f878008afa0 con 0x7f878c101eb0 2024-04-22T23:42:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.095+0000 7f8791626700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f877406c4e0 0x7f877406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:10.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.095+0000 7f8791626700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f877406c4e0 0x7f877406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f877c000f80 tx=0x7f877c0060c0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:10.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.099+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f87800595d0 con 0x7f878c101eb0 2024-04-22T23:42:10.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.299+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f878c068af0 con 0x7f878c101eb0 2024-04-22T23:42:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.303+0000 7f878a7fc700 1 -- 172.21.15.31:0/68229976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8780059160 con 0x7f878c101eb0 2024-04-22T23:42:10.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.305+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f877406c4e0 msgr2=0x7f877406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.305+0000 7f879388a700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f877406c4e0 0x7f877406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f877c000f80 tx=0x7f877c0060c0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.305+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 msgr2=0x7f878c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.305+0000 7f879388a700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 0x7f878c19d3d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f878000b4b0 tx=0x7f878000b7c0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 shutdown_connections 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f877406c4e0 0x7f877406e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f878c0ff540 0x7f878c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 --2- 172.21.15.31:0/68229976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f878c101eb0 0x7f878c19d3d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 >> 172.21.15.31:0/68229976 conn(0x7f878c0fb160 msgr2=0x7f878c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 shutdown_connections 2024-04-22T23:42:10.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:10.306+0000 7f879388a700 1 -- 172.21.15.31:0/68229976 wait complete. 2024-04-22T23:42:10.892 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:42:10.901 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-22T23:42:10.901 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:10.921 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:10.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-22T23:42:11.141 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:10 smithi031 ceph-mon[26721]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:10 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/68229976' entity='client.admin' 2024-04-22T23:42:11.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:10 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:11.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:10 smithi044 ceph-mon[34463]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:11.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:10 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/68229976' entity='client.admin' 2024-04-22T23:42:11.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:10 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:12.163 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:12.164 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:12.164 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:11 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:12.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:12.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:12.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:11 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 -- 172.21.15.31:0/2690886643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 msgr2=0x7f1a48101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/2690886643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48101220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009910 tx=0x7f1a3c009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 -- 172.21.15.31:0/2690886643 shutdown_connections 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/2690886643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48101220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/2690886643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480ffbc0 0x7f1a480fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 -- 172.21.15.31:0/2690886643 >> 172.21.15.31:0/2690886643 conn(0x7f1a480fb160 msgr2=0x7f1a480fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 -- 172.21.15.31:0/2690886643 shutdown_connections 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.657+0000 7f1a4f89b700 1 -- 172.21.15.31:0/2690886643 wait complete. 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4f89b700 1 Processor -- start 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4f89b700 1 -- start start 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4f89b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480ffbc0 0x7f1a481967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4f89b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4f89b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48197330 con 0x7f1a48100dc0 2024-04-22T23:42:12.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4f89b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48197470 con 0x7f1a480ffbc0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4ce36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4ce36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45406/0 (socket says 172.21.15.31:45406) 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4ce36700 1 -- 172.21.15.31:0/1208319919 learned_addr learned my addr 172.21.15.31:0/1208319919 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4d637700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480ffbc0 0x7f1a481967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4ce36700 1 -- 172.21.15.31:0/1208319919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480ffbc0 msgr2=0x7f1a481967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.658+0000 7f1a4ce36700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480ffbc0 0x7f1a481967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a4ce36700 1 -- 172.21.15.31:0/1208319919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0095f0 con 0x7f1a48100dc0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a4ce36700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48196d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0092e0 tx=0x7f1a3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3c01d070 con 0x7f1a48100dc0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c00f460 con 0x7f1a48100dc0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4819bec0 con 0x7f1a48100dc0 2024-04-22T23:42:12.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3c021640 con 0x7f1a48100dc0 2024-04-22T23:42:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.659+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4819c380 con 0x7f1a48100dc0 2024-04-22T23:42:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.660+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a48068af0 con 0x7f1a48100dc0 2024-04-22T23:42:12.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.660+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1a3c00fa90 con 0x7f1a48100dc0 2024-04-22T23:42:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.661+0000 7f1a3a7fc700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a3406c4e0 0x7f1a3406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.661+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1a3c08c850 con 0x7f1a48100dc0 2024-04-22T23:42:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.661+0000 7f1a4d637700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a3406c4e0 0x7f1a3406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:12.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.661+0000 7f1a4d637700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a3406c4e0 0x7f1a3406e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a48100c20 tx=0x7f1a44005cf0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:12.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.665+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a3c05ae00 con 0x7f1a48100dc0 2024-04-22T23:42:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.879+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a48069560 con 0x7f1a3406c4e0 2024-04-22T23:42:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.880+0000 7f1a3a7fc700 1 -- 172.21.15.31:0/1208319919 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1a48069560 con 0x7f1a3406c4e0 2024-04-22T23:42:12.873 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-04-22T23:42:12.873 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-04-22T23:42:12.873 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a3406c4e0 msgr2=0x7f1a3406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a3406c4e0 0x7f1a3406e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a48100c20 tx=0x7f1a44005cf0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 msgr2=0x7f1a48196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48196d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0092e0 tx=0x7f1a3c00be20 comp rx=0 tx=0).stop 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 shutdown_connections 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a3406c4e0 0x7f1a3406e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a480ffbc0 0x7f1a481967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 --2- 172.21.15.31:0/1208319919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a48100dc0 0x7f1a48196d10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 >> 172.21.15.31:0/1208319919 conn(0x7f1a480fb160 msgr2=0x7f1a48068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 shutdown_connections 2024-04-22T23:42:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:12.882+0000 7f1a4f89b700 1 -- 172.21.15.31:0/1208319919 wait complete. 2024-04-22T23:42:13.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:12 smithi031 ceph-mon[26721]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:13.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:12 smithi044 ceph-mon[34463]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:13.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-22T23:42:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:14.077 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:13 smithi031 ceph-mon[26721]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:14.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:13 smithi044 ceph-mon[34463]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:15.109 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:14 smithi031 ceph-mon[26721]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.130+0000 7f5e904e5700 1 -- 172.21.15.31:0/3038075247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 msgr2=0x7f5e881010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.130+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3038075247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e881010d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009910 tx=0x7f5e84009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:15.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 -- 172.21.15.31:0/3038075247 shutdown_connections 2024-04-22T23:42:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3038075247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88101eb0 0x7f5e88102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3038075247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e881010d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 -- 172.21.15.31:0/3038075247 >> 172.21.15.31:0/3038075247 conn(0x7f5e880fc230 msgr2=0x7f5e880fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 -- 172.21.15.31:0/3038075247 shutdown_connections 2024-04-22T23:42:15.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 -- 172.21.15.31:0/3038075247 wait complete. 2024-04-22T23:42:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 Processor -- start 2024-04-22T23:42:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.131+0000 7f5e904e5700 1 -- start start 2024-04-22T23:42:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e904e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e88074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e904e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88101eb0 0x7f5e88073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e904e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880736c0 con 0x7f5e88100cb0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e904e5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88073800 con 0x7f5e88101eb0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e88074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e88074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45420/0 (socket says 172.21.15.31:45420) 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 -- 172.21.15.31:0/3426039980 learned_addr learned my addr 172.21.15.31:0/3426039980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8da80700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88101eb0 0x7f5e88073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 -- 172.21.15.31:0/3426039980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88101eb0 msgr2=0x7f5e88073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88101eb0 0x7f5e88073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 -- 172.21.15.31:0/3426039980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840095f0 con 0x7f5e88100cb0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e8e281700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e88074b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400bd70 tx=0x7f5e8400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e8401d070 con 0x7f5e88100cb0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5e84005a00 con 0x7f5e88100cb0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e88073a80 con 0x7f5e88100cb0 2024-04-22T23:42:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e84021410 con 0x7f5e88100cb0 2024-04-22T23:42:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.132+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e88073f70 con 0x7f5e88100cb0 2024-04-22T23:42:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.133+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e88109810 con 0x7f5e88100cb0 2024-04-22T23:42:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.134+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5e8400f860 con 0x7f5e88100cb0 2024-04-22T23:42:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.134+0000 7f5e7f7fe700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5e7406c4e0 0x7f5e7406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.135+0000 7f5e8da80700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5e7406c4e0 0x7f5e7406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.135+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5e8408c870 con 0x7f5e88100cb0 2024-04-22T23:42:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.135+0000 7f5e8da80700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5e7406c4e0 0x7f5e7406e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005950 tx=0x7f5e7800a300 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:15.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.139+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5e8405ae20 con 0x7f5e88100cb0 2024-04-22T23:42:15.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:14 smithi044 ceph-mon[34463]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:15.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.345+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e8819e940 con 0x7f5e7406c4e0 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.354+0000 7f5e7f7fe700 1 -- 172.21.15.31:0/3426039980 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2840 (secure 0 0 0) 0x7f5e8819e940 con 0x7f5e7406c4e0 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 2m ago 5m 26.7M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (5m) 2m ago 5m 7013k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (4m) 63s ago 4m 6979k - 18.2.1 5be31c24972a 4b26f3176132 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 2m ago 5m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:42:15.347 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (4m) 63s ago 4m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 2m ago 4m 80.4M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:9283,8765,8443 running (6m) 2m ago 6m 482M - 18.2.1 5be31c24972a dcdbc372f09c 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (4m) 63s ago 4m 445M - 18.2.1 5be31c24972a b6f8e3606e77 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (6m) 2m ago 6m 45.6M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (4m) 63s ago 4m 40.3M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (4m) 63s ago 4m 16.4M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 2m ago 3m 40.1M 3948M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 2m ago 2m 42.7M 3948M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 2m ago 2m 40.2M 3948M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (2m) 63s ago 119s 41.0M 4972M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (93s) 63s ago 93s 41.2M 4972M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (68s) 63s ago 67s 13.8M 4972M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:42:15.348 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 2m ago 4m 39.6M - 2.43.0 a07b618ecd1d b125747edf65 2024-04-22T23:42:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5e7406c4e0 msgr2=0x7f5e7406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5e7406c4e0 0x7f5e7406e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005950 tx=0x7f5e7800a300 comp rx=0 tx=0).stop 2024-04-22T23:42:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 msgr2=0x7f5e88074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e88074b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400bd70 tx=0x7f5e8400bda0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 shutdown_connections 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f5e7406c4e0 0x7f5e7406e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e88100cb0 0x7f5e88074b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 --2- 172.21.15.31:0/3426039980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88101eb0 0x7f5e88073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 >> 172.21.15.31:0/3426039980 conn(0x7f5e880fc230 msgr2=0x7f5e881050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 shutdown_connections 2024-04-22T23:42:15.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:15.357+0000 7f5e904e5700 1 -- 172.21.15.31:0/3426039980 wait complete. 2024-04-22T23:42:15.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-22T23:42:16.139 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:16.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:16.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:17.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:16 smithi031 ceph-mon[26721]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:17.144 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:16 smithi031 ceph-mon[26721]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:17.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:16 smithi044 ceph-mon[34463]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:17.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:16 smithi044 ceph-mon[34463]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 -- 172.21.15.31:0/2436370270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab540ffbc0 msgr2=0x7fab540fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 --2- 172.21.15.31:0/2436370270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab540ffbc0 0x7fab540fffe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fab44009910 tx=0x7fab44009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 -- 172.21.15.31:0/2436370270 shutdown_connections 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 --2- 172.21.15.31:0/2436370270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab54100dc0 0x7fab54101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 --2- 172.21.15.31:0/2436370270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab540ffbc0 0x7fab540fffe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 -- 172.21.15.31:0/2436370270 >> 172.21.15.31:0/2436370270 conn(0x7fab540fb160 msgr2=0x7fab540fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 -- 172.21.15.31:0/2436370270 shutdown_connections 2024-04-22T23:42:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.607+0000 7fab58d10700 1 -- 172.21.15.31:0/2436370270 wait complete. 2024-04-22T23:42:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab58d10700 1 Processor -- start 2024-04-22T23:42:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab58d10700 1 -- start start 2024-04-22T23:42:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab58d10700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540ffbc0 0x7fab54069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab58d10700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 0x7fab5419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab58d10700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5419bb70 con 0x7fab54100dc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab58d10700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5419bce0 con 0x7fab540ffbc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab51d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 0x7fab5419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab51d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 0x7fab5419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45430/0 (socket says 172.21.15.31:45430) 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab51d9b700 1 -- 172.21.15.31:0/2874491599 learned_addr learned my addr 172.21.15.31:0/2874491599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab5259c700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540ffbc0 0x7fab54069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab51d9b700 1 -- 172.21.15.31:0/2874491599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540ffbc0 msgr2=0x7fab54069990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.608+0000 7fab51d9b700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540ffbc0 0x7fab54069990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab51d9b700 1 -- 172.21.15.31:0/2874491599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440095f0 con 0x7fab54100dc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab51d9b700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 0x7fab5419b5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b4b0 tx=0x7fab4800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab48011800 con 0x7fab54100dc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fab48011e10 con 0x7fab54100dc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5419bfc0 con 0x7fab54100dc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab4800f2e0 con 0x7fab54100dc0 2024-04-22T23:42:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.609+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5419c4e0 con 0x7fab54100dc0 2024-04-22T23:42:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.610+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5418f5b0 con 0x7fab54100dc0 2024-04-22T23:42:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.610+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fab48011960 con 0x7fab54100dc0 2024-04-22T23:42:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.611+0000 7fab437fe700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fab3c06c4e0 0x7fab3c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.611+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fab4808af80 con 0x7fab54100dc0 2024-04-22T23:42:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.611+0000 7fab5259c700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fab3c06c4e0 0x7fab3c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:17.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.611+0000 7fab5259c700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fab3c06c4e0 0x7fab3c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab44006010 tx=0x7fab4400b540 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.616+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fab480595b0 con 0x7fab54100dc0 2024-04-22T23:42:17.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.826+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fab5402cfa0 con 0x7fab3c06c4e0 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.831+0000 7fab437fe700 1 -- 172.21.15.31:0/2874491599 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7fab5402cfa0 con 0x7fab3c06c4e0 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter 2/2 2m ago 5m * 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:crash 2/2 2m ago 5m * 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 2/2 2m ago 5m count:2 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:mon 2/2 2m ago 5m smithi031:172.21.15.31=smithi031;smithi044:172.21.15.44=smithi044;count:2 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-22T23:42:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:osd 6 2m ago - 2024-04-22T23:42:17.825 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-22T23:42:17.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fab3c06c4e0 msgr2=0x7fab3c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:17.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fab3c06c4e0 0x7fab3c06e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab44006010 tx=0x7fab4400b540 comp rx=0 tx=0).stop 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 msgr2=0x7fab5419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 0x7fab5419b5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b4b0 tx=0x7fab4800b7c0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 shutdown_connections 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fab3c06c4e0 0x7fab3c06e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab540ffbc0 0x7fab54069990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 --2- 172.21.15.31:0/2874491599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab54100dc0 0x7fab5419b5a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 >> 172.21.15.31:0/2874491599 conn(0x7fab540fb160 msgr2=0x7fab541040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 shutdown_connections 2024-04-22T23:42:17.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:17.833+0000 7fab58d10700 1 -- 172.21.15.31:0/2874491599 wait complete. 2024-04-22T23:42:18.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-22T23:42:18.619 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:19.045 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:18 smithi031 ceph-mon[26721]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:19.045 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:18 smithi031 ceph-mon[26721]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:19.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:18 smithi044 ceph-mon[34463]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:19.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:18 smithi044 ceph-mon[34463]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:20.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 -- 172.21.15.31:0/18326620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 msgr2=0x7fb6e0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:20.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/18326620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0102310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009910 tx=0x7fb6d4009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 -- 172.21.15.31:0/18326620 shutdown_connections 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/18326620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0102310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/18326620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e0100cb0 0x7fb6e01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 -- 172.21.15.31:0/18326620 >> 172.21.15.31:0/18326620 conn(0x7fb6e00fc230 msgr2=0x7fb6e00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 -- 172.21.15.31:0/18326620 shutdown_connections 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.130+0000 7fb6e5cff700 1 -- 172.21.15.31:0/18326620 wait complete. 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6e5cff700 1 Processor -- start 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6e5cff700 1 -- start start 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6e5cff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e0100cb0 0x7fb6e0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6e5cff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6e5cff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e00736c0 con 0x7fb6e0101eb0 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6e5cff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0073800 con 0x7fb6e0100cb0 2024-04-22T23:42:20.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45440/0 (socket says 172.21.15.31:45440) 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6deffd700 1 -- 172.21.15.31:0/211817685 learned_addr learned my addr 172.21.15.31:0/211817685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6df7fe700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e0100cb0 0x7fb6e0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6deffd700 1 -- 172.21.15.31:0/211817685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e0100cb0 msgr2=0x7fb6e0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6deffd700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e0100cb0 0x7fb6e0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.131+0000 7fb6deffd700 1 -- 172.21.15.31:0/211817685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40095f0 con 0x7fb6e0101eb0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.132+0000 7fb6deffd700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0073180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005b40 tx=0x7fb6d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.132+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d401d070 con 0x7fb6e0101eb0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.132+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb6d4022470 con 0x7fb6e0101eb0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.132+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0073a80 con 0x7fb6e0101eb0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.132+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d400f690 con 0x7fb6e0101eb0 2024-04-22T23:42:20.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.132+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e0073f70 con 0x7fb6e0101eb0 2024-04-22T23:42:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.133+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e0109810 con 0x7fb6e0101eb0 2024-04-22T23:42:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.133+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb6d40225e0 con 0x7fb6e0101eb0 2024-04-22T23:42:20.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.134+0000 7fb6dcff9700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb6c806c4e0 0x7fb6c806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.134+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb6d408ccc0 con 0x7fb6e0101eb0 2024-04-22T23:42:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.134+0000 7fb6df7fe700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb6c806c4e0 0x7fb6c806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:20.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.134+0000 7fb6df7fe700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb6c806c4e0 0x7fb6c806e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d0005e50 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:20.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.138+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb6d4092020 con 0x7fb6e0101eb0 2024-04-22T23:42:20.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.345+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e0106800 con 0x7fb6c806c4e0 2024-04-22T23:42:20.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.346+0000 7fb6dcff9700 1 -- 172.21.15.31:0/211817685 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7fb6e0106800 con 0x7fb6c806c4e0 2024-04-22T23:42:20.339 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-04-22T23:42:20.339 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-04-22T23:42:20.339 INFO:teuthology.orchestra.run.smithi031.stdout:smithi044 172.21.15.44 2024-04-22T23:42:20.339 INFO:teuthology.orchestra.run.smithi031.stdout:2 hosts in cluster 2024-04-22T23:42:20.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb6c806c4e0 msgr2=0x7fb6c806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:20.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb6c806c4e0 0x7fb6c806e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d0005e50 comp rx=0 tx=0).stop 2024-04-22T23:42:20.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 msgr2=0x7fb6e0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0073180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005b40 tx=0x7fb6d400be20 comp rx=0 tx=0).stop 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 shutdown_connections 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fb6c806c4e0 0x7fb6c806e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6e0100cb0 0x7fb6e0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 --2- 172.21.15.31:0/211817685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6e0101eb0 0x7fb6e0073180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.348+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 >> 172.21.15.31:0/211817685 conn(0x7fb6e00fc230 msgr2=0x7fb6e01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.349+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 shutdown_connections 2024-04-22T23:42:20.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:20.349+0000 7fb6e5cff700 1 -- 172.21.15.31:0/211817685 wait complete. 2024-04-22T23:42:20.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-22T23:42:21.167 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:21.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:20 smithi031 ceph-mon[26721]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:21.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:20 smithi044 ceph-mon[34463]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:22.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:21 smithi031 ceph-mon[26721]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:22.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:21 smithi044 ceph-mon[34463]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 -- 172.21.15.31:0/623728341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23580ffe70 msgr2=0x7f23581002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 --2- 172.21.15.31:0/623728341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23580ffe70 0x7f23581002d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2340009910 tx=0x7f2340009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 -- 172.21.15.31:0/623728341 shutdown_connections 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 --2- 172.21.15.31:0/623728341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23580ffe70 0x7f23581002d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 --2- 172.21.15.31:0/623728341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2358101130 0x7f2358101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 -- 172.21.15.31:0/623728341 >> 172.21.15.31:0/623728341 conn(0x7f23580fa8c0 msgr2=0x7f23580fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 -- 172.21.15.31:0/623728341 shutdown_connections 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.560+0000 7f235d5bd700 1 -- 172.21.15.31:0/623728341 wait complete. 2024-04-22T23:42:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f235d5bd700 1 Processor -- start 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f235d5bd700 1 -- start start 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f235d5bd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23580ffe70 0x7f23581964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f235d5bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 0x7f2358196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f235d5bd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358197020 con 0x7f2358101130 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f235d5bd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358197160 con 0x7f23580ffe70 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f23567fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 0x7f2358196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f23567fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 0x7f2358196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46992/0 (socket says 172.21.15.31:46992) 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f23567fc700 1 -- 172.21.15.31:0/3189021900 learned_addr learned my addr 172.21.15.31:0/3189021900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f2356ffd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23580ffe70 0x7f23581964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f23567fc700 1 -- 172.21.15.31:0/3189021900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23580ffe70 msgr2=0x7f23581964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f23567fc700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23580ffe70 0x7f23581964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.561+0000 7f23567fc700 1 -- 172.21.15.31:0/3189021900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23400095f0 con 0x7f2358101130 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.562+0000 7f23567fc700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 0x7f2358196a00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2340006010 tx=0x7f234000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.562+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234001d070 con 0x7f2358101130 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.562+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f234000f460 con 0x7f2358101130 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.562+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235819bbb0 con 0x7f2358101130 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.562+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2340021640 con 0x7f2358101130 2024-04-22T23:42:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.562+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235819c0a0 con 0x7f2358101130 2024-04-22T23:42:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.563+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2358108710 con 0x7f2358101130 2024-04-22T23:42:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.563+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f234000fab0 con 0x7f2358101130 2024-04-22T23:42:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.564+0000 7f234ffff700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f234406c4e0 0x7f234406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.564+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f234008c700 con 0x7f2358101130 2024-04-22T23:42:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.564+0000 7f2356ffd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f234406c4e0 0x7f234406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.564+0000 7f2356ffd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f234406c4e0 0x7f234406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2358068b90 tx=0x7f234800b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.568+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f234005ac90 con 0x7f2358101130 2024-04-22T23:42:22.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.776+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f23580ff470 con 0x7f234406c4e0 2024-04-22T23:42:22.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.777+0000 7f234ffff700 1 -- 172.21.15.31:0/3189021900 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f23580ff470 con 0x7f234406c4e0 2024-04-22T23:42:22.770 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-22T23:42:22.770 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T23:42:22.770 INFO:teuthology.orchestra.run.smithi031.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 66s ago Has a FileSystem, LVM detected 2024-04-22T23:42:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f234406c4e0 msgr2=0x7f234406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f234406c4e0 0x7f234406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2358068b90 tx=0x7f234800b410 comp rx=0 tx=0).stop 2024-04-22T23:42:22.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 msgr2=0x7f2358196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 0x7f2358196a00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2340006010 tx=0x7f234000bd70 comp rx=0 tx=0).stop 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 shutdown_connections 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f234406c4e0 0x7f234406e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23580ffe70 0x7f23581964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 --2- 172.21.15.31:0/3189021900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2358101130 0x7f2358196a00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.779+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 >> 172.21.15.31:0/3189021900 conn(0x7f23580fa8c0 msgr2=0x7f23580fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.780+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 shutdown_connections 2024-04-22T23:42:22.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:22.780+0000 7f235d5bd700 1 -- 172.21.15.31:0/3189021900 wait complete. 2024-04-22T23:42:23.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:22 smithi031 ceph-mon[26721]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:22 smithi044 ceph-mon[34463]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:23.375 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:23.386 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:23.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-22T23:42:23.602 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:24.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:23 smithi031 ceph-mon[26721]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:24.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:23 smithi044 ceph-mon[34463]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 -- 172.21.15.31:0/1728437601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 msgr2=0x7f0ae4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/1728437601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae4101270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009910 tx=0x7f0ad8009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 -- 172.21.15.31:0/1728437601 shutdown_connections 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/1728437601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae4101270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/1728437601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 -- 172.21.15.31:0/1728437601 >> 172.21.15.31:0/1728437601 conn(0x7f0ae40fb190 msgr2=0x7f0ae40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 -- 172.21.15.31:0/1728437601 shutdown_connections 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.161+0000 7f0ae99d4700 1 -- 172.21.15.31:0/1728437601 wait complete. 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae99d4700 1 Processor -- start 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae99d4700 1 -- start start 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae99d4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae99d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae41021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae99d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4102730 con 0x7f0ae4100e10 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae99d4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4102870 con 0x7f0ae40ffc10 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae41021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae41021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47006/0 (socket says 172.21.15.31:47006) 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae27fc700 1 -- 172.21.15.31:0/2717309614 learned_addr learned my addr 172.21.15.31:0/2717309614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae2ffd700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae27fc700 1 -- 172.21.15.31:0/2717309614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae40ffc10 msgr2=0x7f0ae4103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae27fc700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.162+0000 7f0ae27fc700 1 -- 172.21.15.31:0/2717309614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad80095f0 con 0x7f0ae4100e10 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.163+0000 7f0ae27fc700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae41021f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8006010 tx=0x7f0ad800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.163+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad801d070 con 0x7f0ae4100e10 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.163+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae4102af0 con 0x7f0ae4100e10 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.163+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ad800f460 con 0x7f0ae4100e10 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.163+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad8021640 con 0x7f0ae4100e10 2024-04-22T23:42:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.163+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae4102fe0 con 0x7f0ae4100e10 2024-04-22T23:42:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.164+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae4108700 con 0x7f0ae4100e10 2024-04-22T23:42:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.164+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0ad800fab0 con 0x7f0ae4100e10 2024-04-22T23:42:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.165+0000 7f0ae89d2700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0acc06c4e0 0x7f0acc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.165+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0ad808c730 con 0x7f0ae4100e10 2024-04-22T23:42:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.165+0000 7f0ae2ffd700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0acc06c4e0 0x7f0acc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.165+0000 7f0ae2ffd700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0acc06c4e0 0x7f0acc06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4100c70 tx=0x7f0ad400b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:25.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.169+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ad805ace0 con 0x7f0ae4100e10 2024-04-22T23:42:25.162 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:24 smithi031 ceph-mon[26721]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:25.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:24 smithi044 ceph-mon[34463]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:25.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:25.397+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ae419e900 con 0x7f0acc06c4e0 2024-04-22T23:42:26.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:25 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-22T23:42:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:25 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-22T23:42:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.971+0000 7f0ae89d2700 1 -- 172.21.15.31:0/2717309614 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0ae419e900 con 0x7f0acc06c4e0 2024-04-22T23:42:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0acc06c4e0 msgr2=0x7f0acc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:26.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0acc06c4e0 0x7f0acc06e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4100c70 tx=0x7f0ad400b410 comp rx=0 tx=0).stop 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 msgr2=0x7f0ae41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae41021f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8006010 tx=0x7f0ad800bd70 comp rx=0 tx=0).stop 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 shutdown_connections 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0acc06c4e0 0x7f0acc06e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ae40ffc10 0x7f0ae4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 --2- 172.21.15.31:0/2717309614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4100e10 0x7f0ae41021f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 >> 172.21.15.31:0/2717309614 conn(0x7f0ae40fb190 msgr2=0x7f0ae4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 shutdown_connections 2024-04-22T23:42:26.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:26.974+0000 7f0ae99d4700 1 -- 172.21.15.31:0/2717309614 wait complete. 2024-04-22T23:42:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:26 smithi031 ceph-mon[26721]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:26 smithi031 ceph-mon[26721]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:26 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[26717]: 2024-04-22T23:42:26.947+0000 7f9637d8f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:42:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-22T23:42:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:26 smithi031 ceph-mon[26721]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T23:42:27.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:42:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:26 smithi044 ceph-mon[34463]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:26 smithi044 ceph-mon[34463]: from='client.14468 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-22T23:42:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:26 smithi044 ceph-mon[34463]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T23:42:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:42:27.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T23:42:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-22T23:42:28.176 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: fsmap cephfs:0 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: Saving service mds.cephfs spec with placement count:4 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:28.177 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:27 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:28.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-22T23:42:28.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T23:42:28.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:42:28.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:42:28.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: fsmap cephfs:0 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: Saving service mds.cephfs spec with placement count:4 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:28.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:27 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:29.043 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:28 smithi031 ceph-mon[26721]: pgmap v172: 65 pgs: 9 creating+peering, 55 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:29.043 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:28 smithi031 ceph-mon[26721]: Deploying daemon mds.cephfs.smithi031.ezuwxk on smithi031 2024-04-22T23:42:29.043 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:28 smithi031 ceph-mon[26721]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T23:42:29.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:28 smithi044 ceph-mon[34463]: pgmap v172: 65 pgs: 9 creating+peering, 55 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:29.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:28 smithi044 ceph-mon[34463]: Deploying daemon mds.cephfs.smithi031.ezuwxk on smithi031 2024-04-22T23:42:29.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:28 smithi044 ceph-mon[34463]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T23:42:29.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 -- 172.21.15.31:0/1436616435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0100cb0 msgr2=0x7f0aa01010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:29.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1436616435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0100cb0 0x7f0aa01010d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009910 tx=0x7f0a90009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:29.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 -- 172.21.15.31:0/1436616435 shutdown_connections 2024-04-22T23:42:29.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1436616435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:29.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1436616435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0100cb0 0x7f0aa01010d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 -- 172.21.15.31:0/1436616435 >> 172.21.15.31:0/1436616435 conn(0x7f0aa00fc230 msgr2=0x7f0aa00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 -- 172.21.15.31:0/1436616435 shutdown_connections 2024-04-22T23:42:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.270+0000 7f0aa6857700 1 -- 172.21.15.31:0/1436616435 wait complete. 2024-04-22T23:42:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0aa6857700 1 Processor -- start 2024-04-22T23:42:29.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0aa6857700 1 -- start start 2024-04-22T23:42:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0aa6857700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa0100cb0 0x7f0aa01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0aa6857700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0aa6857700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0199230 con 0x7f0aa0101eb0 2024-04-22T23:42:29.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0aa6857700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa0199370 con 0x7f0aa0100cb0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47014/0 (socket says 172.21.15.31:47014) 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9f7fe700 1 -- 172.21.15.31:0/1942989911 learned_addr learned my addr 172.21.15.31:0/1942989911 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9ffff700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa0100cb0 0x7f0aa01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9f7fe700 1 -- 172.21.15.31:0/1942989911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa0100cb0 msgr2=0x7f0aa01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9f7fe700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa0100cb0 0x7f0aa01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.271+0000 7f0a9f7fe700 1 -- 172.21.15.31:0/1942989911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900095f0 con 0x7f0aa0101eb0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.272+0000 7f0a9f7fe700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0198c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c8a0 tx=0x7f0a9400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.272+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a9400cea0 con 0x7f0aa0101eb0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.272+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa019de20 con 0x7f0aa0101eb0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.272+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a94004830 con 0x7f0aa0101eb0 2024-04-22T23:42:29.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.272+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a940055b0 con 0x7f0aa0101eb0 2024-04-22T23:42:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.272+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa019e340 con 0x7f0aa0101eb0 2024-04-22T23:42:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.273+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa0105f90 con 0x7f0aa0101eb0 2024-04-22T23:42:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.273+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0a94007ad0 con 0x7f0aa0101eb0 2024-04-22T23:42:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.274+0000 7f0a9d7fa700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0a8806c4e0 0x7f0a8806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.274+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0a9408b470 con 0x7f0aa0101eb0 2024-04-22T23:42:29.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.274+0000 7f0a9ffff700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0a8806c4e0 0x7f0a8806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:29.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.274+0000 7f0a9ffff700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0a8806c4e0 0x7f0a8806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006010 tx=0x7f0a90005920 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:29.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.279+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0a94059760 con 0x7f0aa0101eb0 2024-04-22T23:42:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.529+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0aa002cc70 con 0x7f0aa0101eb0 2024-04-22T23:42:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.529+0000 7f0a9d7fa700 1 -- 172.21.15.31:0/1942989911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f0a9401d460 con 0x7f0aa0101eb0 2024-04-22T23:42:29.522 INFO:teuthology.orchestra.run.smithi031.stdout:e2 2024-04-22T23:42:29.522 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 2 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:26.948268+0000 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:42:29.523 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:in 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:up {} 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-04-22T23:42:29.524 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:29.525 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:29.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0a8806c4e0 msgr2=0x7f0a8806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:29.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0a8806c4e0 0x7f0a8806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006010 tx=0x7f0a90005920 comp rx=0 tx=0).stop 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 msgr2=0x7f0aa0198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0198c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c8a0 tx=0x7f0a9400cc60 comp rx=0 tx=0).stop 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 shutdown_connections 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f0a8806c4e0 0x7f0a8806e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0aa0100cb0 0x7f0aa01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 --2- 172.21.15.31:0/1942989911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0aa0101eb0 0x7f0aa0198c10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 >> 172.21.15.31:0/1942989911 conn(0x7f0aa00fc230 msgr2=0x7f0aa01030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 shutdown_connections 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:29.531+0000 7f0aa6857700 1 -- 172.21.15.31:0/1942989911 wait complete. 2024-04-22T23:42:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-22T23:42:30.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:29 smithi031 ceph-mon[26721]: osdmap e45: 6 total, 6 up, 6 in 2024-04-22T23:42:30.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:29 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1942989911' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:29 smithi044 ceph-mon[34463]: osdmap e45: 6 total, 6 up, 6 in 2024-04-22T23:42:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:29 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1942989911' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:30.465 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:30.475 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:30.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-22T23:42:30.959 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:31.179 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:30 smithi031 ceph-mon[26721]: pgmap v175: 65 pgs: 14 creating+peering, 37 unknown, 14 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:31.179 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:31.179 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:30 smithi044 ceph-mon[34463]: pgmap v175: 65 pgs: 14 creating+peering, 37 unknown, 14 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:32.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/3475517071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48104060 msgr2=0x7ffb48106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/3475517071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48104060 0x7ffb48106450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009910 tx=0x7ffb38009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:32.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/3475517071 shutdown_connections 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/3475517071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48104060 0x7ffb48106450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/3475517071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb48101730 0x7ffb48103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/3475517071 >> 172.21.15.31:0/3475517071 conn(0x7ffb480fb120 msgr2=0x7ffb480fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/3475517071 shutdown_connections 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.965+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/3475517071 wait complete. 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4f5ba700 1 Processor -- start 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4f5ba700 1 -- start start 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4f5ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 0x7ffb48196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4f5ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb48104060 0x7ffb481969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4f5ba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48196fe0 con 0x7ffb48101730 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4f5ba700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb48197120 con 0x7ffb48104060 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 0x7ffb48196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 0x7ffb48196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56714/0 (socket says 172.21.15.31:56714) 2024-04-22T23:42:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 -- 172.21.15.31:0/4065644698 learned_addr learned my addr 172.21.15.31:0/4065644698 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4cb55700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb48104060 0x7ffb481969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 -- 172.21.15.31:0/4065644698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb48104060 msgr2=0x7ffb481969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb48104060 0x7ffb481969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 -- 172.21.15.31:0/4065644698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380095f0 con 0x7ffb48101730 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.966+0000 7ffb4d356700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 0x7ffb48196480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400d900 tx=0x7ffb4400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.967+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb440041f0 con 0x7ffb48101730 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.967+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb44004ce0 con 0x7ffb48101730 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.967+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4819bbd0 con 0x7ffb48101730 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.967+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4400b750 con 0x7ffb48101730 2024-04-22T23:42:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.967+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4819c120 con 0x7ffb48101730 2024-04-22T23:42:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.968+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb48190610 con 0x7ffb48101730 2024-04-22T23:42:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.968+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ffb44004e50 con 0x7ffb48101730 2024-04-22T23:42:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.969+0000 7ffb3e7fc700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.969+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7ffb4408b710 con 0x7ffb48101730 2024-04-22T23:42:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.969+0000 7ffb4cb55700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.969+0000 7ffb4cb55700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b5c0 tx=0x7ffb38005ee0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:32.973+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffb44059980 con 0x7ffb48101730 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: pgmap v176: 65 pgs: 14 creating+peering, 6 unknown, 45 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:33.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:32 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:33.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.224+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7ffb48061a40 con 0x7ffb48101730 2024-04-22T23:42:33.284 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: pgmap v176: 65 pgs: 14 creating+peering, 6 unknown, 45 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:33.285 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:33.285 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:33.285 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:33.285 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:33.285 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:33.285 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:32 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:33.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.986+0000 7ffb3e7fc700 1 -- 172.21.15.31:0/4065644698 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7ffb44059510 con 0x7ffb48101730 2024-04-22T23:42:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.988+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ffb3406c4e0 msgr2=0x7ffb3406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.988+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b5c0 tx=0x7ffb38005ee0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.988+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 msgr2=0x7ffb48196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.988+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 0x7ffb48196480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffb4400d900 tx=0x7ffb4400dc10 comp rx=0 tx=0).stop 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 shutdown_connections 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb48101730 0x7ffb48196480 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 --2- 172.21.15.31:0/4065644698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb48104060 0x7ffb481969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 >> 172.21.15.31:0/4065644698 conn(0x7ffb480fb120 msgr2=0x7ffb480fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 shutdown_connections 2024-04-22T23:42:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:33.989+0000 7ffb4f5ba700 1 -- 172.21.15.31:0/4065644698 wait complete. 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: Deploying daemon mds.cephfs.smithi044.ibkztg on smithi044 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.31:6826/3543943016,v1:172.21.15.31:6827/3543943016] up:boot 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: daemon mds.cephfs.smithi031.ezuwxk assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: Cluster is now healthy 2024-04-22T23:42:34.097 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: fsmap cephfs:0 1 up:standby 2024-04-22T23:42:34.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:42:34.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:creating} 2024-04-22T23:42:34.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: daemon mds.cephfs.smithi031.ezuwxk is now active in filesystem cephfs as rank 0 2024-04-22T23:42:34.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:33 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/4065644698' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-22T23:42:34.285 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: Deploying daemon mds.cephfs.smithi044.ibkztg on smithi044 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.31:6826/3543943016,v1:172.21.15.31:6827/3543943016] up:boot 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: daemon mds.cephfs.smithi031.ezuwxk assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: Cluster is now healthy 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: fsmap cephfs:0 1 up:standby 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:creating} 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: daemon mds.cephfs.smithi031.ezuwxk is now active in filesystem cephfs as rank 0 2024-04-22T23:42:34.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:33 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4065644698' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-22T23:42:34.613 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:34.623 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:34.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-04-22T23:42:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:35.053 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:34 smithi044 ceph-mon[34463]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:35.053 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:34 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.31:6826/3543943016,v1:172.21.15.31:6827/3543943016] up:active 2024-04-22T23:42:35.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:34 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/4065644698' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-22T23:42:35.054 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:34 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 2024-04-22T23:42:35.287 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:34 smithi031 ceph-mon[26721]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:42:35.288 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:34 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.31:6826/3543943016,v1:172.21.15.31:6827/3543943016] up:active 2024-04-22T23:42:35.288 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:34 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4065644698' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-22T23:42:35.288 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:34 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 2024-04-22T23:42:36.219 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: pgmap v178: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: Deploying daemon mds.cephfs.smithi031.usiyoa on smithi031 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:boot 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:42:36.220 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:36 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby 2024-04-22T23:42:36.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 -- 172.21.15.31:0/2747122230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 msgr2=0x7f374c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:36.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 --2- 172.21.15.31:0/2747122230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c102310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3740009910 tx=0x7f3740009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:36.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 -- 172.21.15.31:0/2747122230 shutdown_connections 2024-04-22T23:42:36.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 --2- 172.21.15.31:0/2747122230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c102310 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:36.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 --2- 172.21.15.31:0/2747122230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c100cb0 0x7f374c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 -- 172.21.15.31:0/2747122230 >> 172.21.15.31:0/2747122230 conn(0x7f374c0fc230 msgr2=0x7f374c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 -- 172.21.15.31:0/2747122230 shutdown_connections 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.434+0000 7f37524c9700 1 -- 172.21.15.31:0/2747122230 wait complete. 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f37524c9700 1 Processor -- start 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f37524c9700 1 -- start start 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f37524c9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c100cb0 0x7f374c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f37524c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f37524c9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c197010 con 0x7f374c101eb0 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f37524c9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c197150 con 0x7f374c100cb0 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f374b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:36.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f374b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56724/0 (socket says 172.21.15.31:56724) 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f374b7fe700 1 -- 172.21.15.31:0/3572222220 learned_addr learned my addr 172.21.15.31:0/3572222220 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.435+0000 7f374bfff700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c100cb0 0x7f374c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f374b7fe700 1 -- 172.21.15.31:0/3572222220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c100cb0 msgr2=0x7f374c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f374b7fe700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c100cb0 0x7f374c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f374b7fe700 1 -- 172.21.15.31:0/3572222220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400095f0 con 0x7f374c101eb0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f374b7fe700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c1969f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f374000b5c0 tx=0x7f374000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f374001d070 con 0x7f374c101eb0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c19bba0 con 0x7f374c101eb0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f374000f460 con 0x7f374c101eb0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3740021640 con 0x7f374c101eb0 2024-04-22T23:42:36.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.436+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c19c090 con 0x7f374c101eb0 2024-04-22T23:42:36.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.437+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374c109810 con 0x7f374c101eb0 2024-04-22T23:42:36.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.438+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f374000fa60 con 0x7f374c101eb0 2024-04-22T23:42:36.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.438+0000 7f37497fa700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f373406c4e0 0x7f373406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:36.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.438+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f374008ca10 con 0x7f374c101eb0 2024-04-22T23:42:36.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.438+0000 7f374bfff700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f373406c4e0 0x7f373406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:36.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.439+0000 7f374bfff700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f373406c4e0 0x7f373406e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f374c101d10 tx=0x7f373c00b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:36.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.443+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f374005ac80 con 0x7f374c101eb0 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: pgmap v178: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: Deploying daemon mds.cephfs.smithi031.usiyoa on smithi031 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:boot 2024-04-22T23:42:36.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby 2024-04-22T23:42:36.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:42:36.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:36 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby 2024-04-22T23:42:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:36.691+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f374c109a00 con 0x7f374c101eb0 2024-04-22T23:42:37.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.205+0000 7f37497fa700 1 -- 172.21.15.31:0/3572222220 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v8) v1 ==== 121+0+0 (secure 0 0 0) 0x7f374005a810 con 0x7f374c101eb0 2024-04-22T23:42:37.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f373406c4e0 msgr2=0x7f373406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f373406c4e0 0x7f373406e9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f374c101d10 tx=0x7f373c00b410 comp rx=0 tx=0).stop 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 msgr2=0x7f374c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c1969f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f374000b5c0 tx=0x7f374000bd70 comp rx=0 tx=0).stop 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 shutdown_connections 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f373406c4e0 0x7f373406e9a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374c100cb0 0x7f374c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 --2- 172.21.15.31:0/3572222220 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f374c101eb0 0x7f374c1969f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 >> 172.21.15.31:0/3572222220 conn(0x7f374c0fc230 msgr2=0x7f374c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 shutdown_connections 2024-04-22T23:42:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:37.208+0000 7f37524c9700 1 -- 172.21.15.31:0/3572222220 wait complete. 2024-04-22T23:42:37.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:37 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3572222220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T23:42:37.447 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:37 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3572222220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T23:42:38.192 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:38.201 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:38.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-22T23:42:38.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:38 smithi031 ceph-mon[26721]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-22T23:42:38.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:38 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3572222220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T23:42:38.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:38 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby 2024-04-22T23:42:38.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:38 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2024-04-22T23:42:38.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:38 smithi044 ceph-mon[34463]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-22T23:42:38.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:38 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3572222220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T23:42:38.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:38 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby 2024-04-22T23:42:38.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:38 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2024-04-22T23:42:38.729 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: Deploying daemon mds.cephfs.smithi044.onzoro on smithi044 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] up:boot 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 1 up:standby 2024-04-22T23:42:39.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:39 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:42:39.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: Deploying daemon mds.cephfs.smithi044.onzoro on smithi044 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] up:boot 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 1 up:standby 2024-04-22T23:42:39.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:39 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:42:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3376468792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 msgr2=0x7fd0480fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3376468792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480fffc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd038009910 tx=0x7fd038009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3376468792 shutdown_connections 2024-04-22T23:42:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3376468792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd048100da0 0x7fd048101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:40.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3376468792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480fffc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3376468792 >> 172.21.15.31:0/3376468792 conn(0x7fd0480fb120 msgr2=0x7fd0480fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3376468792 shutdown_connections 2024-04-22T23:42:40.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.196+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3376468792 wait complete. 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04cb5e700 1 Processor -- start 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04cb5e700 1 -- start start 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04cb5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04cb5e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd048100da0 0x7fd04819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04cb5e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04819b7f0 con 0x7fd0480ffba0 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04cb5e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04819b930 con 0x7fd048100da0 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56768/0 (socket says 172.21.15.31:56768) 2024-04-22T23:42:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.197+0000 7fd04659c700 1 -- 172.21.15.31:0/3040390284 learned_addr learned my addr 172.21.15.31:0/3040390284 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd045d9b700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd048100da0 0x7fd04819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd04659c700 1 -- 172.21.15.31:0/3040390284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd048100da0 msgr2=0x7fd04819b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd04659c700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd048100da0 0x7fd04819b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd04659c700 1 -- 172.21.15.31:0/3040390284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0380095f0 con 0x7fd0480ffba0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd04659c700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480699f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd03800bda0 tx=0x7fd03800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03801d070 con 0x7fd0480ffba0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd03800f460 con 0x7fd0480ffba0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04819bbb0 con 0x7fd0480ffba0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd038021620 con 0x7fd0480ffba0 2024-04-22T23:42:40.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.198+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04819c0a0 con 0x7fd0480ffba0 2024-04-22T23:42:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.199+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd048108700 con 0x7fd0480ffba0 2024-04-22T23:42:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.200+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd03800f5d0 con 0x7fd0480ffba0 2024-04-22T23:42:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.200+0000 7fd0377fe700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd03006c4e0 0x7fd03006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.200+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fd03808c9a0 con 0x7fd0480ffba0 2024-04-22T23:42:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.200+0000 7fd045d9b700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd03006c4e0 0x7fd03006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.201+0000 7fd045d9b700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd03006c4e0 0x7fd03006e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd03c005950 tx=0x7fd03c00b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:40.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.205+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd03805ac90 con 0x7fd0480ffba0 2024-04-22T23:42:40.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:40.450+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0481088f0 con 0x7fd0480ffba0 2024-04-22T23:42:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.113+0000 7fd0377fe700 1 -- 172.21.15.31:0/3040390284 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7fd03805a820 con 0x7fd0480ffba0 2024-04-22T23:42:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd03006c4e0 msgr2=0x7fd03006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd03006c4e0 0x7fd03006e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd03c005950 tx=0x7fd03c00b410 comp rx=0 tx=0).stop 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 msgr2=0x7fd0480699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480699f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd03800bda0 tx=0x7fd03800bdd0 comp rx=0 tx=0).stop 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 shutdown_connections 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd03006c4e0 0x7fd03006e9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480ffba0 0x7fd0480699f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 --2- 172.21.15.31:0/3040390284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd048100da0 0x7fd04819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 >> 172.21.15.31:0/3040390284 conn(0x7fd0480fb120 msgr2=0x7fd048103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 shutdown_connections 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:41.116+0000 7fd04cb5e700 1 -- 172.21.15.31:0/3040390284 wait complete. 2024-04-22T23:42:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:inline data enabled 2024-04-22T23:42:41.145 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:40 smithi044 ceph-mon[34463]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 202 B/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-22T23:42:41.145 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.145 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:40 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3040390284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T23:42:41.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:40 smithi031 ceph-mon[26721]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 202 B/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-22T23:42:41.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:40 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3040390284' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T23:42:41.703 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:42:41.715 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:42:41.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3040390284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 1 up:standby 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:41.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3040390284' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 1 up:standby 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:42.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:42.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:42.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:42.582 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:42.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:42 smithi031 ceph-mon[26721]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-22T23:42:42.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:42 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:boot 2024-04-22T23:42:42.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:42 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:42:42.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:42 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:42:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:42 smithi044 ceph-mon[34463]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-22T23:42:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:42 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:boot 2024-04-22T23:42:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:42 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:42:43.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:42 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:42:44.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-22T23:42:44.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] up:standby 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:standby-replay 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: Dropping low affinity active daemon mds.cephfs.smithi031.ezuwxk in favor of higher affinity standby. 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: Replacing daemon mds.cephfs.smithi031.ezuwxk as rank 0 with standby daemon mds.cephfs.smithi044.ibkztg 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: osdmap e46: 6 total, 6 up, 6 in 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: fsmap cephfs:1/1 {0=cephfs.smithi044.ibkztg=up:replay} 2 up:standby 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:44.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:44 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:44.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-22T23:42:44.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] up:standby 2024-04-22T23:42:44.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:standby-replay 2024-04-22T23:42:44.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: Dropping low affinity active daemon mds.cephfs.smithi031.ezuwxk in favor of higher affinity standby. 2024-04-22T23:42:44.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: Replacing daemon mds.cephfs.smithi031.ezuwxk as rank 0 with standby daemon mds.cephfs.smithi044.ibkztg 2024-04-22T23:42:44.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:42:44.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:42:44.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: osdmap e46: 6 total, 6 up, 6 in 2024-04-22T23:42:44.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: fsmap cephfs:1/1 {0=cephfs.smithi044.ibkztg=up:replay} 2 up:standby 2024-04-22T23:42:44.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:44.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:44 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:44.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 -- 172.21.15.31:0/2700306307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 msgr2=0x7faf8c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:44.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 --2- 172.21.15.31:0/2700306307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c102310 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faf78009910 tx=0x7faf78009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:44.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 -- 172.21.15.31:0/2700306307 shutdown_connections 2024-04-22T23:42:44.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 --2- 172.21.15.31:0/2700306307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c102310 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:44.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 --2- 172.21.15.31:0/2700306307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8c100cb0 0x7faf8c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 -- 172.21.15.31:0/2700306307 >> 172.21.15.31:0/2700306307 conn(0x7faf8c0fc230 msgr2=0x7faf8c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 -- 172.21.15.31:0/2700306307 shutdown_connections 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.975+0000 7faf92eec700 1 -- 172.21.15.31:0/2700306307 wait complete. 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf92eec700 1 Processor -- start 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf92eec700 1 -- start start 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf92eec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8c100cb0 0x7faf8c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf92eec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf92eec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c197010 con 0x7faf8c101eb0 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf92eec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c197150 con 0x7faf8c100cb0 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57160/0 (socket says 172.21.15.31:57160) 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf8bfff700 1 -- 172.21.15.31:0/1748817838 learned_addr learned my addr 172.21.15.31:0/1748817838 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:44.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.976+0000 7faf90c88700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8c100cb0 0x7faf8c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf8bfff700 1 -- 172.21.15.31:0/1748817838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8c100cb0 msgr2=0x7faf8c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf8bfff700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8c100cb0 0x7faf8c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf8bfff700 1 -- 172.21.15.31:0/1748817838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780095f0 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf8bfff700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c1969f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b5c0 tx=0x7faf7800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf7801d070 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faf7800f460 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c19bba0 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf78021640 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.977+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c19c090 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.978+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c109810 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.979+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7faf7800f5d0 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.979+0000 7faf89ffb700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faf7c06c4e0 0x7faf7c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.979+0000 7faf90c88700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faf7c06c4e0 0x7faf7c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.979+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faf7808cad0 con 0x7faf8c101eb0 2024-04-22T23:42:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.980+0000 7faf90c88700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faf7c06c4e0 0x7faf7c06e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf80005950 tx=0x7faf800058e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:44.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:44.984+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faf7805ad60 con 0x7faf8c101eb0 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.239+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf8c109a00 con 0x7faf8c101eb0 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.239+0000 7faf89ffb700 1 -- 172.21.15.31:0/1748817838 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1830 (secure 0 0 0) 0x7faf78026740 con 0x7faf8c101eb0 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:e16 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:42:45.233 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 16 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:45.125223+0000 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:42:45.234 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:45.236 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:42:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:45.236 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{-1:14490} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:45.236 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:45.236 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:45.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faf7c06c4e0 msgr2=0x7faf7c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faf7c06c4e0 0x7faf7c06e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faf80005950 tx=0x7faf800058e0 comp rx=0 tx=0).stop 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 msgr2=0x7faf8c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c1969f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faf7800b5c0 tx=0x7faf7800be20 comp rx=0 tx=0).stop 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 shutdown_connections 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7faf7c06c4e0 0x7faf7c06e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8c100cb0 0x7faf8c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 --2- 172.21.15.31:0/1748817838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf8c101eb0 0x7faf8c1969f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 >> 172.21.15.31:0/1748817838 conn(0x7faf8c0fc230 msgr2=0x7faf8c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 shutdown_connections 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:45.241+0000 7faf92eec700 1 -- 172.21.15.31:0/1748817838 wait complete. 2024-04-22T23:42:45.237 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 16 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] up:boot 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:reconnect 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: fsmap cephfs:1/1 {0=cephfs.smithi044.ibkztg=up:reconnect} 3 up:standby 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:45.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] up:boot 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:reconnect 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: fsmap cephfs:1/1 {0=cephfs.smithi044.ibkztg=up:reconnect} 3 up:standby 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:45.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:46.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-22T23:42:46.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:46 smithi031 ceph-mon[26721]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-22T23:42:46.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:46 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:rejoin 2024-04-22T23:42:46.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:46 smithi031 ceph-mon[26721]: fsmap cephfs:1/1 {0=cephfs.smithi044.ibkztg=up:rejoin} 3 up:standby 2024-04-22T23:42:46.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:46 smithi031 ceph-mon[26721]: daemon mds.cephfs.smithi044.ibkztg is now active in filesystem cephfs as rank 0 2024-04-22T23:42:46.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:46 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1748817838' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:46.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:46 smithi031 ceph-mon[26721]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:42:46.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:46 smithi044 ceph-mon[34463]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-22T23:42:46.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:46 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:rejoin 2024-04-22T23:42:46.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:46 smithi044 ceph-mon[34463]: fsmap cephfs:1/1 {0=cephfs.smithi044.ibkztg=up:rejoin} 3 up:standby 2024-04-22T23:42:46.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:46 smithi044 ceph-mon[34463]: daemon mds.cephfs.smithi044.ibkztg is now active in filesystem cephfs as rank 0 2024-04-22T23:42:46.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:46 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1748817838' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:46.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:46 smithi044 ceph-mon[34463]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:42:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:47 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:active 2024-04-22T23:42:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:47 smithi044 ceph-mon[34463]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:standby 2024-04-22T23:42:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:47 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:42:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:47 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:42:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:47 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] up:active 2024-04-22T23:42:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:47 smithi031 ceph-mon[26721]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:standby 2024-04-22T23:42:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:47 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:42:47.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:47 smithi031 ceph-mon[26721]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:42:48.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:48 smithi031 ceph-mon[26721]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.9 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-22T23:42:48.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:48 smithi044 ceph-mon[34463]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.9 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-22T23:42:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 -- 172.21.15.31:0/1000530299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 msgr2=0x7fbfbc1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/1000530299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1010d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009910 tx=0x7fbfb8009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 -- 172.21.15.31:0/1000530299 shutdown_connections 2024-04-22T23:42:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/1000530299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 0x7fbfbc102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/1000530299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1010d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:48.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 -- 172.21.15.31:0/1000530299 >> 172.21.15.31:0/1000530299 conn(0x7fbfbc0fc230 msgr2=0x7fbfbc0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:48.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 -- 172.21.15.31:0/1000530299 shutdown_connections 2024-04-22T23:42:48.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.759+0000 7fbfc42f7700 1 -- 172.21.15.31:0/1000530299 wait complete. 2024-04-22T23:42:48.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc42f7700 1 Processor -- start 2024-04-22T23:42:48.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc42f7700 1 -- start start 2024-04-22T23:42:48.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc42f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc42f7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 0x7fbfbc196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc42f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc197020 con 0x7fbfbc100cb0 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc42f7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc197160 con 0x7fbfbc101eb0 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc2093700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc2093700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57178/0 (socket says 172.21.15.31:57178) 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.760+0000 7fbfc2093700 1 -- 172.21.15.31:0/3072141345 learned_addr learned my addr 172.21.15.31:0/3072141345 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc1892700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 0x7fbfbc196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc2093700 1 -- 172.21.15.31:0/3072141345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 msgr2=0x7fbfbc196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc2093700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 0x7fbfbc196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc2093700 1 -- 172.21.15.31:0/3072141345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80095f0 con 0x7fbfbc100cb0 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc1892700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 0x7fbfbc196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc2093700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1964c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800bda0 tx=0x7fbfb800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb801d070 con 0x7fbfbc100cb0 2024-04-22T23:42:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc19bbb0 con 0x7fbfbc100cb0 2024-04-22T23:42:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbfb800f460 con 0x7fbfbc100cb0 2024-04-22T23:42:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb8021640 con 0x7fbfbc100cb0 2024-04-22T23:42:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.761+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc19c0a0 con 0x7fbfbc100cb0 2024-04-22T23:42:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.762+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc109810 con 0x7fbfbc100cb0 2024-04-22T23:42:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.763+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbfb800fa00 con 0x7fbfbc100cb0 2024-04-22T23:42:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.763+0000 7fbfb37fe700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfa806c4e0 0x7fbfa806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.763+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbfb808ca50 con 0x7fbfbc100cb0 2024-04-22T23:42:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.763+0000 7fbfc1892700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfa806c4e0 0x7fbfa806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.764+0000 7fbfc1892700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfa806c4e0 0x7fbfa806e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005fd0 tx=0x7fbfac005ee0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:48.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:48.768+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbfb805ac60 con 0x7fbfbc100cb0 2024-04-22T23:42:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.018+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfbc109a00 con 0x7fbfbc100cb0 2024-04-22T23:42:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.018+0000 7fbfb37fe700 1 -- 172.21.15.31:0/3072141345 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4744 (secure 0 0 0) 0x7fbfb805a7f0 con 0x7fbfbc100cb0 2024-04-22T23:42:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfa806c4e0 msgr2=0x7fbfa806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfa806c4e0 0x7fbfa806e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbfac005fd0 tx=0x7fbfac005ee0 comp rx=0 tx=0).stop 2024-04-22T23:42:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 msgr2=0x7fbfbc1964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1964c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800bda0 tx=0x7fbfb800be80 comp rx=0 tx=0).stop 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 shutdown_connections 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfa806c4e0 0x7fbfa806e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfbc100cb0 0x7fbfbc1964c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 --2- 172.21.15.31:0/3072141345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfbc101eb0 0x7fbfbc196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.020+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 >> 172.21.15.31:0/3072141345 conn(0x7fbfbc0fc230 msgr2=0x7fbfbc1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.021+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 shutdown_connections 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:49.021+0000 7fbfc42f7700 1 -- 172.21.15.31:0/3072141345 wait complete. 2024-04-22T23:42:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:42:49.025 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:42:49.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.5 KiB/s rd, 6 op/s 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3072141345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:42:50.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:49 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.5 KiB/s rd, 6 op/s 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3072141345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:42:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:49 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:50.543 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:50.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:50 smithi031 ceph-mon[26721]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-22T23:42:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:50 smithi044 ceph-mon[34463]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-22T23:42:52.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:51 smithi031 ceph-mon[26721]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.6 KiB/s rd, 10 op/s 2024-04-22T23:42:52.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:52.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:51 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:52.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:51 smithi044 ceph-mon[34463]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.6 KiB/s rd, 10 op/s 2024-04-22T23:42:52.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:51 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:52.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:51 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 -- 172.21.15.31:0/2661208553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc1011a0 msgr2=0x7f09bc1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/2661208553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc1011a0 0x7f09bc1015c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009910 tx=0x7f09b8009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 -- 172.21.15.31:0/2661208553 shutdown_connections 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/2661208553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc0ffe50 0x7f09bc1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/2661208553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc1011a0 0x7f09bc1015c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 -- 172.21.15.31:0/2661208553 >> 172.21.15.31:0/2661208553 conn(0x7f09bc0fa8c0 msgr2=0x7f09bc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 -- 172.21.15.31:0/2661208553 shutdown_connections 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.632+0000 7f09c3b1f700 1 -- 172.21.15.31:0/2661208553 wait complete. 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c3b1f700 1 Processor -- start 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c3b1f700 1 -- start start 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c3b1f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 0x7f09bc196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c3b1f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc1011a0 0x7f09bc196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c3b1f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1970c0 con 0x7f09bc0ffe50 2024-04-22T23:42:52.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c3b1f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc197200 con 0x7f09bc1011a0 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c18bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 0x7f09bc196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c18bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 0x7f09bc196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53366/0 (socket says 172.21.15.31:53366) 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.633+0000 7f09c18bb700 1 -- 172.21.15.31:0/3785676285 learned_addr learned my addr 172.21.15.31:0/3785676285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c10ba700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc1011a0 0x7f09bc196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c18bb700 1 -- 172.21.15.31:0/3785676285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc1011a0 msgr2=0x7f09bc196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c18bb700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc1011a0 0x7f09bc196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c18bb700 1 -- 172.21.15.31:0/3785676285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80095f0 con 0x7f09bc0ffe50 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c10ba700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc1011a0 0x7f09bc196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c18bb700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 0x7f09bc196560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f09b80092e0 tx=0x7f09b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09b801d070 con 0x7f09bc0ffe50 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f09b800f460 con 0x7f09bc0ffe50 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc19bc50 con 0x7f09bc0ffe50 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc19c140 con 0x7f09bc0ffe50 2024-04-22T23:42:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.634+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09b80175f0 con 0x7f09bc0ffe50 2024-04-22T23:42:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.635+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc108720 con 0x7f09bc0ffe50 2024-04-22T23:42:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.636+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f09b800f5d0 con 0x7f09bc0ffe50 2024-04-22T23:42:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.636+0000 7f09aeffd700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f09a806c4e0 0x7f09a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.636+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f09b808cd80 con 0x7f09bc0ffe50 2024-04-22T23:42:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.636+0000 7f09c10ba700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f09a806c4e0 0x7f09a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.637+0000 7f09c10ba700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f09a806c4e0 0x7f09a806e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005950 tx=0x7f09b0005f50 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:52.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.641+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f09b805b010 con 0x7f09bc0ffe50 2024-04-22T23:42:52.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.930+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f09bc108910 con 0x7f09bc0ffe50 2024-04-22T23:42:52.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.931+0000 7f09aeffd700 1 -- 172.21.15.31:0/3785676285 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+83 (secure 0 0 0) 0x7f09b805aba0 con 0x7f09bc0ffe50 2024-04-22T23:42:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f09a806c4e0 msgr2=0x7f09a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f09a806c4e0 0x7f09a806e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005950 tx=0x7f09b0005f50 comp rx=0 tx=0).stop 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 msgr2=0x7f09bc196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 0x7f09bc196560 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f09b80092e0 tx=0x7f09b800be20 comp rx=0 tx=0).stop 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 shutdown_connections 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f09a806c4e0 0x7f09a806e9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09bc0ffe50 0x7f09bc196560 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 --2- 172.21.15.31:0/3785676285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09bc1011a0 0x7f09bc196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 >> 172.21.15.31:0/3785676285 conn(0x7f09bc0fa8c0 msgr2=0x7f09bc0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 shutdown_connections 2024-04-22T23:42:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:52.933+0000 7f09c3b1f700 1 -- 172.21.15.31:0/3785676285 wait complete. 2024-04-22T23:42:52.937 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:42:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:52 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3785676285' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-22T23:42:53.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:52 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3785676285' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-22T23:42:54.095 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-22T23:42:54.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:42:54.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 -- 172.21.15.31:0/3387573962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 msgr2=0x7f1e640ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 --2- 172.21.15.31:0/3387573962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e640ff3d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1e54009880 tx=0x7f1e5402bce0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 -- 172.21.15.31:0/3387573962 shutdown_connections 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 --2- 172.21.15.31:0/3387573962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 0x7f1e64100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 --2- 172.21.15.31:0/3387573962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e640ff3d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 -- 172.21.15.31:0/3387573962 >> 172.21.15.31:0/3387573962 conn(0x7f1e640fa540 msgr2=0x7f1e640fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 -- 172.21.15.31:0/3387573962 shutdown_connections 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.361+0000 7f1e69088640 1 -- 172.21.15.31:0/3387573962 wait complete. 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e69088640 1 Processor -- start 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e69088640 1 -- start start 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e69088640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e64068f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e69088640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 0x7f1e6419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e69088640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419acf0 con 0x7f1e640fefb0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e69088640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6419ae30 con 0x7f1e641001a0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e64068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e64068f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53388/0 (socket says 172.21.15.31:53388) 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 -- 172.21.15.31:0/585464509 learned_addr learned my addr 172.21.15.31:0/585464509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e5bfff640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 0x7f1e6419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 -- 172.21.15.31:0/585464509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 msgr2=0x7f1e6419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 0x7f1e6419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 -- 172.21.15.31:0/585464509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54009560 con 0x7f1e640fefb0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e5bfff640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 0x7f1e6419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.362+0000 7f1e62d9d640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e64068f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e54000c00 tx=0x7f1e54004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.363+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e54004520 con 0x7f1e640fefb0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.363+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6419b0b0 con 0x7f1e640fefb0 2024-04-22T23:42:54.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.363+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1e540054b0 con 0x7f1e640fefb0 2024-04-22T23:42:54.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.363+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e54034670 con 0x7f1e640fefb0 2024-04-22T23:42:54.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.363+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6419b630 con 0x7f1e640fefb0 2024-04-22T23:42:54.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.364+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e64107d60 con 0x7f1e640fefb0 2024-04-22T23:42:54.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.364+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1e54005000 con 0x7f1e640fefb0 2024-04-22T23:42:54.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.365+0000 7f1e60d99640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1e5006c450 0x7f1e5006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.365+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1e540adea0 con 0x7f1e640fefb0 2024-04-22T23:42:54.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.365+0000 7f1e5bfff640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1e5006c450 0x7f1e5006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.365+0000 7f1e5bfff640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1e5006c450 0x7f1e5006e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006fd0 tx=0x7f1e4c008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:54.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.370+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1e54078620 con 0x7f1e640fefb0 2024-04-22T23:42:54.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:53 smithi031 ceph-mon[26721]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:42:54.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:53 smithi044 ceph-mon[34463]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:42:54.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.554+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e6419bd10 con 0x7f1e640fefb0 2024-04-22T23:42:54.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.555+0000 7f1e60d99640 1 -- 172.21.15.31:0/585464509 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15103 (secure 0 0 0) 0x7f1e5407bc40 con 0x7f1e640fefb0 2024-04-22T23:42:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:54.548 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:42:43.116400+0000","last_up_change":"2024-04-22T23:41:14.440871+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:42:25.830464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:42:26.380171+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6803","nonce":1941835757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6805","nonce":1941835757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6809","nonce":1941835757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1941835757},{"type":"v1","addr":"172.21.15.31:6807","nonce":1941835757}]},"public_addr":"172.21.15.31:6803/1941835757","cluster_addr":"172.21.15.31:6805/1941835757","heartbeat_back_addr":"172.21.15.31:6809/1941835757","heartbeat_front_addr":"172.21.15.31:6807/1941835757","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6811","nonce":3091658850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6813","nonce":3091658850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6817","nonce":3091658850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3091658850},{"type":"v1","addr":"172.21.15.31:6815","nonce":3091658850}]},"public_addr":"172.21.15.31:6811/3091658850","cluster_addr":"172.21.15.31:6813/3091658850","heartbeat_back_addr":"172.21.15.31:6817/3091658850","heartbeat_front_addr":"172.21.15.31:6815/3091658850","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6819","nonce":3807947725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6821","nonce":3807947725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6825","nonce":3807947725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3807947725},{"type":"v1","addr":"172.21.15.31:6823","nonce":3807947725}]},"public_addr":"172.21.15.31:6819/3807947725","cluster_addr":"172.21.15.31:6821/3807947725","heartbeat_back_addr":"172.21.15.31:6825/3807947725","heartbeat_front_addr":"172.21.15.31:6823/3807947725","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6801","nonce":3279105995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6803","nonce":3279105995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6807","nonce":3279105995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3279105995},{"type":"v1","addr":"172.21.15.44:6805","nonce":3279105995}]},"public_addr":"172.21.15.44:6801/3279105995","cluster_addr":"172.21.15.44:6803/3279105995","heartbeat_back_addr":"172.21.15.44:6807/3279105995","heartbeat_front_addr":"172.21.15.44:6805/3279105995","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6809","nonce":3049083294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6811","nonce":3049083294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6815","nonce":3049083294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3049083294},{"type":"v1","addr":"172.21.15.44:6813","nonce":3049083294}]},"public_addr":"172.21.15.44:6809/3049083294","cluster_addr":"172.21.15.44:6811/3049083294","heartbeat_back_addr":"172.21.15.44:6815/3049083294","heartbeat_front_addr":"172.21.15.44:6813/3049083294","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6817","nonce":924200036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6819","nonce":924200036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6823","nonce":924200036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":924200036},{"type":"v1","addr":"172.21.15.44:6821","nonce":924200036}]},"public_addr":"172.21.15.44:6817/924200036","cluster_addr":"172.21.15.44:6819/924200036","heartbeat_back_addr":"172.21.15.44:6823/924200036","heartbeat_front_addr":"172.21.15.44:6821/924200036","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6827/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6826/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:42:54.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1e5006c450 msgr2=0x7f1e5006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1e5006c450 0x7f1e5006e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c006fd0 tx=0x7f1e4c008040 comp rx=0 tx=0).stop 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 msgr2=0x7f1e64068f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e64068f50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e54000c00 tx=0x7f1e54004f00 comp rx=0 tx=0).stop 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 shutdown_connections 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1e5006c450 0x7f1e5006e910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e641001a0 0x7f1e6419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.557+0000 7f1e69088640 1 --2- 172.21.15.31:0/585464509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e640fefb0 0x7f1e64068f50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.558+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 >> 172.21.15.31:0/585464509 conn(0x7f1e640fa540 msgr2=0x7f1e641033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.558+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 shutdown_connections 2024-04-22T23:42:54.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.558+0000 7f1e69088640 1 -- 172.21.15.31:0/585464509 wait complete. 2024-04-22T23:42:54.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:39:59.535490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:42:25.830464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:42:26.380171+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:42:54.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:42:54.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 -- 172.21.15.31:0/2608353180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 msgr2=0x7fd4041009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 --2- 172.21.15.31:0/2608353180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd4041009e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd400009880 tx=0x7fd40002bce0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 -- 172.21.15.31:0/2608353180 shutdown_connections 2024-04-22T23:42:54.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 --2- 172.21.15.31:0/2608353180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 0x7fd4040ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 --2- 172.21.15.31:0/2608353180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd4041009e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 -- 172.21.15.31:0/2608353180 >> 172.21.15.31:0/2608353180 conn(0x7fd4040f9cf0 msgr2=0x7fd4040fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 -- 172.21.15.31:0/2608353180 shutdown_connections 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.680+0000 7fd40b658640 1 -- 172.21.15.31:0/2608353180 wait complete. 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 Processor -- start 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 -- start start 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 0x7fd4041959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd404195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041964e0 con 0x7fd4041005c0 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd404196650 con 0x7fd4040ff1c0 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd404195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd404195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53408/0 (socket says 172.21.15.31:53408) 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 -- 172.21.15.31:0/1370652201 learned_addr learned my addr 172.21.15.31:0/1370652201 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:54.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd4093f4640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 0x7fd4041959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 -- 172.21.15.31:0/1370652201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 msgr2=0x7fd4041959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 0x7fd4041959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 -- 172.21.15.31:0/1370652201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400009560 con 0x7fd4041005c0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd4093f4640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 0x7fd4041959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd408bf3640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd404195f10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400d8d0 tx=0x7fd3f400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f4009c00 con 0x7fd4041005c0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40419b0e0 con 0x7fd4041005c0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd3f40044d0 con 0x7fd4041005c0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.681+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f40049d0 con 0x7fd4041005c0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.682+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40419b640 con 0x7fd4041005c0 2024-04-22T23:42:54.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.682+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd404107d10 con 0x7fd4041005c0 2024-04-22T23:42:54.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.683+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fd3f4004020 con 0x7fd4041005c0 2024-04-22T23:42:54.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.683+0000 7fd3fa7fc640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3f006c450 0x7fd3f006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.683+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd3f408b5d0 con 0x7fd4041005c0 2024-04-22T23:42:54.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.683+0000 7fd4093f4640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3f006c450 0x7fd3f006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.684+0000 7fd4093f4640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3f006c450 0x7fd3f006e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd400036040 tx=0x7fd400005fd0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:54.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.688+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd3f4055d50 con 0x7fd4041005c0 2024-04-22T23:42:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.868+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd404107ea0 con 0x7fd4041005c0 2024-04-22T23:42:54.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.869+0000 7fd3fa7fc640 1 -- 172.21.15.31:0/1370652201 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd404107ea0 con 0x7fd4041005c0 2024-04-22T23:42:54.862 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-22T23:42:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3f006c450 msgr2=0x7fd3f006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3f006c450 0x7fd3f006e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd400036040 tx=0x7fd400005fd0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 msgr2=0x7fd404195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd404195f10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400d8d0 tx=0x7fd3f400dd90 comp rx=0 tx=0).stop 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 shutdown_connections 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fd3f006c450 0x7fd3f006e910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4041005c0 0x7fd404195f10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 --2- 172.21.15.31:0/1370652201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4040ff1c0 0x7fd4041959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 >> 172.21.15.31:0/1370652201 conn(0x7fd4040f9cf0 msgr2=0x7fd4040fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 shutdown_connections 2024-04-22T23:42:54.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.871+0000 7fd40b658640 1 -- 172.21.15.31:0/1370652201 wait complete. 2024-04-22T23:42:54.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:42:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 -- 172.21.15.31:0/4125342690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 msgr2=0x7f12841058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 --2- 172.21.15.31:0/4125342690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f12841058a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1278009880 tx=0x7f127802bce0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 -- 172.21.15.31:0/4125342690 shutdown_connections 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 --2- 172.21.15.31:0/4125342690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f12841058a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 --2- 172.21.15.31:0/4125342690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1284100b80 0x7f1284102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 -- 172.21.15.31:0/4125342690 >> 172.21.15.31:0/4125342690 conn(0x7f12840fa4f0 msgr2=0x7f12840fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 -- 172.21.15.31:0/4125342690 shutdown_connections 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.991+0000 7f128b58d640 1 -- 172.21.15.31:0/4125342690 wait complete. 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f128b58d640 1 Processor -- start 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f128b58d640 1 -- start start 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f128b58d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1284100b80 0x7f12841958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f128b58d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f1284195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f128b58d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1284196400 con 0x7f12841034b0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f128b58d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1284196570 con 0x7f1284100b80 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1288b28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f1284195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1288b28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f1284195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53422/0 (socket says 172.21.15.31:53422) 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1288b28640 1 -- 172.21.15.31:0/2904879503 learned_addr learned my addr 172.21.15.31:0/2904879503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1289329640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1284100b80 0x7f12841958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1288b28640 1 -- 172.21.15.31:0/2904879503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1284100b80 msgr2=0x7f12841958f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1288b28640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1284100b80 0x7f12841958f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.992+0000 7f1288b28640 1 -- 172.21.15.31:0/2904879503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009560 con 0x7f12841034b0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.993+0000 7f1288b28640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f1284195e30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f127802c220 tx=0x7f1278004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.993+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278004520 con 0x7f12841034b0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.993+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f12780054b0 con 0x7f12841034b0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.993+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128419afa0 con 0x7f12841034b0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.993+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278003ea0 con 0x7f12841034b0 2024-04-22T23:42:54.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.993+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128419b470 con 0x7f12841034b0 2024-04-22T23:42:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.994+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128418f990 con 0x7f12841034b0 2024-04-22T23:42:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.994+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f12780035f0 con 0x7f12841034b0 2024-04-22T23:42:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.995+0000 7f12827fc640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f126c06c450 0x7f126c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.995+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f12780adcd0 con 0x7f12841034b0 2024-04-22T23:42:54.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.995+0000 7f1289329640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f126c06c450 0x7f126c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:54.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.995+0000 7f1289329640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f126c06c450 0x7f126c06e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f127c005fd0 tx=0x7f127c009760 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:54.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:54.999+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1278078500 con 0x7f12841034b0 2024-04-22T23:42:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.180+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1284108c10 con 0x7f12841034b0 2024-04-22T23:42:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.180+0000 7f12827fc640 1 -- 172.21.15.31:0/2904879503 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f127807bb20 con 0x7f12841034b0 2024-04-22T23:42:55.173 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:42:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.182+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f126c06c450 msgr2=0x7f126c06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.182+0000 7f128b58d640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f126c06c450 0x7f126c06e910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f127c005fd0 tx=0x7f127c009760 comp rx=0 tx=0).stop 2024-04-22T23:42:55.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.182+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 msgr2=0x7f1284195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.182+0000 7f128b58d640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f1284195e30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f127802c220 tx=0x7f1278004f00 comp rx=0 tx=0).stop 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 shutdown_connections 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f126c06c450 0x7f126c06e910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12841034b0 0x7f1284195e30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 --2- 172.21.15.31:0/2904879503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1284100b80 0x7f12841958f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 >> 172.21.15.31:0/2904879503 conn(0x7f12840fa4f0 msgr2=0x7f12840fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 shutdown_connections 2024-04-22T23:42:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.183+0000 7f128b58d640 1 -- 172.21.15.31:0/2904879503 wait complete. 2024-04-22T23:42:55.185 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:54 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/585464509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:42:55.185 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:54 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1370652201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:42:55.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:42:55.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 -- 172.21.15.31:0/789686306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc01016c0 msgr2=0x7fbfc0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/789686306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc01016c0 0x7fbfc0101ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009880 tx=0x7fbfb402bce0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 -- 172.21.15.31:0/789686306 shutdown_connections 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/789686306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc00fe130 0x7fbfc00fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/789686306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc01016c0 0x7fbfc0101ae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 -- 172.21.15.31:0/789686306 >> 172.21.15.31:0/789686306 conn(0x7fbfc00f9cd0 msgr2=0x7fbfc00fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 -- 172.21.15.31:0/789686306 shutdown_connections 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 -- 172.21.15.31:0/789686306 wait complete. 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 Processor -- start 2024-04-22T23:42:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.310+0000 7fbfc6b86640 1 -- start start 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc6b86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 0x7fbfc0195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc6b86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01016c0 0x7fbfc0195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc6b86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01964a0 con 0x7fbfc00fe130 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc6b86640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0196610 con 0x7fbfc01016c0 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 0x7fbfc0195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 0x7fbfc0195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53450/0 (socket says 172.21.15.31:53450) 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 -- 172.21.15.31:0/2409653573 learned_addr learned my addr 172.21.15.31:0/2409653573 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfbffff640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01016c0 0x7fbfc0195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 -- 172.21.15.31:0/2409653573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01016c0 msgr2=0x7fbfc0195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01016c0 0x7fbfc0195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 -- 172.21.15.31:0/2409653573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4009560 con 0x7fbfc00fe130 2024-04-22T23:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc4922640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 0x7fbfc0195990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009230 tx=0x7fbfb4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb40044e0 con 0x7fbfc00fe130 2024-04-22T23:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc019b040 con 0x7fbfc00fe130 2024-04-22T23:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbfb40054b0 con 0x7fbfc00fe130 2024-04-22T23:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.311+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfb4034440 con 0x7fbfc00fe130 2024-04-22T23:42:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.312+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc019b540 con 0x7fbfc00fe130 2024-04-22T23:42:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.313+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc0108e60 con 0x7fbfc00fe130 2024-04-22T23:42:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.313+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbfb4003810 con 0x7fbfc00fe130 2024-04-22T23:42:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.313+0000 7fbfbdffb640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfb006c450 0x7fbfb006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.314+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbfb40adcd0 con 0x7fbfc00fe130 2024-04-22T23:42:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.314+0000 7fbfbffff640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfb006c450 0x7fbfb006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.314+0000 7fbfbffff640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfb006c450 0x7fbfb006e910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0196ea0 tx=0x7fbfac005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.321+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbfb4078500 con 0x7fbfc00fe130 2024-04-22T23:42:55.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:54 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/585464509' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:42:55.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:54 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1370652201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:42:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.524+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fbfc019b760 con 0x7fbfc00fe130 2024-04-22T23:42:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.524+0000 7fbfbdffb640 1 -- 172.21.15.31:0/2409653573 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbfb407bb20 con 0x7fbfc00fe130 2024-04-22T23:42:55.517 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfb006c450 msgr2=0x7fbfb006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfb006c450 0x7fbfb006e910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0196ea0 tx=0x7fbfac005f90 comp rx=0 tx=0).stop 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 msgr2=0x7fbfc0195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 0x7fbfc0195990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009230 tx=0x7fbfb4004dc0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 shutdown_connections 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fbfb006c450 0x7fbfb006e910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01016c0 0x7fbfc0195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 --2- 172.21.15.31:0/2409653573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfc00fe130 0x7fbfc0195990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 >> 172.21.15.31:0/2409653573 conn(0x7fbfc00f9cd0 msgr2=0x7fbfc0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 shutdown_connections 2024-04-22T23:42:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.527+0000 7fbfc6b86640 1 -- 172.21.15.31:0/2409653573 wait complete. 2024-04-22T23:42:55.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T23:42:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 -- 172.21.15.31:0/594423758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fdce0 msgr2=0x7f205c0fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 --2- 172.21.15.31:0/594423758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fdce0 0x7f205c0fe100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2044009880 tx=0x7f204402bce0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 -- 172.21.15.31:0/594423758 shutdown_connections 2024-04-22T23:42:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 --2- 172.21.15.31:0/594423758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fe6d0 0x7f205c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 --2- 172.21.15.31:0/594423758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fdce0 0x7f205c0fe100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 -- 172.21.15.31:0/594423758 >> 172.21.15.31:0/594423758 conn(0x7f205c0f9810 msgr2=0x7f205c0fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:55.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 -- 172.21.15.31:0/594423758 shutdown_connections 2024-04-22T23:42:55.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.640+0000 7f2060efe640 1 -- 172.21.15.31:0/594423758 wait complete. 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2060efe640 1 Processor -- start 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2060efe640 1 -- start start 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2060efe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fdce0 0x7f205c195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2060efe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 0x7f205c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2060efe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c196450 con 0x7f205c0fe6d0 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2060efe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1965c0 con 0x7f205c0fdce0 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2059d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 0x7f205c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2059d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 0x7f205c195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53478/0 (socket says 172.21.15.31:53478) 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.641+0000 7f2059d9b640 1 -- 172.21.15.31:0/1348446302 learned_addr learned my addr 172.21.15.31:0/1348446302 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:55.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f205a59c640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fdce0 0x7f205c195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f2059d9b640 1 -- 172.21.15.31:0/1348446302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fdce0 msgr2=0x7f205c195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f2059d9b640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fdce0 0x7f205c195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f2059d9b640 1 -- 172.21.15.31:0/1348446302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2044009560 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f205a59c640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fdce0 0x7f205c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f2059d9b640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 0x7f205c195e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d6f0 tx=0x7f204c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f204c009a30 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f204c0054d0 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c19b050 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f204c010460 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.642+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c19b580 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.643+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c18f9e0 con 0x7f205c0fe6d0 2024-04-22T23:42:55.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.644+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f204c009b90 con 0x7f205c0fe6d0 2024-04-22T23:42:55.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.644+0000 7f20537fe640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f204806c450 0x7f204806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:55.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.644+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f204c08b420 con 0x7f205c0fe6d0 2024-04-22T23:42:55.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.644+0000 7f205a59c640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f204806c450 0x7f204806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:55.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.644+0000 7f205a59c640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f204806c450 0x7f204806e910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2044036040 tx=0x7f2044005fd0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:55.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.648+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f204c055ba0 con 0x7f205c0fe6d0 2024-04-22T23:42:55.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.863+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f205c108c60 con 0x7f205c0fe6d0 2024-04-22T23:42:55.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.864+0000 7f20537fe640 1 -- 172.21.15.31:0/1348446302 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4744 (secure 0 0 0) 0x7f204c0591c0 con 0x7f205c0fe6d0 2024-04-22T23:42:55.857 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:42:55.857 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14500,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2302742063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2302742063},{"type":"v1","addr":"172.21.15.31:6827","nonce":2302742063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:42:46.131915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14490":{"gid":14490,"name":"cephfs.smithi031.usiyoa","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.31:6829/748820754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":748820754},{"type":"v1","addr":"172.21.15.31:6829","nonce":748820754}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-22T23:42:55.858 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f204806c450 msgr2=0x7f204806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f204806c450 0x7f204806e910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2044036040 tx=0x7f2044005fd0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 msgr2=0x7f205c195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 0x7f205c195e80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d6f0 tx=0x7f204c00dbb0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 shutdown_connections 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f204806c450 0x7f204806e910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c0fe6d0 0x7f205c195e80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 --2- 172.21.15.31:0/1348446302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f205c0fdce0 0x7f205c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 >> 172.21.15.31:0/1348446302 conn(0x7f205c0f9810 msgr2=0x7f205c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 shutdown_connections 2024-04-22T23:42:55.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:55.866+0000 7f2060efe640 1 -- 172.21.15.31:0/1348446302 wait complete. 2024-04-22T23:42:55.870 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-04-22T23:42:55.870 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-22T23:42:55.895 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-22T23:42:55.895 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-22T23:42:55.896 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-22T23:42:55.896 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T23:42:55.896 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T23:42:55.896 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-22T23:42:55.896 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T23:42:55.896 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-22T23:42:55.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:55.896 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-22T23:42:55.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:55.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-22T23:42:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-04-22T23:42:55.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:42:55.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:55.980 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T23:42:56.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:56.000 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-22T23:42:56.075 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-04-22T23:42:56.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:42:56.077 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-22T23:42:56.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2904879503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2409653573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:42:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1348446302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:42:56.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:55 smithi031 ceph-mon[26721]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-22T23:42:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2904879503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:42:56.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2409653573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:42:56.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1348446302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:42:56.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:55 smithi044 ceph-mon[34463]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-22T23:42:57.319 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:56 smithi031 ceph-mon[26721]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:42:57.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:56 smithi044 ceph-mon[34463]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:42:57.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 -- 172.21.15.31:0/3338149826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54101de0 msgr2=0x7f9d541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:57.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/3338149826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54101de0 0x7f9d541021c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009910 tx=0x7f9d44009c20 comp rx=0 tx=0).stop 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 -- 172.21.15.31:0/3338149826 shutdown_connections 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/3338149826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d54102700 0x7f9d5410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/3338149826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54101de0 0x7f9d541021c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 -- 172.21.15.31:0/3338149826 >> 172.21.15.31:0/3338149826 conn(0x7f9d540fb610 msgr2=0x7f9d540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 -- 172.21.15.31:0/3338149826 shutdown_connections 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.633+0000 7f9d59b0c700 1 -- 172.21.15.31:0/3338149826 wait complete. 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d59b0c700 1 Processor -- start 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d59b0c700 1 -- start start 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d59b0c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d54101de0 0x7f9d540ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d59b0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 0x7f9d540fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d59b0c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541004c0 con 0x7f9d54102700 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d59b0c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d54100600 con 0x7f9d54101de0 2024-04-22T23:42:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d52ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 0x7f9d540fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d52ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 0x7f9d540fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53492/0 (socket says 172.21.15.31:53492) 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d52ffd700 1 -- 172.21.15.31:0/1578109336 learned_addr learned my addr 172.21.15.31:0/1578109336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.634+0000 7f9d537fe700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d54101de0 0x7f9d540ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d52ffd700 1 -- 172.21.15.31:0/1578109336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d54101de0 msgr2=0x7f9d540ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d52ffd700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d54101de0 0x7f9d540ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d52ffd700 1 -- 172.21.15.31:0/1578109336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d440095f0 con 0x7f9d54102700 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d52ffd700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 0x7f9d540fff80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800b3d0 tx=0x7f9d4800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d48011800 con 0x7f9d54102700 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d541a2af0 con 0x7f9d54102700 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9d48011e10 con 0x7f9d54102700 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d48010ea0 con 0x7f9d54102700 2024-04-22T23:42:57.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.635+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d541a2f00 con 0x7f9d54102700 2024-04-22T23:42:57.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.636+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d541082f0 con 0x7f9d54102700 2024-04-22T23:42:57.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.636+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9d48010380 con 0x7f9d54102700 2024-04-22T23:42:57.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.637+0000 7f9d50ff9700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f9d3c06c4e0 0x7f9d3c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:57.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.637+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9d4808b400 con 0x7f9d54102700 2024-04-22T23:42:57.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.637+0000 7f9d537fe700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f9d3c06c4e0 0x7f9d3c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:57.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.637+0000 7f9d537fe700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f9d3c06c4e0 0x7f9d3c06e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b5c0 tx=0x7f9d4401a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:57.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.641+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9d48055b80 con 0x7f9d54102700 2024-04-22T23:42:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.883+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d541007c0 con 0x7f9d54102700 2024-04-22T23:42:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.883+0000 7f9d50ff9700 1 -- 172.21.15.31:0/1578109336 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f9d480591a0 con 0x7f9d54102700 2024-04-22T23:42:57.876 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f9d3c06c4e0 msgr2=0x7f9d3c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f9d3c06c4e0 0x7f9d3c06e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400b5c0 tx=0x7f9d4401a040 comp rx=0 tx=0).stop 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 msgr2=0x7f9d540fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 0x7f9d540fff80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800b3d0 tx=0x7f9d4800b790 comp rx=0 tx=0).stop 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 shutdown_connections 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f9d3c06c4e0 0x7f9d3c06e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d54101de0 0x7f9d540ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 --2- 172.21.15.31:0/1578109336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d54102700 0x7f9d540fff80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.885+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 >> 172.21.15.31:0/1578109336 conn(0x7f9d540fb610 msgr2=0x7f9d54105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.886+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 shutdown_connections 2024-04-22T23:42:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:57.886+0000 7f9d59b0c700 1 -- 172.21.15.31:0/1578109336 wait complete. 2024-04-22T23:42:58.174 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:57 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1578109336' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:42:58.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:57 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1578109336' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T23:42:58.473 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T23:42:58.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:58.474 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-04-22T23:42:58.490 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T23:42:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T23:42:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T23:42:58.490 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42291sec preferred_lft 42291sec 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-22T23:42:58.491 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:42:58.492 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:42:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-04-22T23:42:58.492 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-04-22T23:42:58.492 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T23:42:58.492 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:42:58.492 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-22T23:42:58.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-04-22T23:42:58.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-04-22T23:42:58.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-04-22T23:42:58.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T23:42:58.493 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-22T23:42:58.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:42:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:42:58.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:58.679 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T23:42:58.703 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-04-22T23:42:58.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:58.705 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-04-22T23:42:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-04-22T23:42:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T23:42:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T23:42:58.775 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T23:42:58.776 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T23:42:58.776 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:42:58.776 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-22T23:42:58.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T23:42:58.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T23:42:58.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T23:42:58.776 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-22T23:42:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:42:59.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:42:59.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:59.068 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-22T23:42:59.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:59.083 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-04-22T23:42:59.139 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:42:59.140 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-22T23:42:59.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:42:59.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T23:42:59.140 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-22T23:42:59.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:42:59.235 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:42:58 smithi031 ceph-mon[26721]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-22T23:42:59.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:42:59.257 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T23:42:59.257 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-22T23:42:59.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:42:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:42:59.388 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:42:59.388 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-22T23:42:59.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-04-22T23:42:59.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T23:42:59.388 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-22T23:42:59.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:42:59.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:42:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:42:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:42:58 smithi044 ceph-mon[34463]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-22T23:42:59.464 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-22T23:42:59.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:42:59.465 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T23:42:59.479 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T23:42:59.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:42:59.480 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T23:42:59.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-22T23:42:59.599 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T23:42:59.656 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-22T23:42:59.657 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/a09d206a0757552f6b76c7530c19d35b683e7cd9b411e794e57e32c11175645f/merged 2024-04-22T23:42:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/87b71f0b46f1454be1f3b8353dc908667bead06f14cbf11a1bc49c74ba96b9cf/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/8f3362ae6d473fb8b1950fa439e5150c9e2cf06720e1a3fd84a942407b35ebde/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/386de7e20d2fdcf4cd4361890ff37d7e151bbd788cbb4929b492e47771167418/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c5375e8d1ae96482828833f141b056f0967fa6b7d7223d08820b1f46c8904933/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/bef4312d271d22028b5e92069444d9b85392c83ed6e2c70c8fe08a60afaa90b6/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/460a9a16f313ec86ffd851cd23bf246696a26af670c5d768a5739de9ecc0d6ba/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/ea3c67e738444878148e4f73331f51de4dc87641cb55af5323e1b04685734bd9/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/d6044e3ef7432df673cb8f733b77e02511226b4ea669dfeac71de311d8085917/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/8f3aa507f66b8b059d19c45d65aaa954545d1f70e97b4fb9120ff945d1a5f822/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/732a3b6880229a935ab107b5fec5c897310926b98be1a8edaa81a30cb7581a3b/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/a5c70b9b8bf1115608bd923f110ad3c1c1e654be69a610bdc3b99eae7c45de8a/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/0cb193828683425dd7cff9f175b79f8a47f06d2c85d77879e77abb30666f0a4d/merged 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-22T23:42:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:42:59.660 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:42:59.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:59.660 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-22T23:42:59.712 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-22T23:42:59.713 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-22T23:42:59.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-22T23:42:59.780 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T23:42:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-22T23:42:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-22T23:42:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-22T23:42:59.838 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/a09d206a0757552f6b76c7530c19d35b683e7cd9b411e794e57e32c11175645f/merged 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/87b71f0b46f1454be1f3b8353dc908667bead06f14cbf11a1bc49c74ba96b9cf/merged 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/8f3362ae6d473fb8b1950fa439e5150c9e2cf06720e1a3fd84a942407b35ebde/merged 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/386de7e20d2fdcf4cd4361890ff37d7e151bbd788cbb4929b492e47771167418/merged 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c5375e8d1ae96482828833f141b056f0967fa6b7d7223d08820b1f46c8904933/merged 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/bef4312d271d22028b5e92069444d9b85392c83ed6e2c70c8fe08a60afaa90b6/merged 2024-04-22T23:42:59.839 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/460a9a16f313ec86ffd851cd23bf246696a26af670c5d768a5739de9ecc0d6ba/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/ea3c67e738444878148e4f73331f51de4dc87641cb55af5323e1b04685734bd9/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/d6044e3ef7432df673cb8f733b77e02511226b4ea669dfeac71de311d8085917/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/8f3aa507f66b8b059d19c45d65aaa954545d1f70e97b4fb9120ff945d1a5f822/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/732a3b6880229a935ab107b5fec5c897310926b98be1a8edaa81a30cb7581a3b/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/a5c70b9b8bf1115608bd923f110ad3c1c1e654be69a610bdc3b99eae7c45de8a/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/0cb193828683425dd7cff9f175b79f8a47f06d2c85d77879e77abb30666f0a4d/merged 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:42:59.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:42:59.841 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-22T23:42:59.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-04-22T23:42:59.875+0000 7fd53aa3f480 -1 init, newargv = 0x557b6cbaa410 newargc=15 2024-04-22T23:42:59.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[75066]: starting ceph client 2024-04-22T23:42:59.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[75066]: starting fuse 2024-04-22T23:42:59.883 INFO:teuthology.orchestra.run.smithi031.stdout:156 2024-04-22T23:42:59.884 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [156] 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-22T23:42:59.884 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-22T23:42:59.885 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-04-22T23:42:59.886 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-04-22T23:43:00.005 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.75066.asok 2024-04-22T23:43:00.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:00.016 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75066.asok 2024-04-22T23:43:00.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:00.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75066.asok status 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "75066", 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-04-22T23:43:00.154 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 14542, 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 14542 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1415787653 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-22T23:43:00.155 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1415787653 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.14542 192.168.0.1:0/1415787653", 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/1415787653", 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 18, 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 46, 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-04-22T23:43:00.156 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:43:00.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T23:43:00.380 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:43:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:00 smithi031 ceph-mon[26721]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T23:43:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:00 smithi044 ceph-mon[34463]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T23:43:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:01.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 -- 172.21.15.31:0/3593429561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c101840 msgr2=0x7f8f1c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 --2- 172.21.15.31:0/3593429561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c101840 0x7f8f1c105890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c009910 tx=0x7f8f0c009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 -- 172.21.15.31:0/3593429561 shutdown_connections 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 --2- 172.21.15.31:0/3593429561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c101840 0x7f8f1c105890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 --2- 172.21.15.31:0/3593429561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1c100e90 0x7f8f1c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 -- 172.21.15.31:0/3593429561 >> 172.21.15.31:0/3593429561 conn(0x7f8f1c0fc740 msgr2=0x7f8f1c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 -- 172.21.15.31:0/3593429561 shutdown_connections 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.864+0000 7f8f2384f700 1 -- 172.21.15.31:0/3593429561 wait complete. 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f2384f700 1 Processor -- start 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f2384f700 1 -- start start 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f2384f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 0x7f8f1c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f2384f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1c101840 0x7f8f1c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:01.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f2384f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c073b10 con 0x7f8f1c100e90 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f2384f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c073c50 con 0x7f8f1c101840 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f215eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 0x7f8f1c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f215eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 0x7f8f1c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58116/0 (socket says 172.21.15.31:58116) 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f215eb700 1 -- 172.21.15.31:0/4059779126 learned_addr learned my addr 172.21.15.31:0/4059779126 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f20dea700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1c101840 0x7f8f1c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f215eb700 1 -- 172.21.15.31:0/4059779126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1c101840 msgr2=0x7f8f1c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f215eb700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1c101840 0x7f8f1c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.865+0000 7f8f215eb700 1 -- 172.21.15.31:0/4059779126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0095f0 con 0x7f8f1c100e90 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.866+0000 7f8f215eb700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 0x7f8f1c073090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800d8d0 tx=0x7f8f1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.866+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f180098a0 con 0x7f8f1c100e90 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.866+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f18010460 con 0x7f8f1c100e90 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.866+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c1a2b10 con 0x7f8f1c100e90 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.866+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1800f5f0 con 0x7f8f1c100e90 2024-04-22T23:43:01.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.866+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c1a2f20 con 0x7f8f1c100e90 2024-04-22T23:43:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.867+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c109290 con 0x7f8f1c100e90 2024-04-22T23:43:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.867+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8f18009a00 con 0x7f8f1c100e90 2024-04-22T23:43:01.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.868+0000 7f8f127fc700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f8f0806c4e0 0x7f8f0806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.868+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8f1808b770 con 0x7f8f1c100e90 2024-04-22T23:43:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.868+0000 7f8f20dea700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f8f0806c4e0 0x7f8f0806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:01.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.868+0000 7f8f20dea700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f8f0806c4e0 0x7f8f0806e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c074880 tx=0x7f8f0c01a040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:01.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:01.872+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8f18055ef0 con 0x7f8f1c100e90 2024-04-22T23:43:02.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.116+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f1c073e10 con 0x7f8f1c100e90 2024-04-22T23:43:02.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.117+0000 7f8f127fc700 1 -- 172.21.15.31:0/4059779126 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8f1c073e10 con 0x7f8f1c100e90 2024-04-22T23:43:02.109 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-22T23:43:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f8f0806c4e0 msgr2=0x7f8f0806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f8f0806c4e0 0x7f8f0806e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c074880 tx=0x7f8f0c01a040 comp rx=0 tx=0).stop 2024-04-22T23:43:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 msgr2=0x7f8f1c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 0x7f8f1c073090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800d8d0 tx=0x7f8f1800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 shutdown_connections 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f8f0806c4e0 0x7f8f0806e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c100e90 0x7f8f1c073090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 --2- 172.21.15.31:0/4059779126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f1c101840 0x7f8f1c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 >> 172.21.15.31:0/4059779126 conn(0x7f8f1c0fc740 msgr2=0x7f8f1c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 shutdown_connections 2024-04-22T23:43:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:02.120+0000 7f8f2384f700 1 -- 172.21.15.31:0/4059779126 wait complete. 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T23:43:02.680 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-22T23:43:02.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:02.681 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-04-22T23:43:02.700 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42306sec preferred_lft 42306sec 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-04-22T23:43:02.701 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-04-22T23:43:02.702 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T23:43:02.702 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:43:02.702 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T23:43:02.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-04-22T23:43:02.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-04-22T23:43:02.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-04-22T23:43:02.702 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T23:43:02.703 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T23:43:02.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:43:02.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:02.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:02.910 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T23:43:02.935 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-04-22T23:43:02.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:02.937 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-04-22T23:43:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-04-22T23:43:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T23:43:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T23:43:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T23:43:03.010 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T23:43:03.010 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:43:03.013 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T23:43:03.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T23:43:03.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T23:43:03.014 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T23:43:03.014 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T23:43:03.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:43:03.107 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:02 smithi044 ceph-mon[34463]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 967 B/s rd, 0 B/s wr, 1 op/s 2024-04-22T23:43:03.107 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:02 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/4059779126' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:43:03.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:02 smithi031 ceph-mon[26721]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 967 B/s rd, 0 B/s wr, 1 op/s 2024-04-22T23:43:03.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:02 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4059779126' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:43:03.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:03.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:03.283 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T23:43:03.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:03.298 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-04-22T23:43:03.354 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:43:03.354 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T23:43:03.354 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:03.354 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-22T23:43:03.354 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T23:43:03.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:43:03.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:03.471 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-22T23:43:03.471 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:43:03.471 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T23:43:03.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-22T23:43:03.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T23:43:03.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-22T23:43:03.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-22T23:43:03.472 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-22T23:43:03.472 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T23:43:03.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:43:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:03.595 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:43:03.595 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T23:43:03.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-04-22T23:43:03.595 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T23:43:03.595 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T23:43:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:43:03.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:03.672 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-22T23:43:03.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:43:03.672 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:03.687 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-22T23:43:03.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:43:03.687 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:03.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T23:43:03.807 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T23:43:03.864 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T23:43:03.865 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/0 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/0eb886892ee86fe87523550dc6c58123c31cbf5eed4dd884733461715583fa53/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/e5a1dfb56264634a3cc3ad1487a7e76bdbc49e1c91b34f33fb7dc519384506b0/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/d22860ecb8b45596182939093bdf6c3b5e0a073153e4cd4d7d1a4c9c52e6639f/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/034ddc0f4dce40b540c59429068f7c37e789abd82573bbb108def9bd56da0358/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/48f3b0bbd5803b80c9a2024aced775a5d0f96c70a7722ecf7fc5e75666e0ce75/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/a8b60cdc9aea5a44f3d861e6ecc5fd7031e96ccad27b7e11ff8b6938392830b4/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/305e547b80e2abf016e3df3b6a7e0ce94962289fba7ec2e8828bc52742a3bff6/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/02928bda44568d665edfcbb02207d367004896a3fe0dfc44bc0c34aa9c47fb5e/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/968124acfa1885154c0417b9d4c2b7002478ef666a13e83f7cd2908e9c62a03a/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/99374d0a8bf1e18cd353eae42da6887e5f7855c24dd9074e5a307411ff23e891/merged 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T23:43:03.866 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:03.867 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:03.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:03.867 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T23:43:03.918 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-22T23:43:03.918 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-22T23:43:03.961 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T23:43:03.985 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T23:43:04.041 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T23:43:04.042 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/0 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/0eb886892ee86fe87523550dc6c58123c31cbf5eed4dd884733461715583fa53/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/e5a1dfb56264634a3cc3ad1487a7e76bdbc49e1c91b34f33fb7dc519384506b0/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/d22860ecb8b45596182939093bdf6c3b5e0a073153e4cd4d7d1a4c9c52e6639f/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/034ddc0f4dce40b540c59429068f7c37e789abd82573bbb108def9bd56da0358/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/48f3b0bbd5803b80c9a2024aced775a5d0f96c70a7722ecf7fc5e75666e0ce75/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/a8b60cdc9aea5a44f3d861e6ecc5fd7031e96ccad27b7e11ff8b6938392830b4/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/305e547b80e2abf016e3df3b6a7e0ce94962289fba7ec2e8828bc52742a3bff6/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/02928bda44568d665edfcbb02207d367004896a3fe0dfc44bc0c34aa9c47fb5e/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/968124acfa1885154c0417b9d4c2b7002478ef666a13e83f7cd2908e9c62a03a/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/99374d0a8bf1e18cd353eae42da6887e5f7855c24dd9074e5a307411ff23e891/merged 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:04.043 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:04.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:04.044 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T23:43:04.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi044.stderr:2024-04-22T23:43:04.101+0000 7f1635b41480 -1 init, newargv = 0x55c52fb95410 newargc=15 2024-04-22T23:43:04.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi044.stderr:ceph-fuse[60704]: starting ceph client 2024-04-22T23:43:04.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi044.stderr:ceph-fuse[60704]: starting fuse 2024-04-22T23:43:05.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:04 smithi031 ceph-mon[26721]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s rd, 193 B/s wr, 2 op/s 2024-04-22T23:43:05.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T23:43:05.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:04 smithi044 ceph-mon[34463]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s rd, 193 B/s wr, 2 op/s 2024-04-22T23:43:05.229 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T23:43:05.289 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T23:43:05.289 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T23:43:05.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/0 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/0eb886892ee86fe87523550dc6c58123c31cbf5eed4dd884733461715583fa53/merged 2024-04-22T23:43:05.291 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/e5a1dfb56264634a3cc3ad1487a7e76bdbc49e1c91b34f33fb7dc519384506b0/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/d22860ecb8b45596182939093bdf6c3b5e0a073153e4cd4d7d1a4c9c52e6639f/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/034ddc0f4dce40b540c59429068f7c37e789abd82573bbb108def9bd56da0358/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/48f3b0bbd5803b80c9a2024aced775a5d0f96c70a7722ecf7fc5e75666e0ce75/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/a8b60cdc9aea5a44f3d861e6ecc5fd7031e96ccad27b7e11ff8b6938392830b4/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/305e547b80e2abf016e3df3b6a7e0ce94962289fba7ec2e8828bc52742a3bff6/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/02928bda44568d665edfcbb02207d367004896a3fe0dfc44bc0c34aa9c47fb5e/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/968124acfa1885154c0417b9d4c2b7002478ef666a13e83f7cd2908e9c62a03a/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/containers/storage/overlay/99374d0a8bf1e18cd353eae42da6887e5f7855c24dd9074e5a307411ff23e891/merged 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:43:05.292 INFO:teuthology.orchestra.run.smithi044.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:05.293 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T23:43:05.348 INFO:teuthology.orchestra.run.smithi044.stdout:123 2024-04-22T23:43:05.348 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [123] 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-04-22T23:43:05.348 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T23:43:05.349 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.1")) 2024-04-22T23:43:05.350 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-04-22T23:43:05.476 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.1.60704.asok 2024-04-22T23:43:05.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T23:43:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:43:05.487 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.60704.asok 2024-04-22T23:43:05.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:05.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.60704.asok status 2024-04-22T23:43:05.626 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T23:43:05.626 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-04-22T23:43:05.626 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "1", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "60704", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24345, 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 24345 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T23:43:05.627 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3446383703 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 3446383703 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.24345 192.168.0.1:0/3446383703", 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/3446383703", 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 18, 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 46, 2024-04-22T23:43:05.628 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-04-22T23:43:05.629 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-04-22T23:43:05.629 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-04-22T23:43:05.629 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T23:43:05.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:05.635 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-04-22T23:43:05.635 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-22T23:43:05.656 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-04-22T23:43:05.656 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-22T23:43:05.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:05.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T23:43:05.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:05.729 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-04-22T23:43:05.729 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.744 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-04-22T23:43:05.745 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:43:05.745 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.811 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:43:05.821 INFO:teuthology.task.print:**** done client 2024-04-22T23:43:05.821 INFO:teuthology.run_tasks:Running task parallel... 2024-04-22T23:43:05.830 INFO:teuthology.task.parallel:starting parallel... 2024-04-22T23:43:05.830 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-22T23:43:05.831 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:43:05.831 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:43:05.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-22T23:43:05.831 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-22T23:43:05.832 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-22T23:43:05.835 INFO:tasks.workunit:Pulling workunits from ref 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:05.835 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T23:43:05.835 INFO:tasks.workunit:timeout=3h 2024-04-22T23:43:05.835 INFO:tasks.workunit:cleanup=True 2024-04-22T23:43:05.836 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T23:43:05.856 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T23:43:05.856 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-22T23:43:05.856 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 9ch/156d Inode: 1 Links: 2 2024-04-22T23:43:05.857 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:43:05.857 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-22T23:43:05.857 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T23:43:05.857 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-22 23:42:32.982292659 +0000 2024-04-22T23:43:05.857 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-22 23:43:05.734774122 +0000 2024-04-22T23:43:05.857 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-04-22T23:43:05.857 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T23:43:05.857 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T23:43:05.928 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 7bh/123d Inode: 1 Links: 3 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T23:43:05.945 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 23:43:05.931665658 +0000 2024-04-22T23:43:05.946 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 23:43:05.931665658 +0000 2024-04-22T23:43:05.946 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-22T23:43:05.946 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-22T23:43:05.946 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-22T23:43:06.015 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:06.016 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:06.033 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T23:43:06.070 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:43:06.072 INFO:tasks.workunit.client.1.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-22T23:43:07.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:06 smithi031 ceph-mon[26721]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s rd, 193 B/s wr, 2 op/s 2024-04-22T23:43:07.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:06 smithi044 ceph-mon[34463]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s rd, 193 B/s wr, 2 op/s 2024-04-22T23:43:07.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 -- 172.21.15.31:0/1390262683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 msgr2=0x7f59cc1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 --2- 172.21.15.31:0/1390262683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc1010d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009910 tx=0x7f59c0009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 -- 172.21.15.31:0/1390262683 shutdown_connections 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 --2- 172.21.15.31:0/1390262683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc1010d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 --2- 172.21.15.31:0/1390262683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 0x7f59cc0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 -- 172.21.15.31:0/1390262683 >> 172.21.15.31:0/1390262683 conn(0x7f59cc0fafd0 msgr2=0x7f59cc0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 -- 172.21.15.31:0/1390262683 shutdown_connections 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.723+0000 7f59d2986700 1 -- 172.21.15.31:0/1390262683 wait complete. 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59d2986700 1 Processor -- start 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59d2986700 1 -- start start 2024-04-22T23:43:07.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59d2986700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 0x7f59cc1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59d2986700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59d2986700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc196e50 con 0x7f59cc100c70 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59d2986700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc196f90 con 0x7f59cc0ffa70 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59cb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58136/0 (socket says 172.21.15.31:58136) 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59cb7fe700 1 -- 172.21.15.31:0/4226018313 learned_addr learned my addr 172.21.15.31:0/4226018313 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.724+0000 7f59cbfff700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 0x7f59cc1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59cb7fe700 1 -- 172.21.15.31:0/4226018313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 msgr2=0x7f59cc1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59cb7fe700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 0x7f59cc1962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59cb7fe700 1 -- 172.21.15.31:0/4226018313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00095f0 con 0x7f59cc100c70 2024-04-22T23:43:07.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59cbfff700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 0x7f59cc1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:43:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59cb7fe700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc196830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006010 tx=0x7f59c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59c001d070 con 0x7f59cc100c70 2024-04-22T23:43:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f59c000f460 con 0x7f59cc100c70 2024-04-22T23:43:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc19b9e0 con 0x7f59cc100c70 2024-04-22T23:43:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59c0021640 con 0x7f59cc100c70 2024-04-22T23:43:07.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.725+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc19bed0 con 0x7f59cc100c70 2024-04-22T23:43:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.726+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc1085d0 con 0x7f59cc100c70 2024-04-22T23:43:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.726+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f59c0004b20 con 0x7f59cc100c70 2024-04-22T23:43:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.727+0000 7f59c97fa700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f59b406c4e0 0x7f59b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.727+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f59c008cd10 con 0x7f59cc100c70 2024-04-22T23:43:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.727+0000 7f59cbfff700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f59b406c4e0 0x7f59b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:07.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.727+0000 7f59cbfff700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f59b406c4e0 0x7f59b406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59cc100ad0 tx=0x7f59bc005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:07.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.731+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f59c005b040 con 0x7f59cc100c70 2024-04-22T23:43:07.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.932+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f59cc19c8c0 con 0x7f59cc100c70 2024-04-22T23:43:07.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.932+0000 7f59c97fa700 1 -- 172.21.15.31:0/4226018313 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f59c005abd0 con 0x7f59cc100c70 2024-04-22T23:43:07.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f59b406c4e0 msgr2=0x7f59b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:07.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f59b406c4e0 0x7f59b406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59cc100ad0 tx=0x7f59bc005f90 comp rx=0 tx=0).stop 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 msgr2=0x7f59cc196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc196830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006010 tx=0x7f59c000be20 comp rx=0 tx=0).stop 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 shutdown_connections 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f59b406c4e0 0x7f59b406e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59cc0ffa70 0x7f59cc1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 --2- 172.21.15.31:0/4226018313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59cc100c70 0x7f59cc196830 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 >> 172.21.15.31:0/4226018313 conn(0x7f59cc0fafd0 msgr2=0x7f59cc068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 shutdown_connections 2024-04-22T23:43:07.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:07.935+0000 7f59d2986700 1 -- 172.21.15.31:0/4226018313 wait complete. 2024-04-22T23:43:08.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-22T23:43:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:43:09.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:08 smithi031 ceph-mon[26721]: pgmap v196: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-22T23:43:09.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:08 smithi044 ceph-mon[34463]: pgmap v196: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 -- 172.21.15.31:0/1353268790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 msgr2=0x7f5958101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1353268790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958101270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5948009910 tx=0x7f5948009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 -- 172.21.15.31:0/1353268790 shutdown_connections 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1353268790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958101270 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1353268790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59580ffc10 0x7f5958100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 -- 172.21.15.31:0/1353268790 >> 172.21.15.31:0/1353268790 conn(0x7f59580fb190 msgr2=0x7f59580fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 -- 172.21.15.31:0/1353268790 shutdown_connections 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 -- 172.21.15.31:0/1353268790 wait complete. 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 Processor -- start 2024-04-22T23:43:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.377+0000 7f595f4e5700 1 -- start start 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595f4e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59580ffc10 0x7f5958198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595f4e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595f4e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59581991e0 con 0x7f5958100e10 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595f4e5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5958199320 con 0x7f59580ffc10 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58160/0 (socket says 172.21.15.31:58160) 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 -- 172.21.15.31:0/1182835778 learned_addr learned my addr 172.21.15.31:0/1182835778 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 -- 172.21.15.31:0/1182835778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59580ffc10 msgr2=0x7f5958198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59580ffc10 0x7f5958198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 -- 172.21.15.31:0/1182835778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480095f0 con 0x7f5958100e10 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595ca80700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958198bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f594800bd70 tx=0x7f594800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594801d070 con 0x7f5958100e10 2024-04-22T23:43:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595819dd70 con 0x7f5958100e10 2024-04-22T23:43:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5948022470 con 0x7f5958100e10 2024-04-22T23:43:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594800f670 con 0x7f5958100e10 2024-04-22T23:43:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.378+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595819e260 con 0x7f5958100e10 2024-04-22T23:43:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.379+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958108700 con 0x7f5958100e10 2024-04-22T23:43:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.380+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5948022a50 con 0x7f5958100e10 2024-04-22T23:43:10.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.380+0000 7f594e7fc700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f594406c4e0 0x7f594406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:10.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.380+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f594808cef0 con 0x7f5958100e10 2024-04-22T23:43:10.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.380+0000 7f595d281700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f594406c4e0 0x7f594406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:10.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.381+0000 7f595d281700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f594406c4e0 0x7f594406e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5958100c70 tx=0x7f5954005ca0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:10.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.385+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f594805b1b0 con 0x7f5958100e10 2024-04-22T23:43:10.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.583+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f595819e870 con 0x7f5958100e10 2024-04-22T23:43:10.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.583+0000 7f594e7fc700 1 -- 172.21.15.31:0/1182835778 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f594805ad40 con 0x7f5958100e10 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f594406c4e0 msgr2=0x7f594406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f594406c4e0 0x7f594406e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5958100c70 tx=0x7f5954005ca0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 msgr2=0x7f5958198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958198bc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f594800bd70 tx=0x7f594800be50 comp rx=0 tx=0).stop 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 shutdown_connections 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f594406c4e0 0x7f594406e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59580ffc10 0x7f5958198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 --2- 172.21.15.31:0/1182835778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5958100e10 0x7f5958198bc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.585+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 >> 172.21.15.31:0/1182835778 conn(0x7f59580fb190 msgr2=0x7f5958068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.586+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 shutdown_connections 2024-04-22T23:43:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:10.586+0000 7f595f4e5700 1 -- 172.21.15.31:0/1182835778 wait complete. 2024-04-22T23:43:11.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:10 smithi031 ceph-mon[26721]: pgmap v197: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-22T23:43:11.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-22T23:43:11.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:10 smithi044 ceph-mon[34463]: pgmap v197: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-22T23:43:11.409 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 -- 172.21.15.31:0/86766867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 msgr2=0x7f54940fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 --2- 172.21.15.31:0/86766867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f54940fffe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5490009910 tx=0x7f5490009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 -- 172.21.15.31:0/86766867 shutdown_connections 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 --2- 172.21.15.31:0/86766867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5494100dc0 0x7f5494101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 --2- 172.21.15.31:0/86766867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f54940fffe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 -- 172.21.15.31:0/86766867 >> 172.21.15.31:0/86766867 conn(0x7f54940fb160 msgr2=0x7f54940fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 -- 172.21.15.31:0/86766867 shutdown_connections 2024-04-22T23:43:12.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.965+0000 7f549b329700 1 -- 172.21.15.31:0/86766867 wait complete. 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f549b329700 1 Processor -- start 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f549b329700 1 -- start start 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f549b329700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f5494103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f549b329700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5494100dc0 0x7f54941021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f549b329700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54941026e0 con 0x7f54940ffbc0 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f549b329700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494102820 con 0x7f5494100dc0 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f54990c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f5494103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f54990c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f5494103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51888/0 (socket says 172.21.15.31:51888) 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f54990c5700 1 -- 172.21.15.31:0/1969058015 learned_addr learned my addr 172.21.15.31:0/1969058015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f54988c4700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5494100dc0 0x7f54941021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.966+0000 7f54990c5700 1 -- 172.21.15.31:0/1969058015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5494100dc0 msgr2=0x7f54941021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:12.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f54990c5700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5494100dc0 0x7f54941021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f54990c5700 1 -- 172.21.15.31:0/1969058015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900095f0 con 0x7f54940ffbc0 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f54990c5700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f5494103b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f549000bda0 tx=0x7f549000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f549001d070 con 0x7f54940ffbc0 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f549000f460 con 0x7f54940ffbc0 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5494102aa0 con 0x7f54940ffbc0 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5490021620 con 0x7f54940ffbc0 2024-04-22T23:43:12.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.967+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494102f90 con 0x7f54940ffbc0 2024-04-22T23:43:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.968+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494068af0 con 0x7f54940ffbc0 2024-04-22T23:43:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.968+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f549000f5d0 con 0x7f54940ffbc0 2024-04-22T23:43:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.969+0000 7f548a7fc700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f548006c4e0 0x7f548006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:12.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.969+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f549008ca00 con 0x7f54940ffbc0 2024-04-22T23:43:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.969+0000 7f54988c4700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f548006c4e0 0x7f548006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:12.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.969+0000 7f54988c4700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f548006c4e0 0x7f548006e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5484005950 tx=0x7f548400b410 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:12.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:12.973+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f549005ac90 con 0x7f54940ffbc0 2024-04-22T23:43:13.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.171+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f549419e910 con 0x7f54940ffbc0 2024-04-22T23:43:13.164 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:12 smithi031 ceph-mon[26721]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-22T23:43:13.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.171+0000 7f548a7fc700 1 -- 172.21.15.31:0/1969058015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f549005a820 con 0x7f54940ffbc0 2024-04-22T23:43:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f548006c4e0 msgr2=0x7f548006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f548006c4e0 0x7f548006e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5484005950 tx=0x7f548400b410 comp rx=0 tx=0).stop 2024-04-22T23:43:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 msgr2=0x7f5494103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f5494103b50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f549000bda0 tx=0x7f549000bdd0 comp rx=0 tx=0).stop 2024-04-22T23:43:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 shutdown_connections 2024-04-22T23:43:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f548006c4e0 0x7f548006e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940ffbc0 0x7f5494103b50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.173+0000 7f549b329700 1 --2- 172.21.15.31:0/1969058015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5494100dc0 0x7f54941021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.174+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 >> 172.21.15.31:0/1969058015 conn(0x7f54940fb160 msgr2=0x7f5494068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.174+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 shutdown_connections 2024-04-22T23:43:13.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:13.174+0000 7f549b329700 1 -- 172.21.15.31:0/1969058015 wait complete. 2024-04-22T23:43:13.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:12 smithi044 ceph-mon[34463]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-04-22T23:43:13.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-22T23:43:13.976 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:43:15.136 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:14 smithi031 ceph-mon[26721]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-22T23:43:15.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:14 smithi044 ceph-mon[34463]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 -- 172.21.15.31:0/2888482232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 msgr2=0x7f9920102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2888482232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920102460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9914009910 tx=0x7f9914009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 -- 172.21.15.31:0/2888482232 shutdown_connections 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2888482232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920102460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2888482232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 0x7f99200ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 -- 172.21.15.31:0/2888482232 >> 172.21.15.31:0/2888482232 conn(0x7f99200fa8c0 msgr2=0x7f99200fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 -- 172.21.15.31:0/2888482232 shutdown_connections 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.446+0000 7f9924ff5700 1 -- 172.21.15.31:0/2888482232 wait complete. 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f9924ff5700 1 Processor -- start 2024-04-22T23:43:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f9924ff5700 1 -- start start 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f9924ff5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 0x7f9920198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f9924ff5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f9924ff5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99201992e0 con 0x7f9920102000 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f9924ff5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920199420 con 0x7f99200fef80 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51912/0 (socket says 172.21.15.31:51912) 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991dd9b700 1 -- 172.21.15.31:0/2739655413 learned_addr learned my addr 172.21.15.31:0/2739655413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991e59c700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 0x7f9920198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991dd9b700 1 -- 172.21.15.31:0/2739655413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 msgr2=0x7f9920198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991dd9b700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 0x7f9920198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991dd9b700 1 -- 172.21.15.31:0/2739655413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140095f0 con 0x7f9920102000 2024-04-22T23:43:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.447+0000 7f991e59c700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 0x7f9920198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:43:15.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.448+0000 7f991dd9b700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920198cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9914006010 tx=0x7f991400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:15.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.448+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f991401d070 con 0x7f9920102000 2024-04-22T23:43:15.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.448+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f991400f460 con 0x7f9920102000 2024-04-22T23:43:15.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.448+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992019de70 con 0x7f9920102000 2024-04-22T23:43:15.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.448+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9914021640 con 0x7f9920102000 2024-04-22T23:43:15.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.448+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992019e330 con 0x7f9920102000 2024-04-22T23:43:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.449+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99201060e0 con 0x7f9920102000 2024-04-22T23:43:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.449+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f991400fab0 con 0x7f9920102000 2024-04-22T23:43:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.450+0000 7f990f7fe700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f990806c4e0 0x7f990806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.450+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f991408cd20 con 0x7f9920102000 2024-04-22T23:43:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.450+0000 7f991e59c700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f990806c4e0 0x7f990806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.450+0000 7f991e59c700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f990806c4e0 0x7f990806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9920068b50 tx=0x7f9910008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:15.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.454+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f991405af30 con 0x7f9920102000 2024-04-22T23:43:15.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:15.670+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f9920061280 con 0x7f990806c4e0 2024-04-22T23:43:16.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:16.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:17.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:16 smithi031 ceph-mon[26721]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-22T23:43:17.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:16 smithi031 ceph-mon[26721]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:17.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:16 smithi044 ceph-mon[34463]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-22T23:43:17.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:16 smithi044 ceph-mon[34463]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:18 smithi031 ceph-mon[26721]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-22T23:43:19.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:18 smithi044 ceph-mon[34463]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-22T23:43:21.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:20 smithi031 ceph-mon[26721]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-22T23:43:21.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:20 smithi044 ceph-mon[34463]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-22T23:43:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:22 smithi031 ceph-mon[26721]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-22T23:43:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:22 smithi044 ceph-mon[34463]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-04-22T23:43:25.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:24 smithi031 ceph-mon[26721]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-22T23:43:25.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:24 smithi044 ceph-mon[34463]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-04-22T23:43:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:26 smithi031 ceph-mon[26721]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-22T23:43:27.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:26 smithi044 ceph-mon[34463]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-22T23:43:29.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:28 smithi044 ceph-mon[34463]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-22T23:43:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:28 smithi031 ceph-mon[26721]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-22T23:43:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:30 smithi044 ceph-mon[34463]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-22T23:43:31.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:31.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:30 smithi031 ceph-mon[26721]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-22T23:43:31.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:33.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:32 smithi044 ceph-mon[34463]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:32 smithi031 ceph-mon[26721]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:35.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:34 smithi044 ceph-mon[34463]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:35.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:34 smithi031 ceph-mon[26721]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:37.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:36 smithi044 ceph-mon[34463]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:37.246 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:36 smithi031 ceph-mon[26721]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:39.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:38 smithi044 ceph-mon[34463]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:39.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:38 smithi031 ceph-mon[26721]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:41.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:40 smithi031 ceph-mon[26721]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:41.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:40 smithi044 ceph-mon[34463]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:43.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:42 smithi031 ceph-mon[26721]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:43.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:42 smithi044 ceph-mon[34463]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:45.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:44 smithi031 ceph-mon[26721]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:45.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:44 smithi044 ceph-mon[34463]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:46.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:47.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:46 smithi031 ceph-mon[26721]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:47.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:46 smithi044 ceph-mon[34463]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:49.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:48 smithi031 ceph-mon[26721]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:49.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:48 smithi044 ceph-mon[34463]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:49.879 INFO:teuthology.orchestra.run.smithi031.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:49.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.886+0000 7f990f7fe700 1 -- 172.21.15.31:0/2739655413 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f9920061280 con 0x7f990806c4e0 2024-04-22T23:43:49.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f990806c4e0 msgr2=0x7f990806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:49.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f990806c4e0 0x7f990806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9920068b50 tx=0x7f9910008040 comp rx=0 tx=0).stop 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 msgr2=0x7f9920198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920198cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9914006010 tx=0x7f991400be20 comp rx=0 tx=0).stop 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 shutdown_connections 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f990806c4e0 0x7f990806e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99200fef80 0x7f9920198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 --2- 172.21.15.31:0/2739655413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9920102000 0x7f9920198cc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 >> 172.21.15.31:0/2739655413 conn(0x7f99200fa8c0 msgr2=0x7f9920103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 shutdown_connections 2024-04-22T23:43:49.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:49.888+0000 7f9924ff5700 1 -- 172.21.15.31:0/2739655413 wait complete. 2024-04-22T23:43:50.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-22T23:43:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:51.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:50 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:51.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:50 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:52.138 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:51 smithi031 ceph-mon[26721]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:52.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:51 smithi044 ceph-mon[34463]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:43:52.928 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:52 smithi031 ceph-mon[26721]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:53.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:52 smithi044 ceph-mon[34463]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 -- 172.21.15.31:0/3529469713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 msgr2=0x7f1a1c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 --2- 172.21.15.31:0/3529469713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c101270 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009910 tx=0x7f1a0c009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 -- 172.21.15.31:0/3529469713 shutdown_connections 2024-04-22T23:43:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 --2- 172.21.15.31:0/3529469713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c101270 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 --2- 172.21.15.31:0/3529469713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 -- 172.21.15.31:0/3529469713 >> 172.21.15.31:0/3529469713 conn(0x7f1a1c0fb190 msgr2=0x7f1a1c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 -- 172.21.15.31:0/3529469713 shutdown_connections 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 -- 172.21.15.31:0/3529469713 wait complete. 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.595+0000 7f1a23c24700 1 Processor -- start 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a23c24700 1 -- start start 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a23c24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a23c24700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a23c24700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c102730 con 0x7f1a1c100e10 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a23c24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c102870 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a219c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a211bf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a219c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:54810/0 (socket says 172.21.15.31:54810) 2024-04-22T23:43:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a219c0700 1 -- 172.21.15.31:0/2866668668 learned_addr learned my addr 172.21.15.31:0/2866668668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a219c0700 1 -- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 msgr2=0x7f1a1c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a219c0700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a219c0700 1 -- 172.21.15.31:0/2866668668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0095f0 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.596+0000 7f1a211bf700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.597+0000 7f1a219c0700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c103ba0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e8f0 tx=0x7f1a1800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.597+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a18009790 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.597+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c102b50 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.597+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a1800c810 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.597+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a18003d20 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.597+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c19e4e0 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.598+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c108700 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.598+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1a180041f0 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.599+0000 7f1a12ffd700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a0806c4e0 0x7f1a0806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.599+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1a18014070 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.599+0000 7f1a211bf700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a0806c4e0 0x7f1a0806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.599+0000 7f1a211bf700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a0806c4e0 0x7f1a0806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009fd0 tx=0x7f1a0c005940 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:53.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.603+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1a18059d90 con 0x7f1a1c0ffc10 2024-04-22T23:43:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.824+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a1c068c90 con 0x7f1a0806c4e0 2024-04-22T23:43:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.825+0000 7f1a12ffd700 1 -- 172.21.15.31:0/2866668668 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f1a1c068c90 con 0x7f1a0806c4e0 2024-04-22T23:43:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a0806c4e0 msgr2=0x7f1a0806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a0806c4e0 0x7f1a0806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009fd0 tx=0x7f1a0c005940 comp rx=0 tx=0).stop 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 msgr2=0x7f1a1c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c103ba0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e8f0 tx=0x7f1a1800ec00 comp rx=0 tx=0).stop 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 shutdown_connections 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f1a0806c4e0 0x7f1a0806e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a1c0ffc10 0x7f1a1c103ba0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 --2- 172.21.15.31:0/2866668668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a1c100e10 0x7f1a1c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 >> 172.21.15.31:0/2866668668 conn(0x7f1a1c0fb190 msgr2=0x7f1a1c068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 shutdown_connections 2024-04-22T23:43:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.828+0000 7f1a23c24700 1 -- 172.21.15.31:0/2866668668 wait complete. 2024-04-22T23:43:53.831 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:43:53.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.937+0000 7f0579095700 1 -- 172.21.15.31:0/509678394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05740ffc10 msgr2=0x7f0574100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.937+0000 7f0579095700 1 --2- 172.21.15.31:0/509678394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05740ffc10 0x7f0574100030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0564009910 tx=0x7f0564009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.937+0000 7f0579095700 1 -- 172.21.15.31:0/509678394 shutdown_connections 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.937+0000 7f0579095700 1 --2- 172.21.15.31:0/509678394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0574100e10 0x7f0574101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.937+0000 7f0579095700 1 --2- 172.21.15.31:0/509678394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05740ffc10 0x7f0574100030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.938+0000 7f0579095700 1 -- 172.21.15.31:0/509678394 >> 172.21.15.31:0/509678394 conn(0x7f05740fb190 msgr2=0x7f05740fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.938+0000 7f0579095700 1 -- 172.21.15.31:0/509678394 shutdown_connections 2024-04-22T23:43:53.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.938+0000 7f0579095700 1 -- 172.21.15.31:0/509678394 wait complete. 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.938+0000 7f0579095700 1 Processor -- start 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f0579095700 1 -- start start 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f0579095700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740ffc10 0x7f05741964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f0579095700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 0x7f0574196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f0579095700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0574197030 con 0x7f0574100e10 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f0579095700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0574197170 con 0x7f05740ffc10 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 0x7f0574196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 0x7f0574196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41134/0 (socket says 172.21.15.31:41134) 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 -- 172.21.15.31:0/1315073354 learned_addr learned my addr 172.21.15.31:0/1315073354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f0572d9d700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740ffc10 0x7f05741964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 -- 172.21.15.31:0/1315073354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740ffc10 msgr2=0x7f05741964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:53.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740ffc10 0x7f05741964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 -- 172.21.15.31:0/1315073354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640095f0 con 0x7f0574100e10 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.939+0000 7f057259c700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 0x7f0574196a10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f056800d900 tx=0x7f056800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.940+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05680041f0 con 0x7f0574100e10 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.940+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057419bc20 con 0x7f0574100e10 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.940+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0568004ce0 con 0x7f0574100e10 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.940+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056800b750 con 0x7f0574100e10 2024-04-22T23:43:53.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.940+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057419c170 con 0x7f0574100e10 2024-04-22T23:43:53.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.941+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0574108700 con 0x7f0574100e10 2024-04-22T23:43:53.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.941+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0568004e50 con 0x7f0574100e10 2024-04-22T23:43:53.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.942+0000 7f055bfff700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f055c06c4e0 0x7f055c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:53.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.942+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f056808b6f0 con 0x7f0574100e10 2024-04-22T23:43:53.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.942+0000 7f0572d9d700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f055c06c4e0 0x7f055c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:53.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.942+0000 7f0572d9d700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f055c06c4e0 0x7f055c06e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0564006010 tx=0x7f056400b560 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:53.947+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0568059920 con 0x7f0574100e10 2024-04-22T23:43:54.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.170+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0574068c90 con 0x7f055c06c4e0 2024-04-22T23:43:54.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.171+0000 7f055bfff700 1 -- 172.21.15.31:0/1315073354 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0574068c90 con 0x7f055c06c4e0 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f055c06c4e0 msgr2=0x7f055c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f055c06c4e0 0x7f055c06e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0564006010 tx=0x7f056400b560 comp rx=0 tx=0).stop 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 msgr2=0x7f0574196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 0x7f0574196a10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f056800d900 tx=0x7f056800dc10 comp rx=0 tx=0).stop 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 shutdown_connections 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f055c06c4e0 0x7f055c06e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740ffc10 0x7f05741964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 --2- 172.21.15.31:0/1315073354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0574100e10 0x7f0574196a10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 >> 172.21.15.31:0/1315073354 conn(0x7f05740fb190 msgr2=0x7f0574068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 shutdown_connections 2024-04-22T23:43:54.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.174+0000 7f0579095700 1 -- 172.21.15.31:0/1315073354 wait complete. 2024-04-22T23:43:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 -- 172.21.15.31:0/3103745480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 msgr2=0x7f0950102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 --2- 172.21.15.31:0/3103745480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950102300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0938009910 tx=0x7f0938009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 -- 172.21.15.31:0/3103745480 shutdown_connections 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 --2- 172.21.15.31:0/3103745480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950102300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 --2- 172.21.15.31:0/3103745480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0950100ca0 0x7f09501010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 -- 172.21.15.31:0/3103745480 >> 172.21.15.31:0/3103745480 conn(0x7f09500fc260 msgr2=0x7f09500fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 -- 172.21.15.31:0/3103745480 shutdown_connections 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 -- 172.21.15.31:0/3103745480 wait complete. 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.288+0000 7f0955b0a700 1 Processor -- start 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f0955b0a700 1 -- start start 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f0955b0a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0950100ca0 0x7f09501964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f0955b0a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f0955b0a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950197040 con 0x7f0950101ea0 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f0955b0a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950197180 con 0x7f0950100ca0 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41146/0 (socket says 172.21.15.31:41146) 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 -- 172.21.15.31:0/2394700926 learned_addr learned my addr 172.21.15.31:0/2394700926 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094f7fe700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0950100ca0 0x7f09501964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 -- 172.21.15.31:0/2394700926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0950100ca0 msgr2=0x7f09501964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0950100ca0 0x7f09501964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 -- 172.21.15.31:0/2394700926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09380095f0 con 0x7f0950101ea0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.289+0000 7f094effd700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950196a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0938000c00 tx=0x7f093800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.290+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093801d070 con 0x7f0950101ea0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.290+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095019bbd0 con 0x7f0950101ea0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.290+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0938005dc0 con 0x7f0950101ea0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.290+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09380226a0 con 0x7f0950101ea0 2024-04-22T23:43:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.290+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095019c0c0 con 0x7f0950101ea0 2024-04-22T23:43:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.291+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950109800 con 0x7f0950101ea0 2024-04-22T23:43:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.291+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f09380038f0 con 0x7f0950101ea0 2024-04-22T23:43:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.292+0000 7f094cff9700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f093c06c4e0 0x7f093c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.292+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f093808cc50 con 0x7f0950101ea0 2024-04-22T23:43:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.292+0000 7f094f7fe700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f093c06c4e0 0x7f093c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.292+0000 7f094f7fe700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f093c06c4e0 0x7f093c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0950101d00 tx=0x7f0940009260 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.300+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f093805aee0 con 0x7f0950101ea0 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: Upgrade: Target is version 19.0.0-3252-g43be0201 (unknown) 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c'] 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: Upgrade: Need to upgrade myself (mgr.smithi031.fazdcv) 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c on smithi044 2024-04-22T23:43:54.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:54 smithi031 ceph-mon[26721]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:54.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.525+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f09501067f0 con 0x7f093c06c4e0 2024-04-22T23:43:54.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.548+0000 7f094cff9700 1 -- 172.21.15.31:0/2394700926 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f09501067f0 con 0x7f093c06c4e0 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 59s ago 6m 25.6M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (6m) 59s ago 6m 7440k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (65s) 63s ago 5m 7105k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (6m) 59s ago 6m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (5m) 63s ago 5m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 59s ago 6m 87.7M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (82s) 59s ago 82s 11.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (76s) 59s ago 75s 16.8M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (79s) 63s ago 79s 17.6M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:43:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (73s) 63s ago 73s 15.2M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:9283,8765,8443 running (8m) 59s ago 8m 489M - 18.2.1 5be31c24972a dcdbc372f09c 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (5m) 63s ago 5m 445M - 18.2.1 5be31c24972a b6f8e3606e77 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 59s ago 8m 46.8M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (5m) 63s ago 5m 34.8M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 59s ago 6m 16.6M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (5m) 63s ago 5m 16.2M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 59s ago 4m 47.4M 3948M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 59s ago 4m 47.4M 3948M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 59s ago 4m 46.6M 3948M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (3m) 63s ago 3m 46.1M 4972M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (3m) 63s ago 3m 46.0M 4972M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (2m) 63s ago 2m 46.7M 4972M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:43:54.543 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 59s ago 6m 58.0M - 2.43.0 a07b618ecd1d b125747edf65 2024-04-22T23:43:54.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f093c06c4e0 msgr2=0x7f093c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f093c06c4e0 0x7f093c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0950101d00 tx=0x7f0940009260 comp rx=0 tx=0).stop 2024-04-22T23:43:54.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 msgr2=0x7f0950196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950196a20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0938000c00 tx=0x7f093800f690 comp rx=0 tx=0).stop 2024-04-22T23:43:54.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 shutdown_connections 2024-04-22T23:43:54.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f093c06c4e0 0x7f093c06e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0950100ca0 0x7f09501964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 --2- 172.21.15.31:0/2394700926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0950101ea0 0x7f0950196a20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.550+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 >> 172.21.15.31:0/2394700926 conn(0x7f09500fc260 msgr2=0x7f09501050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.551+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 shutdown_connections 2024-04-22T23:43:54.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.551+0000 7f0955b0a700 1 -- 172.21.15.31:0/2394700926 wait complete. 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 -- 172.21.15.31:0/3289745698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378104150 msgr2=0x7f7378106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 --2- 172.21.15.31:0/3289745698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378104150 0x7f7378106540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f736c009910 tx=0x7f736c009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 -- 172.21.15.31:0/3289745698 shutdown_connections 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 --2- 172.21.15.31:0/3289745698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378104150 0x7f7378106540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 --2- 172.21.15.31:0/3289745698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378101820 0x7f7378103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 -- 172.21.15.31:0/3289745698 >> 172.21.15.31:0/3289745698 conn(0x7f73780fb190 msgr2=0x7f73780fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 -- 172.21.15.31:0/3289745698 shutdown_connections 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.663+0000 7f737d294700 1 -- 172.21.15.31:0/3289745698 wait complete. 2024-04-22T23:43:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f737d294700 1 Processor -- start 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f737d294700 1 -- start start 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f737d294700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378101820 0x7f7378198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f737d294700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 0x7f7378198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f737d294700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73781991e0 con 0x7f7378101820 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f737d294700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7378199320 con 0x7f7378104150 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f7376ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378101820 0x7f7378198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f73767fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 0x7f7378198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f73767fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 0x7f7378198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:54848/0 (socket says 172.21.15.31:54848) 2024-04-22T23:43:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.664+0000 7f73767fc700 1 -- 172.21.15.31:0/4267757222 learned_addr learned my addr 172.21.15.31:0/4267757222 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f73767fc700 1 -- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378101820 msgr2=0x7f7378198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f73767fc700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378101820 0x7f7378198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f73767fc700 1 -- 172.21.15.31:0/4267757222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c0095f0 con 0x7f7378104150 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f7376ffd700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378101820 0x7f7378198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f73767fc700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 0x7f7378198bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f736c0092e0 tx=0x7f736c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c01d070 con 0x7f7378104150 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737819dd70 con 0x7f7378104150 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f736c005dc0 con 0x7f7378104150 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c017630 con 0x7f7378104150 2024-04-22T23:43:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.665+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737819e260 con 0x7f7378104150 2024-04-22T23:43:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.666+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73780fcd80 con 0x7f7378104150 2024-04-22T23:43:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.667+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f736c00faf0 con 0x7f7378104150 2024-04-22T23:43:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.667+0000 7f735ffff700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f736006c4e0 0x7f736006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.668+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f736c044080 con 0x7f7378104150 2024-04-22T23:43:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.668+0000 7f7376ffd700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f736006c4e0 0x7f736006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.668+0000 7f7376ffd700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f736006c4e0 0x7f736006e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7368011490 tx=0x7f7368009260 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:54.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.676+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f736c05b1a0 con 0x7f7378104150 2024-04-22T23:43:54.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: Upgrade: Target is version 19.0.0-3252-g43be0201 (unknown) 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c'] 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: Upgrade: Need to upgrade myself (mgr.smithi031.fazdcv) 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c on smithi044 2024-04-22T23:43:54.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:54 smithi044 ceph-mon[34463]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:54.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.989+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f737802ce80 con 0x7f7378104150 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.990+0000 7f735ffff700 1 -- 172.21.15.31:0/4267757222 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f736c05ad30 con 0x7f7378104150 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:43:54.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:43:54.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:43:54.984 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:43:54.984 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-22T23:43:54.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:43:54.984 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.992+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f736006c4e0 msgr2=0x7f736006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.992+0000 7f737d294700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f736006c4e0 0x7f736006e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7368011490 tx=0x7f7368009260 comp rx=0 tx=0).stop 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.992+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 msgr2=0x7f7378198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.992+0000 7f737d294700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 0x7f7378198bc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f736c0092e0 tx=0x7f736c00f690 comp rx=0 tx=0).stop 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 shutdown_connections 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f736006c4e0 0x7f736006e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7378101820 0x7f7378198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 --2- 172.21.15.31:0/4267757222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7378104150 0x7f7378198bc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 >> 172.21.15.31:0/4267757222 conn(0x7f73780fb190 msgr2=0x7f73781010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 shutdown_connections 2024-04-22T23:43:54.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:54.993+0000 7f737d294700 1 -- 172.21.15.31:0/4267757222 wait complete. 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/507410384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38104170 msgr2=0x7f3d38106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/507410384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38104170 0x7f3d38106560 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009910 tx=0x7f3d2c009c20 comp rx=0 tx=0).stop 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/507410384 shutdown_connections 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/507410384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38104170 0x7f3d38106560 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/507410384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38101840 0x7f3d38103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/507410384 >> 172.21.15.31:0/507410384 conn(0x7f3d380fb190 msgr2=0x7f3d380fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/507410384 shutdown_connections 2024-04-22T23:43:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.106+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/507410384 wait complete. 2024-04-22T23:43:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d3d2d0700 1 Processor -- start 2024-04-22T23:43:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d3d2d0700 1 -- start start 2024-04-22T23:43:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d3d2d0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 0x7f3d38198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d3d2d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 0x7f3d38198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d3d2d0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d38199260 con 0x7f3d38101840 2024-04-22T23:43:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d3d2d0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d381993a0 con 0x7f3d38104170 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 0x7f3d38198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d367fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 0x7f3d38198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d367fc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 0x7f3d38198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:54870/0 (socket says 172.21.15.31:54870) 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d367fc700 1 -- 172.21.15.31:0/282119898 learned_addr learned my addr 172.21.15.31:0/282119898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.107+0000 7f3d36ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 0x7f3d38198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41188/0 (socket says 172.21.15.31:41188) 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d367fc700 1 -- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 msgr2=0x7f3d38198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d367fc700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 0x7f3d38198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d367fc700 1 -- 172.21.15.31:0/282119898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c0095f0 con 0x7f3d38104170 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d36ffd700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 0x7f3d38198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d367fc700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 0x7f3d38198c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c000c00 tx=0x7f3d2c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2c01d070 con 0x7f3d38104170 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d2c005dc0 con 0x7f3d38104170 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3819ddf0 con 0x7f3d38104170 2024-04-22T23:43:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2c0226a0 con 0x7f3d38104170 2024-04-22T23:43:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.108+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3819e2b0 con 0x7f3d38104170 2024-04-22T23:43:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.109+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d38109810 con 0x7f3d38104170 2024-04-22T23:43:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.110+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3d2c0038f0 con 0x7f3d38104170 2024-04-22T23:43:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.111+0000 7f3d1ffff700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3d2006c4e0 0x7f3d2006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.111+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3d2c08cd20 con 0x7f3d38104170 2024-04-22T23:43:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.111+0000 7f3d36ffd700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3d2006c4e0 0x7f3d2006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.111+0000 7f3d36ffd700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3d2006c4e0 0x7f3d2006e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d28011490 tx=0x7f3d28009260 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:55.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.117+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3d2c05af10 con 0x7f3d38104170 2024-04-22T23:43:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.341+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d380ffd40 con 0x7f3d2006c4e0 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.343+0000 7f3d1ffff700 1 -- 172.21.15.31:0/282119898 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3d380ffd40 con 0x7f3d2006c4e0 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-22T23:43:55.335 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "0/2 daemons upgraded", 2024-04-22T23:43:55.336 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c image on host smithi044", 2024-04-22T23:43:55.336 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:43:55.336 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:43:55.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3d2006c4e0 msgr2=0x7f3d2006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:55.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3d2006c4e0 0x7f3d2006e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d28011490 tx=0x7f3d28009260 comp rx=0 tx=0).stop 2024-04-22T23:43:55.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 msgr2=0x7f3d38198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:55.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 0x7f3d38198c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c000c00 tx=0x7f3d2c00f690 comp rx=0 tx=0).stop 2024-04-22T23:43:55.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 shutdown_connections 2024-04-22T23:43:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f3d2006c4e0 0x7f3d2006e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d38101840 0x7f3d38198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 --2- 172.21.15.31:0/282119898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d38104170 0x7f3d38198c40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.345+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 >> 172.21.15.31:0/282119898 conn(0x7f3d380fb190 msgr2=0x7f3d380ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.346+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 shutdown_connections 2024-04-22T23:43:55.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:43:55.346+0000 7f3d3d2d0700 1 -- 172.21.15.31:0/282119898 wait complete. 2024-04-22T23:43:56.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:55 smithi031 ceph-mon[26721]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:55 smithi031 ceph-mon[26721]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:56.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:55 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/4267757222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:55 smithi044 ceph-mon[34463]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:55 smithi044 ceph-mon[34463]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:55 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/4267757222' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:57.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:56 smithi031 ceph-mon[26721]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:57.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:56 smithi031 ceph-mon[26721]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:57.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:56 smithi044 ceph-mon[34463]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:57.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:56 smithi044 ceph-mon[34463]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:43:59.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:43:58 smithi031 ceph-mon[26721]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:43:59.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:43:58 smithi044 ceph-mon[34463]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:00 smithi031 ceph-mon[26721]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:00 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:00 smithi044 ceph-mon[34463]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:00 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:03.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:02 smithi031 ceph-mon[26721]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:02 smithi044 ceph-mon[34463]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:05.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:04 smithi031 ceph-mon[26721]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:05.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:04 smithi044 ceph-mon[34463]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:06 smithi031 ceph-mon[26721]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:07.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:06 smithi044 ceph-mon[34463]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:09.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:08 smithi031 ceph-mon[26721]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:09.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:08 smithi044 ceph-mon[34463]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:11.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:10 smithi031 ceph-mon[26721]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:11.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:10 smithi044 ceph-mon[34463]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:13.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:12 smithi031 ceph-mon[26721]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:12 smithi044 ceph-mon[34463]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:15.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:14 smithi044 ceph-mon[34463]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:15.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:14 smithi031 ceph-mon[26721]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:16.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:15 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:16.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:15 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:17.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:16 smithi044 ceph-mon[34463]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:16 smithi031 ceph-mon[26721]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:19.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:18 smithi044 ceph-mon[34463]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:18 smithi031 ceph-mon[26721]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:21.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:20 smithi044 ceph-mon[34463]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:21.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:20 smithi031 ceph-mon[26721]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:23.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:22 smithi044 ceph-mon[34463]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:23.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:22 smithi031 ceph-mon[26721]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:25.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:24 smithi044 ceph-mon[34463]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:24 smithi031 ceph-mon[26721]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 -- 172.21.15.31:0/987238378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c0ffbb0 msgr2=0x7f670c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 --2- 172.21.15.31:0/987238378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c0ffbb0 0x7f670c0fffd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009910 tx=0x7f66fc009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 -- 172.21.15.31:0/987238378 shutdown_connections 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 --2- 172.21.15.31:0/987238378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f670c100db0 0x7f670c101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 --2- 172.21.15.31:0/987238378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c0ffbb0 0x7f670c0fffd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 -- 172.21.15.31:0/987238378 >> 172.21.15.31:0/987238378 conn(0x7f670c0fb150 msgr2=0x7f670c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 -- 172.21.15.31:0/987238378 shutdown_connections 2024-04-22T23:44:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.515+0000 7f6711451700 1 -- 172.21.15.31:0/987238378 wait complete. 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.516+0000 7f6711451700 1 Processor -- start 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.516+0000 7f6711451700 1 -- start start 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.516+0000 7f6711451700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f670c0ffbb0 0x7f670c1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.516+0000 7f6711451700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 0x7f670c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f6711451700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c197020 con 0x7f670c100db0 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f6711451700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c197160 con 0x7f670c0ffbb0 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 0x7f670c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 0x7f670c196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36168/0 (socket says 172.21.15.31:36168) 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 -- 172.21.15.31:0/243776963 learned_addr learned my addr 172.21.15.31:0/243776963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:25.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 -- 172.21.15.31:0/243776963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f670c0ffbb0 msgr2=0x7f670c1964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f670c0ffbb0 0x7f670c1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 -- 172.21.15.31:0/243776963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0095f0 con 0x7f670c100db0 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.517+0000 7f670a7fc700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 0x7f670c196a00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f670000e920 tx=0x7f670000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.518+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6700009790 con 0x7f670c100db0 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.518+0000 7f6711451700 1 -- 172.21.15.31:0/243776963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c19bc10 con 0x7f670c100db0 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.518+0000 7f6711451700 1 -- 172.21.15.31:0/243776963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c19c160 con 0x7f670c100db0 2024-04-22T23:44:25.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.518+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f670000cba0 con 0x7f670c100db0 2024-04-22T23:44:25.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.518+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6700018630 con 0x7f670c100db0 2024-04-22T23:44:25.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.519+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c109810 con 0x7f670c100db0 2024-04-22T23:44:25.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.521+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f670000cd10 con 0x7f670c100db0 2024-04-22T23:44:25.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.521+0000 7f66f3fff700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f66f406c4e0 0x7f66f406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.522+0000 7f670affd700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f66f406c4e0 0x7f66f406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.522+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6700014070 con 0x7f670c100db0 2024-04-22T23:44:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.522+0000 7f670affd700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f66f406c4e0 0x7f66f406e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009fd0 tx=0x7f66fc000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:25.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.529+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f670005a260 con 0x7f670c100db0 2024-04-22T23:44:25.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.969+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f670c1099a0 con 0x7f66f406c4e0 2024-04-22T23:44:25.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.970+0000 7f66f3fff700 1 -- 172.21.15.31:0/243776963 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f670c1099a0 con 0x7f66f406c4e0 2024-04-22T23:44:25.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f66f406c4e0 msgr2=0x7f66f406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:25.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f66f406c4e0 0x7f66f406e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009fd0 tx=0x7f66fc000bc0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 msgr2=0x7f670c196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 0x7f670c196a00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f670000e920 tx=0x7f670000ece0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 shutdown_connections 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f66f406c4e0 0x7f66f406e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f670c0ffbb0 0x7f670c1964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 --2- 172.21.15.31:0/243776963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f670c100db0 0x7f670c196a00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 >> 172.21.15.31:0/243776963 conn(0x7f670c0fb150 msgr2=0x7f670c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 shutdown_connections 2024-04-22T23:44:25.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:25.973+0000 7f66f1ffb700 1 -- 172.21.15.31:0/243776963 wait complete. 2024-04-22T23:44:25.980 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:44:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.088+0000 7f75179d3700 1 -- 172.21.15.31:0/2635296919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 msgr2=0x7f75100ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.088+0000 7f75179d3700 1 --2- 172.21.15.31:0/2635296919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75100ff9b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f750c009910 tx=0x7f750c009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.088+0000 7f75179d3700 1 -- 172.21.15.31:0/2635296919 shutdown_connections 2024-04-22T23:44:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.088+0000 7f75179d3700 1 --2- 172.21.15.31:0/2635296919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 0x7f7510100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.088+0000 7f75179d3700 1 --2- 172.21.15.31:0/2635296919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75100ff9b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.088+0000 7f75179d3700 1 -- 172.21.15.31:0/2635296919 >> 172.21.15.31:0/2635296919 conn(0x7f75100fb150 msgr2=0x7f75100fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.089+0000 7f75179d3700 1 -- 172.21.15.31:0/2635296919 shutdown_connections 2024-04-22T23:44:26.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.089+0000 7f75179d3700 1 -- 172.21.15.31:0/2635296919 wait complete. 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.089+0000 7f75179d3700 1 Processor -- start 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.089+0000 7f75179d3700 1 -- start start 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f75179d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75101986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f75179d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 0x7f7510198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f75179d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510199220 con 0x7f75100ff590 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f75179d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510199360 con 0x7f75100ffef0 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f751576f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75101986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f751576f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75101986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36194/0 (socket says 172.21.15.31:36194) 2024-04-22T23:44:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f751576f700 1 -- 172.21.15.31:0/9706980 learned_addr learned my addr 172.21.15.31:0/9706980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f7514f6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 0x7f7510198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f7514f6e700 1 -- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 msgr2=0x7f75101986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f7514f6e700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75101986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f7514f6e700 1 -- 172.21.15.31:0/9706980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c0095f0 con 0x7f75100ffef0 2024-04-22T23:44:26.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.090+0000 7f7514f6e700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 0x7f7510198c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f750000e970 tx=0x7f750000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:26.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.091+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f750000c9d0 con 0x7f75100ffef0 2024-04-22T23:44:26.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.091+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7500010430 con 0x7f75100ffef0 2024-04-22T23:44:26.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.091+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7500018650 con 0x7f75100ffef0 2024-04-22T23:44:26.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.091+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751019de10 con 0x7f75100ffef0 2024-04-22T23:44:26.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.092+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751019e270 con 0x7f75100ffef0 2024-04-22T23:44:26.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.093+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751010ba10 con 0x7f75100ffef0 2024-04-22T23:44:26.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.093+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f75000105a0 con 0x7f75100ffef0 2024-04-22T23:44:26.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.094+0000 7f75067fc700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f74fc06c2d0 0x7f74fc06e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.094+0000 7f751576f700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f74fc06c2d0 0x7f74fc06e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.094+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7500014070 con 0x7f75100ffef0 2024-04-22T23:44:26.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.095+0000 7f751576f700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f74fc06c2d0 0x7f74fc06e790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f750c00b760 tx=0x7f750c005940 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:26.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.103+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7500059e60 con 0x7f75100ffef0 2024-04-22T23:44:26.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:26 smithi044 ceph-mon[34463]: Upgrade: Updating mgr.smithi044.iglcti 2024-04-22T23:44:26.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:44:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:44:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:26 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:26.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.323+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75101092e0 con 0x7f74fc06c2d0 2024-04-22T23:44:26.316 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:26 smithi031 ceph-mon[26721]: Upgrade: Updating mgr.smithi044.iglcti 2024-04-22T23:44:26.316 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:26.316 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:44:26.316 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:44:26.316 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:26 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:26.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.325+0000 7f75067fc700 1 -- 172.21.15.31:0/9706980 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f75101092e0 con 0x7f74fc06c2d0 2024-04-22T23:44:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f74fc06c2d0 msgr2=0x7f74fc06e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f74fc06c2d0 0x7f74fc06e790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f750c00b760 tx=0x7f750c005940 comp rx=0 tx=0).stop 2024-04-22T23:44:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 msgr2=0x7f7510198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 0x7f7510198c00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f750000e970 tx=0x7f750000ec80 comp rx=0 tx=0).stop 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 shutdown_connections 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f74fc06c2d0 0x7f74fc06e790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75100ff590 0x7f75101986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 --2- 172.21.15.31:0/9706980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75100ffef0 0x7f7510198c00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.328+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 >> 172.21.15.31:0/9706980 conn(0x7f75100fb150 msgr2=0x7f7510107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.329+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 shutdown_connections 2024-04-22T23:44:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.329+0000 7f75179d3700 1 -- 172.21.15.31:0/9706980 wait complete. 2024-04-22T23:44:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 -- 172.21.15.31:0/3096196408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 msgr2=0x7f6c840ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/3096196408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c840ffa30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009910 tx=0x7f6c74009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 -- 172.21.15.31:0/3096196408 shutdown_connections 2024-04-22T23:44:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/3096196408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 0x7f6c841003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/3096196408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c840ffa30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 -- 172.21.15.31:0/3096196408 >> 172.21.15.31:0/3096196408 conn(0x7f6c840fb190 msgr2=0x7f6c840fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:26.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.443+0000 7f6c88ffd700 1 -- 172.21.15.31:0/3096196408 shutdown_connections 2024-04-22T23:44:26.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.444+0000 7f6c88ffd700 1 -- 172.21.15.31:0/3096196408 wait complete. 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.444+0000 7f6c88ffd700 1 Processor -- start 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.444+0000 7f6c88ffd700 1 -- start start 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c88ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c841986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c88ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 0x7f6c84198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c88ffd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84199220 con 0x7f6c840ff610 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c88ffd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c84199360 con 0x7f6c840fff70 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c81d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 0x7f6c84198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c81d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 0x7f6c84198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:35552/0 (socket says 172.21.15.31:35552) 2024-04-22T23:44:26.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c81d9b700 1 -- 172.21.15.31:0/2828021725 learned_addr learned my addr 172.21.15.31:0/2828021725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c8259c700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c841986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c8259c700 1 -- 172.21.15.31:0/2828021725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 msgr2=0x7f6c84198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c8259c700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 0x7f6c84198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.445+0000 7f6c8259c700 1 -- 172.21.15.31:0/2828021725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c740095f0 con 0x7f6c840ff610 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.446+0000 7f6c8259c700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c841986c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c7400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.446+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7401d070 con 0x7f6c840ff610 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.446+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8419ddb0 con 0x7f6c840ff610 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.446+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c74005dc0 con 0x7f6c840ff610 2024-04-22T23:44:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.446+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c74017400 con 0x7f6c840ff610 2024-04-22T23:44:26.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.446+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8419e240 con 0x7f6c840ff610 2024-04-22T23:44:26.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.448+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8410b9f0 con 0x7f6c840ff610 2024-04-22T23:44:26.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.448+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f6c74003810 con 0x7f6c840ff610 2024-04-22T23:44:26.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.448+0000 7f6c737fe700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f6c6c06c4e0 0x7f6c6c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.449+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6c7408cb70 con 0x7f6c840ff610 2024-04-22T23:44:26.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.449+0000 7f6c81d9b700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f6c6c06c4e0 0x7f6c6c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.449+0000 7f6c81d9b700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f6c6c06c4e0 0x7f6c6c06e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009d20 tx=0x7f6c78009260 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:26.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.457+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6c7405ae00 con 0x7f6c840ff610 2024-04-22T23:44:26.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.673+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c84109360 con 0x7f6c6c06c4e0 2024-04-22T23:44:26.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.693+0000 7f6c737fe700 1 -- 172.21.15.31:0/2828021725 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f6c84109360 con 0x7f6c6c06c4e0 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 91s ago 7m 25.6M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (7m) 91s ago 7m 7440k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (98s) 95s ago 6m 7105k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 91s ago 7m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (6m) 95s ago 6m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 91s ago 7m 87.7M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (114s) 91s ago 114s 11.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (108s) 91s ago 107s 16.8M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (111s) 95s ago 111s 17.6M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (105s) 95s ago 105s 15.2M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:9283,8765,8443 running (8m) 91s ago 8m 489M - 18.2.1 5be31c24972a dcdbc372f09c 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (6m) 95s ago 6m 445M - 18.2.1 5be31c24972a b6f8e3606e77 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 91s ago 8m 46.8M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:44:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (6m) 95s ago 6m 34.8M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 91s ago 7m 16.6M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 95s ago 6m 16.2M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 91s ago 5m 47.4M 3948M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 91s ago 5m 47.4M 3948M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 91s ago 4m 46.6M 3948M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (4m) 95s ago 4m 46.1M 4972M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (3m) 95s ago 3m 46.0M 4972M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (3m) 95s ago 3m 46.7M 4972M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:44:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 91s ago 6m 58.0M - 2.43.0 a07b618ecd1d b125747edf65 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f6c6c06c4e0 msgr2=0x7f6c6c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f6c6c06c4e0 0x7f6c6c06e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009d20 tx=0x7f6c78009260 comp rx=0 tx=0).stop 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 msgr2=0x7f6c841986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c841986c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c7400f690 comp rx=0 tx=0).stop 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 shutdown_connections 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7f6c6c06c4e0 0x7f6c6c06e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c840ff610 0x7f6c841986c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 --2- 172.21.15.31:0/2828021725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c840fff70 0x7f6c84198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 >> 172.21.15.31:0/2828021725 conn(0x7f6c840fb190 msgr2=0x7f6c84107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.695+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 shutdown_connections 2024-04-22T23:44:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.696+0000 7f6c88ffd700 1 -- 172.21.15.31:0/2828021725 wait complete. 2024-04-22T23:44:26.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.804+0000 7ff7c9e55700 1 -- 172.21.15.31:0/3129359851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 msgr2=0x7ff7c4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.804+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/3129359851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 0x7ff7c4100030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009910 tx=0x7ff7b4009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.804+0000 7ff7c9e55700 1 -- 172.21.15.31:0/3129359851 shutdown_connections 2024-04-22T23:44:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.804+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/3129359851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.804+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/3129359851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 0x7ff7c4100030 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.804+0000 7ff7c9e55700 1 -- 172.21.15.31:0/3129359851 >> 172.21.15.31:0/3129359851 conn(0x7ff7c40fb190 msgr2=0x7ff7c40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:26.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.805+0000 7ff7c9e55700 1 -- 172.21.15.31:0/3129359851 shutdown_connections 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.805+0000 7ff7c9e55700 1 -- 172.21.15.31:0/3129359851 wait complete. 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.805+0000 7ff7c9e55700 1 Processor -- start 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c9e55700 1 -- start start 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c9e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 0x7ff7c41953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c9e55700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c9e55700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4195f30 con 0x7ff7c40ffc10 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c9e55700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4196070 con 0x7ff7c4100e10 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c2ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:35556/0 (socket says 172.21.15.31:35556) 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c2ffd700 1 -- 172.21.15.31:0/311334333 learned_addr learned my addr 172.21.15.31:0/311334333 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c37fe700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 0x7ff7c41953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c2ffd700 1 -- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 msgr2=0x7ff7c41953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c2ffd700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 0x7ff7c41953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.806+0000 7ff7c2ffd700 1 -- 172.21.15.31:0/311334333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40095f0 con 0x7ff7c4100e10 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.807+0000 7ff7c2ffd700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4195910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800e920 tx=0x7ff7b800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.807+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b80139e0 con 0x7ff7c4100e10 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.807+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7b800c7c0 con 0x7ff7c4100e10 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.807+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c40685e0 con 0x7ff7c4100e10 2024-04-22T23:44:26.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.808+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b801b9a0 con 0x7ff7c4100e10 2024-04-22T23:44:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.808+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c4068b30 con 0x7ff7c4100e10 2024-04-22T23:44:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.809+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff7b8013b40 con 0x7ff7c4100e10 2024-04-22T23:44:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.809+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4108700 con 0x7ff7c4100e10 2024-04-22T23:44:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.810+0000 7ff7c0ff9700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ff7ac06c2d0 0x7ff7ac06e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.810+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff7b808ff30 con 0x7ff7c4100e10 2024-04-22T23:44:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.810+0000 7ff7c37fe700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ff7ac06c2d0 0x7ff7ac06e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:26.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.810+0000 7ff7c37fe700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ff7ac06c2d0 0x7ff7ac06e790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b6f0 tx=0x7ff7b4005940 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:26.818+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff7b805e0a0 con 0x7ff7c4100e10 2024-04-22T23:44:27.003 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:27 smithi044 ceph-mon[34463]: Deploying daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:44:27.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:27 smithi044 ceph-mon[34463]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:27.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:27 smithi044 ceph-mon[34463]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:27.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:27 smithi044 ceph-mon[34463]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:27.004 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:27 smithi044 ceph-mon[34463]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:27.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.142+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff7c419a9d0 con 0x7ff7c4100e10 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.143+0000 7ff7c0ff9700 1 -- 172.21.15.31:0/311334333 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7ff7b805dc30 con 0x7ff7c4100e10 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:44:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:44:27.136 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:44:27.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ff7ac06c2d0 msgr2=0x7ff7ac06e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:27.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ff7ac06c2d0 0x7ff7ac06e790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400b6f0 tx=0x7ff7b4005940 comp rx=0 tx=0).stop 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 msgr2=0x7ff7c4195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4195910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800e920 tx=0x7ff7b800ec30 comp rx=0 tx=0).stop 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 shutdown_connections 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7ff7ac06c2d0 0x7ff7ac06e790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c40ffc10 0x7ff7c41953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 --2- 172.21.15.31:0/311334333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4100e10 0x7ff7c4195910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.145+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 >> 172.21.15.31:0/311334333 conn(0x7ff7c40fb190 msgr2=0x7ff7c4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.146+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 shutdown_connections 2024-04-22T23:44:27.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.146+0000 7ff7c9e55700 1 -- 172.21.15.31:0/311334333 wait complete. 2024-04-22T23:44:27.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 -- 172.21.15.31:0/2128600965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41017a0 msgr2=0x7fdce4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:27.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/2128600965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41017a0 0x7fdce4103b90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009910 tx=0x7fdcd4009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 -- 172.21.15.31:0/2128600965 shutdown_connections 2024-04-22T23:44:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/2128600965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41040d0 0x7fdce41064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/2128600965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41017a0 0x7fdce4103b90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 -- 172.21.15.31:0/2128600965 >> 172.21.15.31:0/2128600965 conn(0x7fdce40fb190 msgr2=0x7fdce40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 -- 172.21.15.31:0/2128600965 shutdown_connections 2024-04-22T23:44:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.258+0000 7fdce8c8a700 1 -- 172.21.15.31:0/2128600965 wait complete. 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.259+0000 7fdce8c8a700 1 Processor -- start 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.259+0000 7fdce8c8a700 1 -- start start 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.259+0000 7fdce8c8a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41017a0 0x7fdce4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.259+0000 7fdce8c8a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41040d0 0x7fdce41969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.259+0000 7fdce8c8a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce4196fc0 con 0x7fdce41040d0 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.259+0000 7fdce8c8a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce4197100 con 0x7fdce41017a0 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.260+0000 7fdce1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41040d0 0x7fdce41969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.260+0000 7fdce1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41040d0 0x7fdce41969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36266/0 (socket says 172.21.15.31:36266) 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.260+0000 7fdce1d9b700 1 -- 172.21.15.31:0/1014198557 learned_addr learned my addr 172.21.15.31:0/1014198557 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.260+0000 7fdce259c700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41017a0 0x7fdce4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:27.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:27 smithi031 ceph-mon[26721]: Deploying daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:44:27.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:27 smithi031 ceph-mon[26721]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:27.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:27 smithi031 ceph-mon[26721]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:27.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:27 smithi031 ceph-mon[26721]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:27.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:27 smithi031 ceph-mon[26721]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.261+0000 7fdce259c700 1 -- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41040d0 msgr2=0x7fdce41969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.261+0000 7fdce259c700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41040d0 0x7fdce41969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.261+0000 7fdce259c700 1 -- 172.21.15.31:0/1014198557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40095f0 con 0x7fdce41017a0 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.261+0000 7fdce259c700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41017a0 0x7fdce4196460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009910 tx=0x7fdcd400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.262+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd401d070 con 0x7fdce41017a0 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.262+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce40ff590 con 0x7fdce41017a0 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.262+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce40ffa80 con 0x7fdce41017a0 2024-04-22T23:44:27.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.262+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcd400f460 con 0x7fdce41017a0 2024-04-22T23:44:27.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.262+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd4021620 con 0x7fdce41017a0 2024-04-22T23:44:27.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.263+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce4190610 con 0x7fdce41017a0 2024-04-22T23:44:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.264+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fdcd4003a60 con 0x7fdce41017a0 2024-04-22T23:44:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.265+0000 7fdcd37fe700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fdccc06c4e0 0x7fdccc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.265+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdcd408cad0 con 0x7fdce41017a0 2024-04-22T23:44:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.266+0000 7fdce1d9b700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fdccc06c4e0 0x7fdccc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.267+0000 7fdce1d9b700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fdccc06c4e0 0x7fdccc06e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005950 tx=0x7fdcd800a400 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:27.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.273+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdcd405acc0 con 0x7fdce41017a0 2024-04-22T23:44:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.491+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 --> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdce4100ee0 con 0x7fdccc06c4e0 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.492+0000 7fdcd37fe700 1 -- 172.21.15.31:0/1014198557 <== mgr.14211 v2:172.21.15.31:6800/2402370856 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fdce4100ee0 con 0x7fdccc06c4e0 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "0/2 daemons upgraded", 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:44:27.485 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:44:27.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fdccc06c4e0 msgr2=0x7fdccc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:27.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fdccc06c4e0 0x7fdccc06e9a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8005950 tx=0x7fdcd800a400 comp rx=0 tx=0).stop 2024-04-22T23:44:27.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41017a0 msgr2=0x7fdce4196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41017a0 0x7fdce4196460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009910 tx=0x7fdcd400bd70 comp rx=0 tx=0).stop 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 shutdown_connections 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:6800/2402370856,v1:172.21.15.31:6801/2402370856] conn(0x7fdccc06c4e0 0x7fdccc06e9a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdce41017a0 0x7fdce4196460 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4009910 tx=0x7fdcd400bd70 comp rx=0 tx=0).stop 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 --2- 172.21.15.31:0/1014198557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdce41040d0 0x7fdce41969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 >> 172.21.15.31:0/1014198557 conn(0x7fdce40fb190 msgr2=0x7fdce40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 shutdown_connections 2024-04-22T23:44:27.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:27.495+0000 7fdce8c8a700 1 -- 172.21.15.31:0/1014198557 wait complete. 2024-04-22T23:44:28.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:28 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/311334333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:28.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:28 smithi044 ceph-mon[34463]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:28.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:28 smithi044 ceph-mon[34463]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:28.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:28 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/311334333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:28.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:28 smithi031 ceph-mon[26721]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:28.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:28 smithi031 ceph-mon[26721]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:30.348 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:30.348 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:30.348 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:30.348 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:30 smithi044 ceph-mon[34463]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:30.348 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:30 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:30.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:30.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:30.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:30.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:30 smithi031 ceph-mon[26721]: pgmap v237: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:30.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:30 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:32.843 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:32 smithi044 ceph-mon[34463]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:32.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:32 smithi031 ceph-mon[26721]: pgmap v238: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:34.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:34 smithi044 ceph-mon[34463]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:34.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:34 smithi031 ceph-mon[26721]: pgmap v239: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:36.053 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 93% (11467/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to '43be020184947e53516056c9931e1ac5bdbbb1a5'. 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T23:44:36.376 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-22T23:44:36.377 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at 43be0201849 Merge branch 'host-drain-test-ignore-stray-host' of https://github.com/adk3798/ceph into wip-adk-testing-2024-04-22-1618 2024-04-22T23:44:36.382 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T23:44:36.412 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T23:44:36.414 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T23:44:36.414 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T23:44:37.029 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T23:44:37.085 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T23:44:37.125 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T23:44:37.127 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T23:44:37.127 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T23:44:37.167 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T23:44:37.170 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:44:37.171 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T23:44:37.174 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:36 smithi044 ceph-mon[34463]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:36 smithi031 ceph-mon[26721]: pgmap v240: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:44:37.210 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-22T23:44:37.212 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-22T23:44:37.212 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43be020184947e53516056c9931e1ac5bdbbb1a5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-22T23:44:37.277 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p fsstress 2024-04-22T23:44:37.280 INFO:tasks.workunit.client.0.smithi031.stderr:+ pushd fsstress 2024-04-22T23:44:37.282 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:44:37.282 INFO:tasks.workunit.client.0.smithi031.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-22T23:44:38.073 INFO:tasks.workunit.client.0.smithi031.stderr:+ tar xzf ltp-full.tgz 2024-04-22T23:44:38.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:37 smithi031 ceph-mon[26721]: Standby manager daemon smithi044.iglcti restarted 2024-04-22T23:44:38.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:37 smithi031 ceph-mon[26721]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:44:38.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:37 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:44:38.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:37 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:44:38.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:37 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:44:38.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:37 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:44:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:37 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti restarted 2024-04-22T23:44:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:37 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:44:38.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:37 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:44:38.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:37 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:44:38.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:37 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:44:38.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:37 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:44:39.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:38 smithi031 ceph-mon[26721]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:39.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:38 smithi031 ceph-mon[26721]: mgrmap e20: smithi031.fazdcv(active, since 6m), standbys: smithi044.iglcti 2024-04-22T23:44:39.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:38 smithi044 ceph-mon[34463]: pgmap v241: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:44:39.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:38 smithi044 ceph-mon[34463]: mgrmap e20: smithi031.fazdcv(active, since 6m), standbys: smithi044.iglcti 2024-04-22T23:44:40.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:40.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:40 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:40.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:40 smithi031 ceph-mon[26721]: pgmap v242: 65 pgs: 65 active+clean; 655 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 6.0 KiB/s wr, 3 op/s 2024-04-22T23:44:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:40 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:40.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:40 smithi044 ceph-mon[34463]: pgmap v242: 65 pgs: 65 active+clean; 655 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 6.0 KiB/s wr, 3 op/s 2024-04-22T23:44:41.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:41.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:41 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:41.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:41.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:41 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:42.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:42 smithi044 ceph-mon[34463]: pgmap v243: 65 pgs: 65 active+clean; 9.4 MiB data, 183 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 759 KiB/s wr, 14 op/s 2024-04-22T23:44:42.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:42 smithi031 ceph-mon[26721]: pgmap v243: 65 pgs: 65 active+clean; 9.4 MiB data, 183 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 759 KiB/s wr, 14 op/s 2024-04-22T23:44:44.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:44 smithi031 ceph-mon[26721]: pgmap v244: 65 pgs: 65 active+clean; 14 MiB data, 202 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 30 op/s 2024-04-22T23:44:44.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:44 smithi044 ceph-mon[34463]: pgmap v244: 65 pgs: 65 active+clean; 14 MiB data, 202 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s rd, 1.1 MiB/s wr, 30 op/s 2024-04-22T23:44:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:45 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:45 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:46.554 INFO:tasks.workunit.client.1.smithi044.stderr:Updating files: 79% (9779/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T23:44:47.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:46 smithi031 ceph-mon[26721]: pgmap v245: 65 pgs: 65 active+clean; 15 MiB data, 214 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-04-22T23:44:47.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:46 smithi044 ceph-mon[34463]: pgmap v245: 65 pgs: 65 active+clean; 15 MiB data, 214 MiB used, 536 GiB / 536 GiB avail; 112 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr:Note: switching to '43be020184947e53516056c9931e1ac5bdbbb1a5'. 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T23:44:47.498 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr: git switch -c 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr:Or undo this operation with: 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr: git switch - 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr: 2024-04-22T23:44:47.499 INFO:tasks.workunit.client.1.smithi044.stderr:HEAD is now at 43be0201849 Merge branch 'host-drain-test-ignore-stray-host' of https://github.com/adk3798/ceph into wip-adk-testing-2024-04-22-1618 2024-04-22T23:44:47.505 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-22T23:44:47.797 INFO:tasks.workunit.client.1.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T23:44:47.799 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-22T23:44:47.800 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T23:44:49.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:48 smithi031 ceph-mon[26721]: pgmap v246: 65 pgs: 65 active+clean; 18 MiB data, 218 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 61 op/s 2024-04-22T23:44:49.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:48 smithi044 ceph-mon[34463]: pgmap v246: 65 pgs: 65 active+clean; 18 MiB data, 218 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 61 op/s 2024-04-22T23:44:51.130 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T23:44:51.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:50 smithi031 ceph-mon[26721]: pgmap v247: 65 pgs: 65 active+clean; 23 MiB data, 230 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 1.9 MiB/s wr, 79 op/s 2024-04-22T23:44:51.206 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T23:44:51.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:50 smithi044 ceph-mon[34463]: pgmap v247: 65 pgs: 65 active+clean; 23 MiB data, 230 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 1.9 MiB/s wr, 79 op/s 2024-04-22T23:44:51.267 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-22T23:44:51.269 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-22T23:44:51.269 INFO:tasks.workunit.client.1.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T23:44:51.333 INFO:tasks.workunit.client.1.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-22T23:44:51.338 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T23:44:51.338 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-22T23:44:51.359 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-22T23:44:51.360 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-22T23:44:51.360 DEBUG:teuthology.orchestra.run.smithi044:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43be020184947e53516056c9931e1ac5bdbbb1a5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-22T23:44:51.433 INFO:tasks.workunit.client.1.smithi044.stderr:+ mkdir -p fsstress 2024-04-22T23:44:51.436 INFO:tasks.workunit.client.1.smithi044.stderr:+ pushd fsstress 2024-04-22T23:44:51.438 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:44:51.438 INFO:tasks.workunit.client.1.smithi044.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-22T23:44:52.125 INFO:tasks.workunit.client.1.smithi044.stderr:+ tar xzf ltp-full.tgz 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: pgmap v248: 65 pgs: 65 active+clean; 27 MiB data, 247 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:52.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr fail", "who": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: osdmap e47: 6 total, 6 up, 6 in 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "mgr fail", "who": "smithi031.fazdcv"}]': finished 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: mgrmap e21: smithi044.iglcti(active, starting, since 0.00647901s) 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:44:52.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: Manager daemon smithi044.iglcti is now available 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/trash_purge_schedule"}]: dispatch 2024-04-22T23:44:52.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:52 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/trash_purge_schedule"}]: dispatch 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: pgmap v248: 65 pgs: 65 active+clean; 27 MiB data, 247 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 2.2 MiB/s wr, 98 op/s 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr fail", "who": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: osdmap e47: 6 total, 6 up, 6 in 2024-04-22T23:44:53.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.14211 172.21.15.31:0/4080648398' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "mgr fail", "who": "smithi031.fazdcv"}]': finished 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: mgrmap e21: smithi044.iglcti(active, starting, since 0.00647901s) 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:44:53.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: Manager daemon smithi044.iglcti is now available 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/trash_purge_schedule"}]: dispatch 2024-04-22T23:44:53.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:52 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi044.iglcti/trash_purge_schedule"}]: dispatch 2024-04-22T23:44:54.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: Deploying cephadm binary to smithi031 2024-04-22T23:44:54.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: Deploying cephadm binary to smithi044 2024-04-22T23:44:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: mgrmap e22: smithi044.iglcti(active, since 1.00908s) 2024-04-22T23:44:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: pgmap v3: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: [22/Apr/2024:23:44:53] ENGINE Bus STARTING 2024-04-22T23:44:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: [22/Apr/2024:23:44:53] ENGINE Serving on http://172.21.15.44:8765 2024-04-22T23:44:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: [22/Apr/2024:23:44:54] ENGINE Serving on https://172.21.15.44:7150 2024-04-22T23:44:54.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:54 smithi031 ceph-mon[26721]: [22/Apr/2024:23:44:54] ENGINE Bus STARTED 2024-04-22T23:44:54.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: Deploying cephadm binary to smithi031 2024-04-22T23:44:54.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: Deploying cephadm binary to smithi044 2024-04-22T23:44:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: mgrmap e22: smithi044.iglcti(active, since 1.00908s) 2024-04-22T23:44:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: pgmap v3: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: [22/Apr/2024:23:44:53] ENGINE Bus STARTING 2024-04-22T23:44:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: [22/Apr/2024:23:44:53] ENGINE Serving on http://172.21.15.44:8765 2024-04-22T23:44:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: [22/Apr/2024:23:44:54] ENGINE Serving on https://172.21.15.44:7150 2024-04-22T23:44:54.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:54 smithi044 ceph-mon[34463]: [22/Apr/2024:23:44:54] ENGINE Bus STARTED 2024-04-22T23:44:56.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:55 smithi031 ceph-mon[26721]: pgmap v4: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:55 smithi044 ceph-mon[34463]: pgmap v4: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:57.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:56 smithi031 ceph-mon[26721]: mgrmap e23: smithi044.iglcti(active, since 3s) 2024-04-22T23:44:57.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:56 smithi044 ceph-mon[34463]: mgrmap e23: smithi044.iglcti(active, since 3s) 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 -- 172.21.15.31:0/2539722382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 msgr2=0x7f6744106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 --2- 172.21.15.31:0/2539722382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 0x7f6744106520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6738009910 tx=0x7f6738009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 -- 172.21.15.31:0/2539722382 shutdown_connections 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 --2- 172.21.15.31:0/2539722382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 0x7f6744106520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 --2- 172.21.15.31:0/2539722382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 -- 172.21.15.31:0/2539722382 >> 172.21.15.31:0/2539722382 conn(0x7f67440fb190 msgr2=0x7f67440fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 -- 172.21.15.31:0/2539722382 shutdown_connections 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.612+0000 7f67489c0700 1 -- 172.21.15.31:0/2539722382 wait complete. 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f67489c0700 1 Processor -- start 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f67489c0700 1 -- start start 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f67489c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f67489c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 0x7f6744196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f67489c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744196fb0 con 0x7f6744104130 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f67489c0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67441970f0 con 0x7f6744101800 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f674259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f6741d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 0x7f6744196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f674259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:57156/0 (socket says 172.21.15.31:57156) 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.613+0000 7f674259c700 1 -- 172.21.15.31:0/1009814015 learned_addr learned my addr 172.21.15.31:0/1009814015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f674259c700 1 -- 172.21.15.31:0/1009814015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 msgr2=0x7f6744196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f674259c700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 0x7f6744196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f674259c700 1 -- 172.21.15.31:0/1009814015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6734009520 con 0x7f6744101800 2024-04-22T23:44:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f674259c700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744196450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f673400eab0 tx=0x7f673400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f673400cba0 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67380095f0 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f67340044d0 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6734005020 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.614+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674419bf60 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.615+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 46697+0+0 (secure 0 0 0) 0x7f67340052f0 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.616+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6744190610 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.616+0000 7f67337fe700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f672c0398c0 0x7f672c03bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.616+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6734014070 con 0x7f6744101800 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.616+0000 7f6741d9b700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f672c0398c0 0x7f672c03bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:57.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.617+0000 7f6741d9b700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f672c0398c0 0x7f672c03bd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6738009fd0 tx=0x7f6738005fd0 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.626+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f673401f7b0 con 0x7f6744101800 2024-04-22T23:44:57.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.906+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67440611d0 con 0x7f672c0398c0 2024-04-22T23:44:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.919+0000 7f67337fe700 1 -- 172.21.15.31:0/1009814015 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f67440611d0 con 0x7f672c0398c0 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f672c0398c0 msgr2=0x7f672c03bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f672c0398c0 0x7f672c03bd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6738009fd0 tx=0x7f6738005fd0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 msgr2=0x7f6744196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744196450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f673400eab0 tx=0x7f673400edc0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 shutdown_connections 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f672c0398c0 0x7f672c03bd80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6744101800 0x7f6744196450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 --2- 172.21.15.31:0/1009814015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6744104130 0x7f6744196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.922+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 >> 172.21.15.31:0/1009814015 conn(0x7f67440fb190 msgr2=0x7f67440fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:57.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.923+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 shutdown_connections 2024-04-22T23:44:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:57.923+0000 7f67489c0700 1 -- 172.21.15.31:0/1009814015 wait complete. 2024-04-22T23:44:57.931 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:44:58.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/2195581738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 msgr2=0x7f7e28101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/2195581738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 0x7f7e28101640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009910 tx=0x7f7e18009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:58.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/2195581738 shutdown_connections 2024-04-22T23:44:58.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/2195581738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e28100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/2195581738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 0x7f7e28101640 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/2195581738 >> 172.21.15.31:0/2195581738 conn(0x7f7e280fa8c0 msgr2=0x7f7e280fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:58.043 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:57 smithi031 ceph-mon[26721]: pgmap v5: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:58.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/2195581738 shutdown_connections 2024-04-22T23:44:58.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.050+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/2195581738 wait complete. 2024-04-22T23:44:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e2d7eb700 1 Processor -- start 2024-04-22T23:44:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e2d7eb700 1 -- start start 2024-04-22T23:44:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e2d7eb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e281953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e2d7eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 0x7f7e28195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e2d7eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28195f20 con 0x7f7e28101220 2024-04-22T23:44:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e2d7eb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28196060 con 0x7f7e280ffdf0 2024-04-22T23:44:58.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e26ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e281953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e26ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e281953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:57170/0 (socket says 172.21.15.31:57170) 2024-04-22T23:44:58.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.051+0000 7f7e26ffd700 1 -- 172.21.15.31:0/295612290 learned_addr learned my addr 172.21.15.31:0/295612290 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:58.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.052+0000 7f7e267fc700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 0x7f7e28195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.052+0000 7f7e26ffd700 1 -- 172.21.15.31:0/295612290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 msgr2=0x7f7e28195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.052+0000 7f7e26ffd700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 0x7f7e28195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.052+0000 7f7e26ffd700 1 -- 172.21.15.31:0/295612290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e180095f0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.053+0000 7f7e26ffd700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e281953c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000c00 tx=0x7f7e1800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.053+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1801d070 con 0x7f7e280ffdf0 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.053+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2819aab0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.053+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2819afa0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.053+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7e18005dc0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.053+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e18022640 con 0x7f7e280ffdf0 2024-04-22T23:44:58.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.054+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 46697+0+0 (secure 0 0 0) 0x7f7e180227a0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.054+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e14005320 con 0x7f7e280ffdf0 2024-04-22T23:44:58.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.055+0000 7f7e0ffff700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7e100398c0 0x7f7e1003bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.055+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7e1804f0d0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.055+0000 7f7e267fc700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7e100398c0 0x7f7e1003bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.055+0000 7f7e267fc700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7e100398c0 0x7f7e1003bd80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009c60 tx=0x7f7e1c009260 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:58.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.060+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f7e180213d0 con 0x7f7e280ffdf0 2024-04-22T23:44:58.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:57 smithi044 ceph-mon[34463]: pgmap v5: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.318+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e14000bf0 con 0x7f7e100398c0 2024-04-22T23:44:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.321+0000 7f7e0ffff700 1 -- 172.21.15.31:0/295612290 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7e14000bf0 con 0x7f7e100398c0 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7e100398c0 msgr2=0x7f7e1003bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7e100398c0 0x7f7e1003bd80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009c60 tx=0x7f7e1c009260 comp rx=0 tx=0).stop 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 msgr2=0x7f7e281953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e281953c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000c00 tx=0x7f7e1800f690 comp rx=0 tx=0).stop 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 shutdown_connections 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7e100398c0 0x7f7e1003bd80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e280ffdf0 0x7f7e281953c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 --2- 172.21.15.31:0/295612290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e28101220 0x7f7e28195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 >> 172.21.15.31:0/295612290 conn(0x7f7e280fa8c0 msgr2=0x7f7e28104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 shutdown_connections 2024-04-22T23:44:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.323+0000 7f7e2d7eb700 1 -- 172.21.15.31:0/295612290 wait complete. 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.447+0000 7f7b9baa9700 1 -- 172.21.15.31:0/3303375218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b940ffba0 msgr2=0x7f7b940fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.447+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/3303375218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b940ffba0 0x7f7b940fffc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000fa70 tx=0x7f7b9000fd80 comp rx=0 tx=0).stop 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.447+0000 7f7b9baa9700 1 -- 172.21.15.31:0/3303375218 shutdown_connections 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.447+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/3303375218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b94100da0 0x7f7b94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.447+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/3303375218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b940ffba0 0x7f7b940fffc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.447+0000 7f7b9baa9700 1 -- 172.21.15.31:0/3303375218 >> 172.21.15.31:0/3303375218 conn(0x7f7b940fb120 msgr2=0x7f7b940fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.448+0000 7f7b9baa9700 1 -- 172.21.15.31:0/3303375218 shutdown_connections 2024-04-22T23:44:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.448+0000 7f7b9baa9700 1 -- 172.21.15.31:0/3303375218 wait complete. 2024-04-22T23:44:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b9baa9700 1 Processor -- start 2024-04-22T23:44:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b9baa9700 1 -- start start 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b9baa9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b940ffba0 0x7f7b941a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b9baa9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 0x7f7b941a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b9baa9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b941a5c10 con 0x7f7b94100da0 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b9baa9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b941a5d50 con 0x7f7b940ffba0 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b99044700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 0x7f7b941a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b99044700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 0x7f7b941a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47158/0 (socket says 172.21.15.31:47158) 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b99044700 1 -- 172.21.15.31:0/750553307 learned_addr learned my addr 172.21.15.31:0/750553307 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.449+0000 7f7b99845700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b940ffba0 0x7f7b941a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b99845700 1 -- 172.21.15.31:0/750553307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 msgr2=0x7f7b941a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b99845700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 0x7f7b941a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b99845700 1 -- 172.21.15.31:0/750553307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9000f750 con 0x7f7b940ffba0 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b99044700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 0x7f7b941a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b99845700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b940ffba0 0x7f7b941a50b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000fa70 tx=0x7f7b9000f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b900052d0 con 0x7f7b940ffba0 2024-04-22T23:44:58.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7b90017430 con 0x7f7b940ffba0 2024-04-22T23:44:58.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b941aa7a0 con 0x7f7b940ffba0 2024-04-22T23:44:58.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b9000f5a0 con 0x7f7b940ffba0 2024-04-22T23:44:58.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.450+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b941aacf0 con 0x7f7b940ffba0 2024-04-22T23:44:58.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.452+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) v1 ==== 46697+0+0 (secure 0 0 0) 0x7f7b90003a60 con 0x7f7b940ffba0 2024-04-22T23:44:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.452+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9406ef70 con 0x7f7b940ffba0 2024-04-22T23:44:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.452+0000 7f7b8affd700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7b800398c0 0x7f7b8003bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.452+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7b9004ecd0 con 0x7f7b940ffba0 2024-04-22T23:44:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.453+0000 7f7b99044700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7b800398c0 0x7f7b8003bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.453+0000 7f7b99044700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7b800398c0 0x7f7b8003bd80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b84011550 tx=0x7f7b84009260 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:58.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.461+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f7b90025070 con 0x7f7b940ffba0 2024-04-22T23:44:58.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.825+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b940611d0 con 0x7f7b800398c0 2024-04-22T23:44:58.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.840+0000 7f7b8affd700 1 -- 172.21.15.31:0/750553307 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7b940611d0 con 0x7f7b800398c0 2024-04-22T23:44:58.832 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 2m ago 7m 25.6M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (8m) 2m ago 8m 7440k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (2m) 19s ago 7m 7478k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 2m ago 8m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (7m) 19s ago 6m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 2m ago 7m 87.7M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (2m) 2m ago 2m 11.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (2m) 2m ago 2m 16.8M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (2m) 19s ago 2m 24.8M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (2m) 19s ago 2m 16.3M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:9283,8765,8443 running (9m) 2m ago 9m 489M - 18.2.1 5be31c24972a dcdbc372f09c 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (30s) 19s ago 6m 48.0M - 19.0.0-3252-g43be0201 a0434699c1d7 05ec5942bb78 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 2m ago 9m 46.8M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:44:58.833 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (6m) 19s ago 6m 35.7M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 2m ago 7m 16.6M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 19s ago 6m 16.6M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 2m ago 6m 47.4M 3948M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 2m ago 5m 47.4M 3948M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 2m ago 5m 46.6M 3948M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (4m) 19s ago 4m 47.1M 4972M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (4m) 19s ago 4m 47.4M 4972M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (3m) 19s ago 3m 47.0M 4972M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:44:58.834 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 2m ago 7m 58.0M - 2.43.0 a07b618ecd1d b125747edf65 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7b800398c0 msgr2=0x7f7b8003bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7b800398c0 0x7f7b8003bd80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7b84011550 tx=0x7f7b84009260 comp rx=0 tx=0).stop 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b940ffba0 msgr2=0x7f7b941a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b940ffba0 0x7f7b941a50b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000fa70 tx=0x7f7b9000f280 comp rx=0 tx=0).stop 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 shutdown_connections 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f7b800398c0 0x7f7b8003bd80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b940ffba0 0x7f7b941a50b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 --2- 172.21.15.31:0/750553307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b94100da0 0x7f7b941a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.843+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 >> 172.21.15.31:0/750553307 conn(0x7f7b940fb120 msgr2=0x7f7b940fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:58.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.844+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 shutdown_connections 2024-04-22T23:44:58.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.844+0000 7f7b9baa9700 1 -- 172.21.15.31:0/750553307 wait complete. 2024-04-22T23:44:58.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/283495555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 msgr2=0x7f9aa8100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/283495555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8100030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009910 tx=0x7f9aa4009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:58.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/283495555 shutdown_connections 2024-04-22T23:44:58.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/283495555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aa8100e10 0x7f9aa8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/283495555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8100030 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/283495555 >> 172.21.15.31:0/283495555 conn(0x7f9aa80fb190 msgr2=0x7f9aa80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:58.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/283495555 shutdown_connections 2024-04-22T23:44:58.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.976+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/283495555 wait complete. 2024-04-22T23:44:58.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aaf7e8700 1 Processor -- start 2024-04-22T23:44:58.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aaf7e8700 1 -- start start 2024-04-22T23:44:58.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aaf7e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aaf7e8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aa8100e10 0x7f9aa81021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aaf7e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8102730 con 0x7f9aa80ffc10 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aaf7e8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8102870 con 0x7f9aa8100e10 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aad584700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aad584700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47180/0 (socket says 172.21.15.31:47180) 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aad584700 1 -- 172.21.15.31:0/3528321717 learned_addr learned my addr 172.21.15.31:0/3528321717 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aacd83700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aa8100e10 0x7f9aa81021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aad584700 1 -- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aa8100e10 msgr2=0x7f9aa81021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aad584700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aa8100e10 0x7f9aa81021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.977+0000 7f9aad584700 1 -- 172.21.15.31:0/3528321717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40095f0 con 0x7f9aa80ffc10 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.978+0000 7f9aad584700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8103ba0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400fa00 tx=0x7f9aa400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.978+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aa401d070 con 0x7f9aa80ffc10 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.978+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9aa400fe90 con 0x7f9aa80ffc10 2024-04-22T23:44:58.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.978+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa8102af0 con 0x7f9aa80ffc10 2024-04-22T23:44:58.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.978+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aa4017630 con 0x7f9aa80ffc10 2024-04-22T23:44:58.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.978+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa8102fe0 con 0x7f9aa80ffc10 2024-04-22T23:44:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.979+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 46697+0+0 (secure 0 0 0) 0x7f9aa40178a0 con 0x7f9aa80ffc10 2024-04-22T23:44:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.979+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa8108720 con 0x7f9aa80ffc10 2024-04-22T23:44:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.980+0000 7f9a9e7fc700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f9a940398c0 0x7f9a9403bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.980+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f9aa404efd0 con 0x7f9aa80ffc10 2024-04-22T23:44:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.980+0000 7f9aacd83700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f9a940398c0 0x7f9a9403bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.981+0000 7f9aacd83700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f9a940398c0 0x7f9a9403bd80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009c60 tx=0x7f9a98009260 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:58.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:58.985+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f9aa4020370 con 0x7f9aa80ffc10 2024-04-22T23:44:59.068 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:58 smithi031 ceph-mon[26721]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:59.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:58 smithi044 ceph-mon[34463]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.369+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9aa819e950 con 0x7f9aa80ffc10 2024-04-22T23:44:59.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.370+0000 7f9a9e7fc700 1 -- 172.21.15.31:0/3528321717 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f9aa4003e60 con 0x7f9aa80ffc10 2024-04-22T23:44:59.362 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:44:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:44:59.364 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f9a940398c0 msgr2=0x7f9a9403bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f9a940398c0 0x7f9a9403bd80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009c60 tx=0x7f9a98009260 comp rx=0 tx=0).stop 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 msgr2=0x7f9aa8103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8103ba0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400fa00 tx=0x7f9aa400fa30 comp rx=0 tx=0).stop 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 shutdown_connections 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f9a940398c0 0x7f9a9403bd80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9aa80ffc10 0x7f9aa8103ba0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 --2- 172.21.15.31:0/3528321717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9aa8100e10 0x7f9aa81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 >> 172.21.15.31:0/3528321717 conn(0x7f9aa80fb190 msgr2=0x7f9aa8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 shutdown_connections 2024-04-22T23:44:59.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.372+0000 7f9aaf7e8700 1 -- 172.21.15.31:0/3528321717 wait complete. 2024-04-22T23:44:59.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.486+0000 7fef6671b700 1 -- 172.21.15.31:0/723060213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef600ffbc0 msgr2=0x7fef600fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:59.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.486+0000 7fef6671b700 1 --2- 172.21.15.31:0/723060213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef600ffbc0 0x7fef600fffe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef50009910 tx=0x7fef50009c20 comp rx=0 tx=0).stop 2024-04-22T23:44:59.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.486+0000 7fef6671b700 1 -- 172.21.15.31:0/723060213 shutdown_connections 2024-04-22T23:44:59.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.486+0000 7fef6671b700 1 --2- 172.21.15.31:0/723060213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef60100dc0 0x7fef60101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.486+0000 7fef6671b700 1 --2- 172.21.15.31:0/723060213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef600ffbc0 0x7fef600fffe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.486+0000 7fef6671b700 1 -- 172.21.15.31:0/723060213 >> 172.21.15.31:0/723060213 conn(0x7fef600fb160 msgr2=0x7fef600fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.487+0000 7fef6671b700 1 -- 172.21.15.31:0/723060213 shutdown_connections 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.487+0000 7fef6671b700 1 -- 172.21.15.31:0/723060213 wait complete. 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.487+0000 7fef6671b700 1 Processor -- start 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef6671b700 1 -- start start 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef6671b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 0x7fef601967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef6671b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 0x7fef60196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:59.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef6671b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60197350 con 0x7fef60100dc0 2024-04-22T23:44:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef6671b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef60197490 con 0x7fef600ffbc0 2024-04-22T23:44:59.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 0x7fef60196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 0x7fef601967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5ffff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 0x7fef601967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:57220/0 (socket says 172.21.15.31:57220) 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5ffff700 1 -- 172.21.15.31:0/831946262 learned_addr learned my addr 172.21.15.31:0/831946262 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5f7fe700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 0x7fef60196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47198/0 (socket says 172.21.15.31:47198) 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5f7fe700 1 -- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 msgr2=0x7fef601967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5f7fe700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 0x7fef601967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5f7fe700 1 -- 172.21.15.31:0/831946262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef500095f0 con 0x7fef60100dc0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.488+0000 7fef5ffff700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 0x7fef601967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.489+0000 7fef5f7fe700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 0x7fef60196d30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef5400e970 tx=0x7fef5400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.489+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef5400c9d0 con 0x7fef60100dc0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.489+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef54010430 con 0x7fef60100dc0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.489+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6019bf40 con 0x7fef60100dc0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.489+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef54018650 con 0x7fef60100dc0 2024-04-22T23:44:59.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.489+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6019c460 con 0x7fef60100dc0 2024-04-22T23:44:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.490+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 46697+0+0 (secure 0 0 0) 0x7fef54018910 con 0x7fef60100dc0 2024-04-22T23:44:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.490+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef60068af0 con 0x7fef60100dc0 2024-04-22T23:44:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.490+0000 7fef5d7fa700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fef480398c0 0x7fef4803bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.490+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fef54014070 con 0x7fef60100dc0 2024-04-22T23:44:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.491+0000 7fef5ffff700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fef480398c0 0x7fef4803bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:59.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.491+0000 7fef5ffff700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fef480398c0 0x7fef4803bd80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fef50009fd0 tx=0x7fef50005940 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.500+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fef5401fc30 con 0x7fef60100dc0 2024-04-22T23:44:59.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.829+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef600695e0 con 0x7fef480398c0 2024-04-22T23:44:59.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.840+0000 7fef5d7fa700 1 -- 172.21.15.31:0/831946262 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fef600695e0 con 0x7fef480398c0 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/2 daemons upgraded", 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:44:59.833 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.842+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fef480398c0 msgr2=0x7fef4803bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.842+0000 7fef6671b700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fef480398c0 0x7fef4803bd80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fef50009fd0 tx=0x7fef50005940 comp rx=0 tx=0).stop 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.842+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 msgr2=0x7fef60196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.842+0000 7fef6671b700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 0x7fef60196d30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef5400e970 tx=0x7fef5400ec80 comp rx=0 tx=0).stop 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 shutdown_connections 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fef480398c0 0x7fef4803bd80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef600ffbc0 0x7fef601967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 --2- 172.21.15.31:0/831946262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60100dc0 0x7fef60196d30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 >> 172.21.15.31:0/831946262 conn(0x7fef600fb160 msgr2=0x7fef60068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 shutdown_connections 2024-04-22T23:44:59.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:44:59.843+0000 7fef6671b700 1 -- 172.21.15.31:0/831946262 wait complete. 2024-04-22T23:45:00.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:59 smithi031 ceph-mon[26721]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:00.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:59 smithi031 ceph-mon[26721]: pgmap v6: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:00.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:59 smithi031 ceph-mon[26721]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:00.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:44:59 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3528321717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:00.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:59 smithi044 ceph-mon[34463]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:00.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:59 smithi044 ceph-mon[34463]: pgmap v6: 65 pgs: 65 active+clean; 59 MiB data, 361 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:00.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:59 smithi044 ceph-mon[34463]: from='client.24429 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:00.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:44:59 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3528321717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:01.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:00 smithi031 ceph-mon[26721]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:00 smithi031 ceph-mon[26721]: Standby manager daemon smithi031.fazdcv started 2024-04-22T23:45:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:00 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/crt"}]: dispatch 2024-04-22T23:45:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:00 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:45:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:00 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/key"}]: dispatch 2024-04-22T23:45:01.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:00 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:45:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:00 smithi044 ceph-mon[34463]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:00 smithi044 ceph-mon[34463]: Standby manager daemon smithi031.fazdcv started 2024-04-22T23:45:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:00 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/crt"}]: dispatch 2024-04-22T23:45:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:00 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:45:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:00 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/key"}]: dispatch 2024-04-22T23:45:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:00 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/171963317' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:45:02.141 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:01 smithi044 ceph-mon[34463]: pgmap v7: 65 pgs: 65 active+clean; 69 MiB data, 399 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-04-22T23:45:02.141 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:01 smithi044 ceph-mon[34463]: mgrmap e24: smithi044.iglcti(active, since 8s), standbys: smithi031.fazdcv 2024-04-22T23:45:02.141 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:01 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:45:02.429 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:01 smithi031 ceph-mon[26721]: pgmap v7: 65 pgs: 65 active+clean; 69 MiB data, 399 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-04-22T23:45:02.429 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:01 smithi031 ceph-mon[26721]: mgrmap e24: smithi044.iglcti(active, since 8s), standbys: smithi031.fazdcv 2024-04-22T23:45:02.429 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:01 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:45:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:03 smithi044 ceph-mon[34463]: pgmap v8: 65 pgs: 65 active+clean; 69 MiB data, 399 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-04-22T23:45:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:03 smithi031 ceph-mon[26721]: pgmap v8: 65 pgs: 65 active+clean; 69 MiB data, 399 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-04-22T23:45:06.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:05 smithi044 ceph-mon[34463]: pgmap v9: 65 pgs: 65 active+clean; 85 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-04-22T23:45:06.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:05 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:06.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:05 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:05 smithi031 ceph-mon[26721]: pgmap v9: 65 pgs: 65 active+clean; 85 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 154 op/s 2024-04-22T23:45:06.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:05 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:06.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:05 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:07 smithi044 ceph-mon[34463]: pgmap v10: 65 pgs: 65 active+clean; 85 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-04-22T23:45:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:07 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:08.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:07 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:08.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:07 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:08.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:07 smithi031 ceph-mon[26721]: pgmap v10: 65 pgs: 65 active+clean; 85 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-04-22T23:45:08.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:07 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:08.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:07 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:08.237 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:07 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:10.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:09 smithi031 ceph-mon[26721]: pgmap v11: 65 pgs: 65 active+clean; 85 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-04-22T23:45:10.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:09 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:10.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:09 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:09 smithi044 ceph-mon[34463]: pgmap v11: 65 pgs: 65 active+clean; 85 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-04-22T23:45:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:09 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:09 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi031 to 1217M 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:11 smithi031 ceph-mon[26721]: pgmap v12: 65 pgs: 65 active+clean; 107 MiB data, 518 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 223 op/s 2024-04-22T23:45:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi031 to 1217M 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:11.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:11 smithi044 ceph-mon[34463]: pgmap v12: 65 pgs: 65 active+clean; 107 MiB data, 518 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 223 op/s 2024-04-22T23:45:13.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:13 smithi031 ceph-mon[26721]: pgmap v13: 65 pgs: 65 active+clean; 107 MiB data, 518 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 138 op/s 2024-04-22T23:45:13.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:13 smithi044 ceph-mon[34463]: pgmap v13: 65 pgs: 65 active+clean; 107 MiB data, 518 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 138 op/s 2024-04-22T23:45:15.673 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Adjusting osd_memory_target on smithi044 to 2241M 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:15.674 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:45:15.675 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:45:15.675 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: pgmap v14: 65 pgs: 65 active+clean; 125 MiB data, 598 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-04-22T23:45:15.675 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:45:15.675 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:45:15.675 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:45:15.675 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:15 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:45:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Adjusting osd_memory_target on smithi044 to 2241M 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: pgmap v14: 65 pgs: 65 active+clean; 125 MiB data, 598 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 234 op/s 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:45:15.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:15 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:45:16.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:45:16.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:45:16.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.807 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.807 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.807 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.807 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-22T23:45:16.807 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:16 smithi031 ceph-mon[26721]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-22T23:45:16.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-22T23:45:16.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:16 smithi044 ceph-mon[34463]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-22T23:45:17.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:17 smithi031 ceph-mon[26721]: pgmap v15: 65 pgs: 65 active+clean; 125 MiB data, 598 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-04-22T23:45:17.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:17 smithi044 ceph-mon[34463]: pgmap v15: 65 pgs: 65 active+clean; 125 MiB data, 598 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-04-22T23:45:19.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:19 smithi031 ceph-mon[26721]: pgmap v16: 65 pgs: 65 active+clean; 125 MiB data, 598 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-04-22T23:45:19.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:19 smithi044 ceph-mon[34463]: pgmap v16: 65 pgs: 65 active+clean; 125 MiB data, 598 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-04-22T23:45:21.953 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:21 smithi031 ceph-mon[26721]: pgmap v17: 65 pgs: 65 active+clean; 140 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 282 op/s 2024-04-22T23:45:22.168 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:21 smithi044 ceph-mon[34463]: pgmap v17: 65 pgs: 65 active+clean; 140 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 282 op/s 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: Upgrade: Need to upgrade myself (mgr.smithi044.iglcti) 2024-04-22T23:45:23.132 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: Upgrade: Need to upgrade myself (mgr.smithi044.iglcti) 2024-04-22T23:45:23.133 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:22 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:23.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:22 smithi044 ceph-mon[34463]: Upgrade: Need to upgrade myself (mgr.smithi044.iglcti) 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:23 smithi044 ceph-mon[34463]: Upgrade: Need to upgrade myself (mgr.smithi044.iglcti) 2024-04-22T23:45:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:23 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:24.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:24 smithi031 ceph-mon[26721]: pgmap v18: 65 pgs: 65 active+clean; 140 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2024-04-22T23:45:24.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:24 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:24.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:24 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:45:24.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:24 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:45:24.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:24 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:45:24.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:24 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:24 smithi044 ceph-mon[34463]: pgmap v18: 65 pgs: 65 active+clean; 140 MiB data, 661 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2024-04-22T23:45:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:24 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:24 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:45:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:24 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:45:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:24 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:45:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:24 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:25 smithi031 ceph-mon[26721]: Upgrade: Updating mgr.smithi031.fazdcv 2024-04-22T23:45:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:25 smithi031 ceph-mon[26721]: Deploying daemon mgr.smithi031.fazdcv on smithi031 2024-04-22T23:45:25.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:25 smithi044 ceph-mon[34463]: Upgrade: Updating mgr.smithi031.fazdcv 2024-04-22T23:45:25.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:25 smithi044 ceph-mon[34463]: Deploying daemon mgr.smithi031.fazdcv on smithi031 2024-04-22T23:45:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:26 smithi031 ceph-mon[26721]: pgmap v19: 65 pgs: 65 active+clean; 158 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-22T23:45:26.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:26 smithi044 ceph-mon[34463]: pgmap v19: 65 pgs: 65 active+clean; 158 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-22T23:45:28.140 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:28 smithi031 ceph-mon[26721]: pgmap v20: 65 pgs: 65 active+clean; 158 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-22T23:45:28.441 INFO:tasks.workunit.client.0.smithi031.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-22T23:45:28.448 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:45:28.448 INFO:tasks.workunit.client.0.smithi031.stderr:+ make 2024-04-22T23:45:28.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:28 smithi044 ceph-mon[34463]: pgmap v20: 65 pgs: 65 active+clean; 158 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-22T23:45:29.112 INFO:tasks.workunit.client.0.smithi031.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-22T23:45:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:29 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:29 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:29 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:29 smithi031 ceph-mon[26721]: pgmap v21: 65 pgs: 65 active+clean; 158 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-22T23:45:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:29 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:29 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:29.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:29 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:29.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:29 smithi044 ceph-mon[34463]: pgmap v21: 65 pgs: 65 active+clean; 158 MiB data, 737 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-22T23:45:29.895 INFO:tasks.workunit.client.0.smithi031.stderr:++ readlink -f fsstress 2024-04-22T23:45:29.897 INFO:tasks.workunit.client.0.smithi031.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-22T23:45:29.897 INFO:tasks.workunit.client.0.smithi031.stderr:+ popd 2024-04-22T23:45:29.898 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:45:29.899 INFO:tasks.workunit.client.0.smithi031.stderr:+ popd 2024-04-22T23:45:29.900 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:45:29.900 INFO:tasks.workunit.client.0.smithi031.stderr:++ mktemp -d -p . 2024-04-22T23:45:29.903 INFO:tasks.workunit.client.0.smithi031.stderr:+ T=./tmp.7vhw0Vx6mu 2024-04-22T23:45:29.903 INFO:tasks.workunit.client.0.smithi031.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.7vhw0Vx6mu -l 1 -n 1000 -p 10 -v 2024-04-22T23:45:29.908 INFO:tasks.workunit.client.0.smithi031.stdout:seed = 1714096231 2024-04-22T23:45:29.914 INFO:tasks.workunit.client.0.smithi031.stdout:1/0: fdatasync - no filename 2024-04-22T23:45:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/1: dread - no filename 2024-04-22T23:45:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/2: fdatasync - no filename 2024-04-22T23:45:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/3: dwrite - no filename 2024-04-22T23:45:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/4: unlink - no file 2024-04-22T23:45:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/5: unlink - no file 2024-04-22T23:45:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/6: write - no filename 2024-04-22T23:45:29.916 INFO:tasks.workunit.client.0.smithi031.stdout:0/0: mkdir d0 0 2024-04-22T23:45:29.918 INFO:tasks.workunit.client.0.smithi031.stdout:3/0: readlink - no filename 2024-04-22T23:45:29.918 INFO:tasks.workunit.client.0.smithi031.stdout:3/1: dread - no filename 2024-04-22T23:45:29.919 INFO:tasks.workunit.client.0.smithi031.stdout:0/1: mkdir d0/d1 0 2024-04-22T23:45:29.919 INFO:tasks.workunit.client.0.smithi031.stdout:0/2: dread - no filename 2024-04-22T23:45:29.919 INFO:tasks.workunit.client.0.smithi031.stdout:2/0: mknod c0 0 2024-04-22T23:45:29.919 INFO:tasks.workunit.client.0.smithi031.stdout:2/1: read - no filename 2024-04-22T23:45:29.920 INFO:tasks.workunit.client.0.smithi031.stdout:2/2: write - no filename 2024-04-22T23:45:29.921 INFO:tasks.workunit.client.0.smithi031.stdout:0/3: rmdir d0 39 2024-04-22T23:45:29.921 INFO:tasks.workunit.client.0.smithi031.stdout:4/0: write - no filename 2024-04-22T23:45:29.922 INFO:tasks.workunit.client.0.smithi031.stdout:4/1: dwrite - no filename 2024-04-22T23:45:29.922 INFO:tasks.workunit.client.0.smithi031.stdout:4/2: write - no filename 2024-04-22T23:45:29.923 INFO:tasks.workunit.client.0.smithi031.stdout:2/3: symlink l1 0 2024-04-22T23:45:29.923 INFO:tasks.workunit.client.0.smithi031.stdout:5/0: fdatasync - no filename 2024-04-22T23:45:29.923 INFO:tasks.workunit.client.0.smithi031.stdout:5/1: dwrite - no filename 2024-04-22T23:45:29.925 INFO:tasks.workunit.client.0.smithi031.stdout:2/4: symlink l2 0 2024-04-22T23:45:29.925 INFO:tasks.workunit.client.0.smithi031.stdout:2/5: write - no filename 2024-04-22T23:45:29.926 INFO:tasks.workunit.client.0.smithi031.stdout:2/6: dwrite - no filename 2024-04-22T23:45:29.926 INFO:tasks.workunit.client.0.smithi031.stdout:2/7: dread - no filename 2024-04-22T23:45:29.927 INFO:tasks.workunit.client.0.smithi031.stdout:6/0: chown . 482996 1 2024-04-22T23:45:29.927 INFO:tasks.workunit.client.0.smithi031.stdout:4/3: symlink l0 0 2024-04-22T23:45:29.927 INFO:tasks.workunit.client.0.smithi031.stdout:4/4: write - no filename 2024-04-22T23:45:29.929 INFO:tasks.workunit.client.0.smithi031.stdout:2/8: creat f3 x:0 0 0 2024-04-22T23:45:29.930 INFO:tasks.workunit.client.0.smithi031.stdout:5/2: creat f0 x:0 0 0 2024-04-22T23:45:29.931 INFO:tasks.workunit.client.0.smithi031.stdout:7/0: truncate - no filename 2024-04-22T23:45:29.931 INFO:tasks.workunit.client.0.smithi031.stdout:7/1: rename - no filename 2024-04-22T23:45:29.932 INFO:tasks.workunit.client.0.smithi031.stdout:4/5: creat f1 x:0 0 0 2024-04-22T23:45:29.932 INFO:tasks.workunit.client.0.smithi031.stdout:2/9: creat f4 x:0 0 0 2024-04-22T23:45:29.933 INFO:tasks.workunit.client.0.smithi031.stdout:2/10: chown c0 0 1 2024-04-22T23:45:29.933 INFO:tasks.workunit.client.0.smithi031.stdout:2/11: read - f3 zero size 2024-04-22T23:45:29.934 INFO:tasks.workunit.client.0.smithi031.stdout:2/12: chown c0 318530 1 2024-04-22T23:45:29.935 INFO:tasks.workunit.client.0.smithi031.stdout:6/1: mknod c0 0 2024-04-22T23:45:29.936 INFO:tasks.workunit.client.0.smithi031.stdout:6/2: chown c0 486 1 2024-04-22T23:45:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:9/0: rename - no filename 2024-04-22T23:45:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:9/1: dread - no filename 2024-04-22T23:45:29.939 INFO:tasks.workunit.client.0.smithi031.stdout:4/6: creat f2 x:0 0 0 2024-04-22T23:45:29.941 INFO:tasks.workunit.client.0.smithi031.stdout:7/2: creat f0 x:0 0 0 2024-04-22T23:45:29.941 INFO:tasks.workunit.client.0.smithi031.stdout:4/7: write f2 [284088,50928] 0 2024-04-22T23:45:29.943 INFO:tasks.workunit.client.0.smithi031.stdout:6/3: rename c0 to c1 0 2024-04-22T23:45:29.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/13: dwrite f4 [0,4194304] 0 2024-04-22T23:45:29.946 INFO:tasks.workunit.client.0.smithi031.stdout:8/0: truncate - no filename 2024-04-22T23:45:29.948 INFO:tasks.workunit.client.0.smithi031.stdout:7/3: symlink l1 0 2024-04-22T23:45:29.949 INFO:tasks.workunit.client.0.smithi031.stdout:7/4: fdatasync f0 0 2024-04-22T23:45:29.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/5: write f0 [734703,122378] 0 2024-04-22T23:45:29.951 INFO:tasks.workunit.client.0.smithi031.stdout:4/8: symlink l3 0 2024-04-22T23:45:29.952 INFO:tasks.workunit.client.0.smithi031.stdout:6/4: symlink l2 0 2024-04-22T23:45:29.953 INFO:tasks.workunit.client.0.smithi031.stdout:9/2: symlink l0 0 2024-04-22T23:45:29.955 INFO:tasks.workunit.client.0.smithi031.stdout:7/6: read f0 [202601,34312] 0 2024-04-22T23:45:29.957 INFO:tasks.workunit.client.0.smithi031.stdout:2/14: symlink l5 0 2024-04-22T23:45:29.958 INFO:tasks.workunit.client.0.smithi031.stdout:2/15: dread - f3 zero size 2024-04-22T23:45:29.959 INFO:tasks.workunit.client.0.smithi031.stdout:2/16: read f4 [2949923,25648] 0 2024-04-22T23:45:29.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/17: write f3 [334609,82645] 0 2024-04-22T23:45:29.964 INFO:tasks.workunit.client.0.smithi031.stdout:9/3: unlink l0 0 2024-04-22T23:45:29.964 INFO:tasks.workunit.client.0.smithi031.stdout:2/18: write f3 [1134317,97844] 0 2024-04-22T23:45:29.966 INFO:tasks.workunit.client.0.smithi031.stdout:8/1: creat f0 x:0 0 0 2024-04-22T23:45:29.967 INFO:tasks.workunit.client.0.smithi031.stdout:2/19: chown l2 530123047 1 2024-04-22T23:45:29.968 INFO:tasks.workunit.client.0.smithi031.stdout:7/7: dwrite f0 [0,4194304] 0 2024-04-22T23:45:29.968 INFO:tasks.workunit.client.0.smithi031.stdout:7/8: rmdir - no directory 2024-04-22T23:45:29.974 INFO:tasks.workunit.client.0.smithi031.stdout:9/4: creat f1 x:0 0 0 2024-04-22T23:45:29.974 INFO:tasks.workunit.client.0.smithi031.stdout:8/2: creat f1 x:0 0 0 2024-04-22T23:45:29.976 INFO:tasks.workunit.client.0.smithi031.stdout:9/5: read - f1 zero size 2024-04-22T23:45:29.978 INFO:tasks.workunit.client.0.smithi031.stdout:7/9: dread f0 [0,4194304] 0 2024-04-22T23:45:29.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/3: write f1 [593126,24403] 0 2024-04-22T23:45:29.979 INFO:tasks.workunit.client.0.smithi031.stdout:8/4: stat f0 0 2024-04-22T23:45:29.980 INFO:tasks.workunit.client.0.smithi031.stdout:9/6: write f1 [760735,102137] 0 2024-04-22T23:45:29.981 INFO:tasks.workunit.client.0.smithi031.stdout:8/5: write f0 [518860,37796] 0 2024-04-22T23:45:29.982 INFO:tasks.workunit.client.0.smithi031.stdout:7/10: write f0 [1351792,55471] 0 2024-04-22T23:45:29.984 INFO:tasks.workunit.client.0.smithi031.stdout:9/7: mkdir d2 0 2024-04-22T23:45:29.985 INFO:tasks.workunit.client.0.smithi031.stdout:8/6: mknod c2 0 2024-04-22T23:45:29.985 INFO:tasks.workunit.client.0.smithi031.stdout:9/8: write f1 [368405,77697] 0 2024-04-22T23:45:29.987 INFO:tasks.workunit.client.0.smithi031.stdout:2/20: dwrite f4 [0,4194304] 0 2024-04-22T23:45:29.989 INFO:tasks.workunit.client.0.smithi031.stdout:2/21: write f3 [2051990,80973] 0 2024-04-22T23:45:29.991 INFO:tasks.workunit.client.0.smithi031.stdout:2/22: read f4 [2263609,79492] 0 2024-04-22T23:45:29.991 INFO:tasks.workunit.client.0.smithi031.stdout:2/23: rmdir - no directory 2024-04-22T23:45:29.992 INFO:tasks.workunit.client.0.smithi031.stdout:9/9: symlink d2/l3 0 2024-04-22T23:45:29.992 INFO:tasks.workunit.client.0.smithi031.stdout:7/11: dwrite f0 [0,4194304] 0 2024-04-22T23:45:29.993 INFO:tasks.workunit.client.0.smithi031.stdout:9/10: write f1 [152772,54749] 0 2024-04-22T23:45:29.997 INFO:tasks.workunit.client.0.smithi031.stdout:2/24: mkdir d6 0 2024-04-22T23:45:30.000 INFO:tasks.workunit.client.0.smithi031.stdout:7/12: creat f2 x:0 0 0 2024-04-22T23:45:30.001 INFO:tasks.workunit.client.0.smithi031.stdout:7/13: truncate f2 635392 0 2024-04-22T23:45:30.061 INFO:tasks.workunit.client.0.smithi031.stdout:7/14: creat f3 x:0 0 0 2024-04-22T23:45:30.061 INFO:tasks.workunit.client.0.smithi031.stdout:7/15: rmdir - no directory 2024-04-22T23:45:30.061 INFO:tasks.workunit.client.0.smithi031.stdout:2/25: symlink d6/l7 0 2024-04-22T23:45:30.061 INFO:tasks.workunit.client.0.smithi031.stdout:7/16: read f0 [1780141,102901] 0 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 -- 172.21.15.31:0/2145071905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 msgr2=0x7f2728075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 --2- 172.21.15.31:0/2145071905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f2728075b60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f272000b0a0 tx=0x7f272000b3b0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 -- 172.21.15.31:0/2145071905 shutdown_connections 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 --2- 172.21.15.31:0/2145071905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 0x7f2728076e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 --2- 172.21.15.31:0/2145071905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f2728075b60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 -- 172.21.15.31:0/2145071905 >> 172.21.15.31:0/2145071905 conn(0x7f27280fe470 msgr2=0x7f2728100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 -- 172.21.15.31:0/2145071905 shutdown_connections 2024-04-22T23:45:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.070+0000 7f272cee0700 1 -- 172.21.15.31:0/2145071905 wait complete. 2024-04-22T23:45:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.071+0000 7f272cee0700 1 Processor -- start 2024-04-22T23:45:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.071+0000 7f272cee0700 1 -- start start 2024-04-22T23:45:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.072+0000 7f272cee0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f272819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.072+0000 7f272cee0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 0x7f272819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.072+0000 7f272cee0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272819d660 con 0x7f2728075740 2024-04-22T23:45:30.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.072+0000 7f272cee0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272819d7a0 con 0x7f2728076990 2024-04-22T23:45:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f272659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f272819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f272659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f272819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53880/0 (socket says 172.21.15.31:53880) 2024-04-22T23:45:30.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f272659c700 1 -- 172.21.15.31:0/3609287957 learned_addr learned my addr 172.21.15.31:0/3609287957 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:45:30.065 INFO:tasks.workunit.client.0.smithi031.stdout:7/17: creat f4 x:0 0 0 2024-04-22T23:45:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f2725d9b700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 0x7f272819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f2725d9b700 1 -- 172.21.15.31:0/3609287957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 msgr2=0x7f272819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f2725d9b700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f272819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.073+0000 7f2725d9b700 1 -- 172.21.15.31:0/3609287957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720009d00 con 0x7f2728076990 2024-04-22T23:45:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.074+0000 7f2725d9b700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 0x7f272819d040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f271c00e970 tx=0x7f271c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:30.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.074+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c00c9d0 con 0x7f2728076990 2024-04-22T23:45:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.074+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2728102a90 con 0x7f2728076990 2024-04-22T23:45:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.074+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f271c010430 con 0x7f2728076990 2024-04-22T23:45:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.074+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728102fe0 con 0x7f2728076990 2024-04-22T23:45:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.074+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c018650 con 0x7f2728076990 2024-04-22T23:45:30.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.075+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272810fe00 con 0x7f2728076990 2024-04-22T23:45:30.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.076+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91557+0+0 (secure 0 0 0) 0x7f271c0105a0 con 0x7f2728076990 2024-04-22T23:45:30.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.076+0000 7f27177fe700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f271006d8a0 0x7f271006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.076+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f271c014070 con 0x7f2728076990 2024-04-22T23:45:30.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.076+0000 7f272659c700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f271006d8a0 0x7f271006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.077+0000 7f272659c700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f271006d8a0 0x7f271006fd60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2720009480 tx=0x7f272000b700 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.086+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f271c053e20 con 0x7f2728076990 2024-04-22T23:45:30.082 INFO:tasks.workunit.client.0.smithi031.stdout:9/11: fdatasync f1 0 2024-04-22T23:45:30.082 INFO:tasks.workunit.client.0.smithi031.stdout:9/12: write f1 [1083625,26590] 0 2024-04-22T23:45:30.102 INFO:tasks.workunit.client.0.smithi031.stdout:7/18: creat f5 x:0 0 0 2024-04-22T23:45:30.105 INFO:tasks.workunit.client.0.smithi031.stdout:2/26: link c0 d6/c8 0 2024-04-22T23:45:30.105 INFO:tasks.workunit.client.0.smithi031.stdout:2/27: stat l5 0 2024-04-22T23:45:30.110 INFO:tasks.workunit.client.0.smithi031.stdout:7/19: link f4 f6 0 2024-04-22T23:45:30.114 INFO:tasks.workunit.client.0.smithi031.stdout:7/20: rename f0 to f7 0 2024-04-22T23:45:30.121 INFO:tasks.workunit.client.0.smithi031.stdout:7/21: dwrite f3 [0,4194304] 0 2024-04-22T23:45:30.125 INFO:tasks.workunit.client.0.smithi031.stdout:7/22: rename f5 to f8 0 2024-04-22T23:45:30.129 INFO:tasks.workunit.client.0.smithi031.stdout:7/23: mkdir d9 0 2024-04-22T23:45:30.129 INFO:tasks.workunit.client.0.smithi031.stdout:7/24: read f7 [1768890,109727] 0 2024-04-22T23:45:30.130 INFO:tasks.workunit.client.0.smithi031.stdout:7/25: chown f3 609156362 1 2024-04-22T23:45:30.132 INFO:tasks.workunit.client.0.smithi031.stdout:7/26: creat d9/fa x:0 0 0 2024-04-22T23:45:30.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/27: fdatasync f8 0 2024-04-22T23:45:30.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/28: mknod d9/cb 0 2024-04-22T23:45:30.136 INFO:tasks.workunit.client.0.smithi031.stdout:7/29: chown f6 63158948 1 2024-04-22T23:45:30.136 INFO:tasks.workunit.client.0.smithi031.stdout:7/30: chown f2 194947 1 2024-04-22T23:45:30.138 INFO:tasks.workunit.client.0.smithi031.stdout:7/31: mknod d9/cc 0 2024-04-22T23:45:30.139 INFO:tasks.workunit.client.0.smithi031.stdout:7/32: mkdir d9/dd 0 2024-04-22T23:45:30.148 INFO:tasks.workunit.client.0.smithi031.stdout:7/33: dwrite f6 [0,4194304] 0 2024-04-22T23:45:30.151 INFO:tasks.workunit.client.0.smithi031.stdout:7/34: mknod d9/ce 0 2024-04-22T23:45:30.153 INFO:tasks.workunit.client.0.smithi031.stdout:7/35: creat d9/ff x:0 0 0 2024-04-22T23:45:30.163 INFO:tasks.workunit.client.0.smithi031.stdout:7/36: dwrite d9/fa [0,4194304] 0 2024-04-22T23:45:30.168 INFO:tasks.workunit.client.0.smithi031.stdout:7/37: rmdir d9/dd 0 2024-04-22T23:45:30.169 INFO:tasks.workunit.client.0.smithi031.stdout:7/38: write f8 [714615,38257] 0 2024-04-22T23:45:30.174 INFO:tasks.workunit.client.0.smithi031.stdout:7/39: dread f7 [0,4194304] 0 2024-04-22T23:45:30.175 INFO:tasks.workunit.client.0.smithi031.stdout:7/40: write f7 [2838837,13780] 0 2024-04-22T23:45:30.180 INFO:tasks.workunit.client.0.smithi031.stdout:7/41: dread f8 [0,4194304] 0 2024-04-22T23:45:30.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/42: chown d9/cb 41 1 2024-04-22T23:45:30.183 INFO:tasks.workunit.client.0.smithi031.stdout:7/43: rename f4 to d9/f10 0 2024-04-22T23:45:30.183 INFO:tasks.workunit.client.0.smithi031.stdout:7/44: chown f8 32 1 2024-04-22T23:45:30.186 INFO:tasks.workunit.client.0.smithi031.stdout:7/45: symlink d9/l11 0 2024-04-22T23:45:30.190 INFO:tasks.workunit.client.0.smithi031.stdout:7/46: link d9/l11 d9/l12 0 2024-04-22T23:45:30.192 INFO:tasks.workunit.client.0.smithi031.stdout:7/47: mknod d9/c13 0 2024-04-22T23:45:30.192 INFO:tasks.workunit.client.0.smithi031.stdout:7/48: fdatasync d9/fa 0 2024-04-22T23:45:30.195 INFO:tasks.workunit.client.0.smithi031.stdout:7/49: mkdir d9/d14 0 2024-04-22T23:45:30.195 INFO:tasks.workunit.client.0.smithi031.stdout:7/50: dread - d9/ff zero size 2024-04-22T23:45:30.203 INFO:tasks.workunit.client.0.smithi031.stdout:7/51: dwrite f2 [0,4194304] 0 2024-04-22T23:45:30.203 INFO:tasks.workunit.client.0.smithi031.stdout:7/52: chown f6 0 1 2024-04-22T23:45:30.204 INFO:tasks.workunit.client.0.smithi031.stdout:7/53: write f3 [4070481,26474] 0 2024-04-22T23:45:30.214 INFO:tasks.workunit.client.0.smithi031.stdout:7/54: dwrite f2 [0,4194304] 0 2024-04-22T23:45:30.260 INFO:tasks.workunit.client.0.smithi031.stdout:0/4: chown d0 1775 1 2024-04-22T23:45:30.263 INFO:tasks.workunit.client.0.smithi031.stdout:0/5: mkdir d0/d1/d2 0 2024-04-22T23:45:30.265 INFO:tasks.workunit.client.0.smithi031.stdout:0/6: mkdir d0/d1/d2/d3 0 2024-04-22T23:45:30.265 INFO:tasks.workunit.client.0.smithi031.stdout:0/7: dread - no filename 2024-04-22T23:45:30.267 INFO:tasks.workunit.client.0.smithi031.stdout:0/8: symlink d0/d1/d2/l4 0 2024-04-22T23:45:30.267 INFO:tasks.workunit.client.0.smithi031.stdout:0/9: dwrite - no filename 2024-04-22T23:45:30.267 INFO:tasks.workunit.client.0.smithi031.stdout:0/10: write - no filename 2024-04-22T23:45:30.268 INFO:tasks.workunit.client.0.smithi031.stdout:0/11: rename d0/d1 to d0/d1/d5 22 2024-04-22T23:45:30.268 INFO:tasks.workunit.client.0.smithi031.stdout:0/12: dwrite - no filename 2024-04-22T23:45:30.268 INFO:tasks.workunit.client.0.smithi031.stdout:0/13: truncate - no filename 2024-04-22T23:45:30.269 INFO:tasks.workunit.client.0.smithi031.stdout:5/3: fsync f0 0 2024-04-22T23:45:30.270 INFO:tasks.workunit.client.0.smithi031.stdout:0/14: symlink d0/d1/d2/l6 0 2024-04-22T23:45:30.270 INFO:tasks.workunit.client.0.smithi031.stdout:5/4: truncate f0 736687 0 2024-04-22T23:45:30.272 INFO:tasks.workunit.client.0.smithi031.stdout:5/5: symlink l1 0 2024-04-22T23:45:30.273 INFO:tasks.workunit.client.0.smithi031.stdout:0/15: rmdir d0/d1/d2 39 2024-04-22T23:45:30.278 INFO:tasks.workunit.client.0.smithi031.stdout:6/5: unlink c1 0 2024-04-22T23:45:30.279 INFO:tasks.workunit.client.0.smithi031.stdout:5/6: dwrite f0 [0,4194304] 0 2024-04-22T23:45:30.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/6: mkdir d3 0 2024-04-22T23:45:30.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/7: read - no filename 2024-04-22T23:45:30.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/8: read - no filename 2024-04-22T23:45:30.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/9: chown l2 3024773 1 2024-04-22T23:45:30.282 INFO:tasks.workunit.client.0.smithi031.stdout:6/10: write - no filename 2024-04-22T23:45:30.282 INFO:tasks.workunit.client.0.smithi031.stdout:4/9: getdents . 0 2024-04-22T23:45:30.282 INFO:tasks.workunit.client.0.smithi031.stdout:4/10: dread - f1 zero size 2024-04-22T23:45:30.286 INFO:tasks.workunit.client.0.smithi031.stdout:5/7: link f0 f2 0 2024-04-22T23:45:30.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/11: creat f4 x:0 0 0 2024-04-22T23:45:30.289 INFO:tasks.workunit.client.0.smithi031.stdout:6/11: creat d3/f4 x:0 0 0 2024-04-22T23:45:30.290 INFO:tasks.workunit.client.0.smithi031.stdout:4/12: symlink l5 0 2024-04-22T23:45:30.290 INFO:tasks.workunit.client.0.smithi031.stdout:4/13: dread - f4 zero size 2024-04-22T23:45:30.292 INFO:tasks.workunit.client.0.smithi031.stdout:6/12: mknod d3/c5 0 2024-04-22T23:45:30.292 INFO:tasks.workunit.client.0.smithi031.stdout:5/8: dread f2 [0,4194304] 0 2024-04-22T23:45:30.293 INFO:tasks.workunit.client.0.smithi031.stdout:9/13: fsync f1 0 2024-04-22T23:45:30.293 INFO:tasks.workunit.client.0.smithi031.stdout:4/14: creat f6 x:0 0 0 2024-04-22T23:45:30.297 INFO:tasks.workunit.client.0.smithi031.stdout:5/9: write f0 [2121034,103004] 0 2024-04-22T23:45:30.297 INFO:tasks.workunit.client.0.smithi031.stdout:5/10: rmdir - no directory 2024-04-22T23:45:30.299 INFO:tasks.workunit.client.0.smithi031.stdout:5/11: readlink l1 0 2024-04-22T23:45:30.301 INFO:tasks.workunit.client.0.smithi031.stdout:5/12: fdatasync f0 0 2024-04-22T23:45:30.301 INFO:tasks.workunit.client.0.smithi031.stdout:9/14: dread f1 [0,4194304] 0 2024-04-22T23:45:30.303 INFO:tasks.workunit.client.0.smithi031.stdout:6/13: dwrite d3/f4 [0,4194304] 0 2024-04-22T23:45:30.305 INFO:tasks.workunit.client.0.smithi031.stdout:9/15: symlink d2/l4 0 2024-04-22T23:45:30.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/13: link f2 f3 0 2024-04-22T23:45:30.307 INFO:tasks.workunit.client.0.smithi031.stdout:5/14: readlink l1 0 2024-04-22T23:45:30.307 INFO:tasks.workunit.client.0.smithi031.stdout:5/15: stat f2 0 2024-04-22T23:45:30.307 INFO:tasks.workunit.client.0.smithi031.stdout:6/14: mknod d3/c6 0 2024-04-22T23:45:30.308 INFO:tasks.workunit.client.0.smithi031.stdout:9/16: rmdir d2 39 2024-04-22T23:45:30.311 INFO:tasks.workunit.client.0.smithi031.stdout:5/16: symlink l4 0 2024-04-22T23:45:30.317 INFO:tasks.workunit.client.0.smithi031.stdout:9/17: rename f1 to d2/f5 0 2024-04-22T23:45:30.320 INFO:tasks.workunit.client.0.smithi031.stdout:5/17: link f2 f5 0 2024-04-22T23:45:30.333 INFO:tasks.workunit.client.0.smithi031.stdout:2/28: truncate f3 1285875 0 2024-04-22T23:45:30.334 INFO:tasks.workunit.client.0.smithi031.stdout:2/29: fsync f4 0 2024-04-22T23:45:30.363 INFO:tasks.workunit.client.0.smithi031.stdout:4/15: fsync f4 0 2024-04-22T23:45:30.364 INFO:tasks.workunit.client.0.smithi031.stdout:4/16: dread - f4 zero size 2024-04-22T23:45:30.365 INFO:tasks.workunit.client.0.smithi031.stdout:4/17: mknod c7 0 2024-04-22T23:45:30.366 INFO:tasks.workunit.client.0.smithi031.stdout:4/18: readlink l0 0 2024-04-22T23:45:30.366 INFO:tasks.workunit.client.0.smithi031.stdout:4/19: rmdir - no directory 2024-04-22T23:45:30.366 INFO:tasks.workunit.client.0.smithi031.stdout:5/18: truncate f5 676346 0 2024-04-22T23:45:30.367 INFO:tasks.workunit.client.0.smithi031.stdout:4/20: symlink l8 0 2024-04-22T23:45:30.368 INFO:tasks.workunit.client.0.smithi031.stdout:5/19: rename l4 to l6 0 2024-04-22T23:45:30.370 INFO:tasks.workunit.client.0.smithi031.stdout:4/21: rename f6 to f9 0 2024-04-22T23:45:30.370 INFO:tasks.workunit.client.0.smithi031.stdout:4/22: chown l8 307067 1 2024-04-22T23:45:30.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/18: fsync d2/f5 0 2024-04-22T23:45:30.374 INFO:tasks.workunit.client.0.smithi031.stdout:6/15: truncate d3/f4 3639814 0 2024-04-22T23:45:30.376 INFO:tasks.workunit.client.0.smithi031.stdout:9/19: mknod d2/c6 0 2024-04-22T23:45:30.376 INFO:tasks.workunit.client.0.smithi031.stdout:4/23: dwrite f2 [0,4194304] 0 2024-04-22T23:45:30.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/16: symlink d3/l7 0 2024-04-22T23:45:30.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/24: rename l3 to la 0 2024-04-22T23:45:30.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/25: mkdir db 0 2024-04-22T23:45:30.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/26: chown l8 11026 1 2024-04-22T23:45:30.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.396+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2728102c50 con 0x7f271006d8a0 2024-04-22T23:45:30.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.398+0000 7f27177fe700 1 -- 172.21.15.31:0/3609287957 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f2728102c50 con 0x7f271006d8a0 2024-04-22T23:45:30.391 INFO:tasks.workunit.client.0.smithi031.stdout:4/27: creat db/fc x:0 0 0 2024-04-22T23:45:30.392 INFO:tasks.workunit.client.0.smithi031.stdout:4/28: write f2 [3335691,56744] 0 2024-04-22T23:45:30.393 INFO:tasks.workunit.client.0.smithi031.stdout:6/17: fsync d3/f4 0 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.401+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f271006d8a0 msgr2=0x7f271006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.401+0000 7f272cee0700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f271006d8a0 0x7f271006fd60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2720009480 tx=0x7f272000b700 comp rx=0 tx=0).stop 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.401+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 msgr2=0x7f272819d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.401+0000 7f272cee0700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 0x7f272819d040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f271c00e970 tx=0x7f271c00ec80 comp rx=0 tx=0).stop 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.401+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 shutdown_connections 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.402+0000 7f272cee0700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f271006d8a0 0x7f271006fd60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.402+0000 7f272cee0700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2728075740 0x7f272819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.402+0000 7f272cee0700 1 --2- 172.21.15.31:0/3609287957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2728076990 0x7f272819d040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.402+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 >> 172.21.15.31:0/3609287957 conn(0x7f27280fe470 msgr2=0x7f272810d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.402+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 shutdown_connections 2024-04-22T23:45:30.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.402+0000 7f272cee0700 1 -- 172.21.15.31:0/3609287957 wait complete. 2024-04-22T23:45:30.395 INFO:tasks.workunit.client.0.smithi031.stdout:6/18: read d3/f4 [663944,53551] 0 2024-04-22T23:45:30.397 INFO:tasks.workunit.client.0.smithi031.stdout:6/19: read d3/f4 [2177770,81966] 0 2024-04-22T23:45:30.402 INFO:tasks.workunit.client.0.smithi031.stdout:6/20: creat d3/f8 x:0 0 0 2024-04-22T23:45:30.405 INFO:tasks.workunit.client.0.smithi031.stdout:4/29: dwrite f1 [0,4194304] 0 2024-04-22T23:45:30.406 INFO:tasks.workunit.client.0.smithi031.stdout:2/30: dwrite f3 [0,4194304] 0 2024-04-22T23:45:30.408 INFO:tasks.workunit.client.0.smithi031.stdout:2/31: fdatasync f4 0 2024-04-22T23:45:30.414 INFO:tasks.workunit.client.0.smithi031.stdout:2/32: creat d6/f9 x:0 0 0 2024-04-22T23:45:30.417 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:45:30.419 INFO:tasks.workunit.client.0.smithi031.stdout:5/20: dwrite f2 [0,4194304] 0 2024-04-22T23:45:30.426 INFO:tasks.workunit.client.0.smithi031.stdout:2/33: creat d6/fa x:0 0 0 2024-04-22T23:45:30.426 INFO:tasks.workunit.client.0.smithi031.stdout:6/21: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:45:30.429 INFO:tasks.workunit.client.0.smithi031.stdout:5/21: creat f7 x:0 0 0 2024-04-22T23:45:30.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/22: mknod d3/c9 0 2024-04-22T23:45:30.432 INFO:tasks.workunit.client.0.smithi031.stdout:4/30: dwrite f9 [0,4194304] 0 2024-04-22T23:45:30.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/34: unlink d6/fa 0 2024-04-22T23:45:30.435 INFO:tasks.workunit.client.0.smithi031.stdout:5/22: dread f0 [0,4194304] 0 2024-04-22T23:45:30.440 INFO:tasks.workunit.client.0.smithi031.stdout:4/31: unlink c7 0 2024-04-22T23:45:30.440 INFO:tasks.workunit.client.0.smithi031.stdout:4/32: stat db 0 2024-04-22T23:45:30.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/35: mknod d6/cb 0 2024-04-22T23:45:30.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/36: write d6/f9 [109172,24567] 0 2024-04-22T23:45:30.444 INFO:tasks.workunit.client.0.smithi031.stdout:4/33: mknod db/cd 0 2024-04-22T23:45:30.445 INFO:tasks.workunit.client.0.smithi031.stdout:6/23: link d3/f4 d3/fa 0 2024-04-22T23:45:30.449 INFO:tasks.workunit.client.0.smithi031.stdout:2/37: dread f3 [0,4194304] 0 2024-04-22T23:45:30.452 INFO:tasks.workunit.client.0.smithi031.stdout:4/34: dwrite f9 [0,4194304] 0 2024-04-22T23:45:30.457 INFO:tasks.workunit.client.0.smithi031.stdout:6/24: dwrite d3/fa [0,4194304] 0 2024-04-22T23:45:30.459 INFO:tasks.workunit.client.0.smithi031.stdout:4/35: dread f9 [0,4194304] 0 2024-04-22T23:45:30.461 INFO:tasks.workunit.client.0.smithi031.stdout:6/25: dread d3/fa [0,4194304] 0 2024-04-22T23:45:30.465 INFO:tasks.workunit.client.0.smithi031.stdout:6/26: creat d3/fb x:0 0 0 2024-04-22T23:45:30.466 INFO:tasks.workunit.client.0.smithi031.stdout:6/27: chown d3/c5 7978939 1 2024-04-22T23:45:30.469 INFO:tasks.workunit.client.0.smithi031.stdout:6/28: symlink d3/lc 0 2024-04-22T23:45:30.469 INFO:tasks.workunit.client.0.smithi031.stdout:2/38: rmdir d6 39 2024-04-22T23:45:30.470 INFO:tasks.workunit.client.0.smithi031.stdout:4/36: getdents db 0 2024-04-22T23:45:30.471 INFO:tasks.workunit.client.0.smithi031.stdout:4/37: stat l5 0 2024-04-22T23:45:30.475 INFO:tasks.workunit.client.0.smithi031.stdout:4/38: symlink db/le 0 2024-04-22T23:45:30.476 INFO:tasks.workunit.client.0.smithi031.stdout:6/29: write d3/fa [2535655,27673] 0 2024-04-22T23:45:30.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/30: rename d3/c9 to d3/cd 0 2024-04-22T23:45:30.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/39: dwrite d6/f9 [0,4194304] 0 2024-04-22T23:45:30.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/40: chown f4 342405 1 2024-04-22T23:45:30.483 INFO:tasks.workunit.client.0.smithi031.stdout:4/39: rename la to db/lf 0 2024-04-22T23:45:30.483 INFO:tasks.workunit.client.0.smithi031.stdout:4/40: stat l5 0 2024-04-22T23:45:30.484 INFO:tasks.workunit.client.0.smithi031.stdout:4/41: dread - db/fc zero size 2024-04-22T23:45:30.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/42: write db/fc [515548,125037] 0 2024-04-22T23:45:30.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/31: creat d3/fe x:0 0 0 2024-04-22T23:45:30.487 INFO:tasks.workunit.client.0.smithi031.stdout:2/41: write f3 [5024357,30441] 0 2024-04-22T23:45:30.489 INFO:tasks.workunit.client.0.smithi031.stdout:4/43: mkdir db/d10 0 2024-04-22T23:45:30.495 INFO:tasks.workunit.client.0.smithi031.stdout:4/44: link f1 db/f11 0 2024-04-22T23:45:30.496 INFO:tasks.workunit.client.0.smithi031.stdout:6/32: dwrite d3/f4 [0,4194304] 0 2024-04-22T23:45:30.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/42: dwrite f3 [0,4194304] 0 2024-04-22T23:45:30.502 INFO:tasks.workunit.client.0.smithi031.stdout:6/33: link d3/lc d3/lf 0 2024-04-22T23:45:30.526 INFO:tasks.workunit.client.0.smithi031.stdout:4/45: dwrite db/f11 [0,4194304] 0 2024-04-22T23:45:30.527 INFO:tasks.workunit.client.0.smithi031.stdout:6/34: write d3/f4 [3808616,117373] 0 2024-04-22T23:45:30.531 INFO:tasks.workunit.client.0.smithi031.stdout:4/46: chown f4 3715 1 2024-04-22T23:45:30.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/43: dwrite f4 [0,4194304] 0 2024-04-22T23:45:30.539 INFO:tasks.workunit.client.0.smithi031.stdout:4/47: rename f2 to db/d10/f12 0 2024-04-22T23:45:30.540 INFO:tasks.workunit.client.0.smithi031.stdout:6/35: rmdir d3 39 2024-04-22T23:45:30.540 INFO:tasks.workunit.client.0.smithi031.stdout:2/44: dread f4 [0,4194304] 0 2024-04-22T23:45:30.549 INFO:tasks.workunit.client.0.smithi031.stdout:2/45: write f4 [16358,43936] 0 2024-04-22T23:45:30.553 INFO:tasks.workunit.client.0.smithi031.stdout:2/46: mkdir d6/dc 0 2024-04-22T23:45:30.553 INFO:tasks.workunit.client.0.smithi031.stdout:4/48: link l5 db/d10/l13 0 2024-04-22T23:45:30.554 INFO:tasks.workunit.client.0.smithi031.stdout:4/49: stat l8 0 2024-04-22T23:45:30.554 INFO:tasks.workunit.client.0.smithi031.stdout:2/47: chown d6/l7 14129052 1 2024-04-22T23:45:30.555 INFO:tasks.workunit.client.0.smithi031.stdout:2/48: chown d6/f9 230 1 2024-04-22T23:45:30.555 INFO:tasks.workunit.client.0.smithi031.stdout:2/49: chown d6/l7 6744412 1 2024-04-22T23:45:30.556 INFO:tasks.workunit.client.0.smithi031.stdout:6/36: link d3/lf d3/l10 0 2024-04-22T23:45:30.556 INFO:tasks.workunit.client.0.smithi031.stdout:6/37: dread - d3/fe zero size 2024-04-22T23:45:30.557 INFO:tasks.workunit.client.0.smithi031.stdout:2/50: unlink d6/cb 0 2024-04-22T23:45:30.558 INFO:tasks.workunit.client.0.smithi031.stdout:6/38: creat d3/f11 x:0 0 0 2024-04-22T23:45:30.559 INFO:tasks.workunit.client.0.smithi031.stdout:2/51: stat c0 0 2024-04-22T23:45:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75beffd700 1 -- 172.21.15.31:0/1744530972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b00054b0 con 0x7f75c01072a0 2024-04-22T23:45:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 -- 172.21.15.31:0/1744530972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 msgr2=0x7f75c01076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 --2- 172.21.15.31:0/1744530972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 0x7f75c01076c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009910 tx=0x7f75b0009c20 comp rx=0 tx=0).stop 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 -- 172.21.15.31:0/1744530972 shutdown_connections 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 --2- 172.21.15.31:0/1744530972 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 --2- 172.21.15.31:0/1744530972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 0x7f75c01076c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 -- 172.21.15.31:0/1744530972 >> 172.21.15.31:0/1744530972 conn(0x7f75c0076290 msgr2=0x7f75c00786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 -- 172.21.15.31:0/1744530972 shutdown_connections 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.576+0000 7f75c69f6700 1 -- 172.21.15.31:0/1744530972 wait complete. 2024-04-22T23:45:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.577+0000 7f75c69f6700 1 Processor -- start 2024-04-22T23:45:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.577+0000 7f75c69f6700 1 -- start start 2024-04-22T23:45:30.570 INFO:tasks.workunit.client.0.smithi031.stdout:6/39: dwrite d3/fe [0,4194304] 0 2024-04-22T23:45:30.570 INFO:tasks.workunit.client.0.smithi031.stdout:2/52: dwrite f4 [0,4194304] 0 2024-04-22T23:45:30.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75c69f6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 0x7f75c019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75c69f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75c69f6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c019d640 con 0x7f75c01072a0 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75c69f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c019d780 con 0x7f75c01084a0 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75bf7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:49800/0 (socket says 172.21.15.31:49800) 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.578+0000 7f75bf7fe700 1 -- 172.21.15.31:0/2956712589 learned_addr learned my addr 172.21.15.31:0/2956712589 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.579+0000 7f75bffff700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 0x7f75c019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.579+0000 7f75bf7fe700 1 -- 172.21.15.31:0/2956712589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 msgr2=0x7f75c019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.579+0000 7f75bf7fe700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 0x7f75c019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.579+0000 7f75bf7fe700 1 -- 172.21.15.31:0/2956712589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00095f0 con 0x7f75c01084a0 2024-04-22T23:45:30.573 INFO:tasks.workunit.client.0.smithi031.stdout:4/50: dread db/fc [0,4194304] 0 2024-04-22T23:45:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.580+0000 7f75bf7fe700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c019d020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b4b0 tx=0x7f75b400b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.581+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b4011800 con 0x7f75c01084a0 2024-04-22T23:45:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.581+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f75b4011e10 con 0x7f75c01084a0 2024-04-22T23:45:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.581+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75b400f2e0 con 0x7f75c01084a0 2024-04-22T23:45:30.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.581+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c01a2230 con 0x7f75c01084a0 2024-04-22T23:45:30.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.581+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c01a2780 con 0x7f75c01084a0 2024-04-22T23:45:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/40: mknod d3/c12 0 2024-04-22T23:45:30.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.583+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91557+0+0 (secure 0 0 0) 0x7f75b4010380 con 0x7f75c01084a0 2024-04-22T23:45:30.576 INFO:tasks.workunit.client.0.smithi031.stdout:4/51: mkdir db/d14 0 2024-04-22T23:45:30.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.583+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c010fe80 con 0x7f75c01084a0 2024-04-22T23:45:30.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.585+0000 7f75bd7fa700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f75a806d950 0x7f75a806fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.585+0000 7f75bffff700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f75a806d950 0x7f75a806fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.585+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f75b408ccc0 con 0x7f75c01084a0 2024-04-22T23:45:30.578 INFO:tasks.workunit.client.0.smithi031.stdout:6/41: mknod d3/c13 0 2024-04-22T23:45:30.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/42: truncate d3/fa 4497239 0 2024-04-22T23:45:30.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.587+0000 7f75bffff700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f75a806d950 0x7f75a806fe10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f75b0006010 tx=0x7f75b0005280 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:30.580 INFO:tasks.workunit.client.0.smithi031.stdout:4/52: write db/d10/f12 [344154,24365] 0 2024-04-22T23:45:30.583 INFO:tasks.workunit.client.0.smithi031.stdout:6/43: creat d3/f14 x:0 0 0 2024-04-22T23:45:30.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.593+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f75b4059bd0 con 0x7f75c01084a0 2024-04-22T23:45:30.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/44: creat d3/f15 x:0 0 0 2024-04-22T23:45:30.586 INFO:tasks.workunit.client.0.smithi031.stdout:4/53: getdents db/d10 0 2024-04-22T23:45:30.587 INFO:tasks.workunit.client.0.smithi031.stdout:6/45: fsync d3/f8 0 2024-04-22T23:45:30.588 INFO:tasks.workunit.client.0.smithi031.stdout:6/46: write d3/fb [1035546,93977] 0 2024-04-22T23:45:30.591 INFO:tasks.workunit.client.0.smithi031.stdout:4/54: dread db/f11 [0,4194304] 0 2024-04-22T23:45:30.592 INFO:tasks.workunit.client.0.smithi031.stdout:6/47: mknod d3/c16 0 2024-04-22T23:45:30.594 INFO:tasks.workunit.client.0.smithi031.stdout:6/48: chown d3/l10 1137 1 2024-04-22T23:45:30.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/55: rmdir db 39 2024-04-22T23:45:30.597 INFO:tasks.workunit.client.0.smithi031.stdout:6/49: rename d3/f4 to d3/f17 0 2024-04-22T23:45:30.598 INFO:tasks.workunit.client.0.smithi031.stdout:6/50: read d3/f8 [2057879,41641] 0 2024-04-22T23:45:30.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/56: dread f1 [0,4194304] 0 2024-04-22T23:45:30.599 INFO:tasks.workunit.client.0.smithi031.stdout:6/51: chown d3/f17 6219983 1 2024-04-22T23:45:30.600 INFO:tasks.workunit.client.0.smithi031.stdout:6/52: dread - d3/f14 zero size 2024-04-22T23:45:30.600 INFO:tasks.workunit.client.0.smithi031.stdout:6/53: fdatasync d3/fe 0 2024-04-22T23:45:30.601 INFO:tasks.workunit.client.0.smithi031.stdout:4/57: chown db/d14 0 1 2024-04-22T23:45:30.602 INFO:tasks.workunit.client.0.smithi031.stdout:4/58: truncate f4 599296 0 2024-04-22T23:45:30.605 INFO:tasks.workunit.client.0.smithi031.stdout:6/54: dread d3/f8 [0,4194304] 0 2024-04-22T23:45:30.605 INFO:tasks.workunit.client.0.smithi031.stdout:4/59: mknod db/d10/c15 0 2024-04-22T23:45:30.606 INFO:tasks.workunit.client.0.smithi031.stdout:4/60: truncate f4 1436174 0 2024-04-22T23:45:30.607 INFO:tasks.workunit.client.0.smithi031.stdout:6/55: creat d3/f18 x:0 0 0 2024-04-22T23:45:30.607 INFO:tasks.workunit.client.0.smithi031.stdout:6/56: chown d3/f17 73403241 1 2024-04-22T23:45:30.611 INFO:tasks.workunit.client.0.smithi031.stdout:6/57: rename d3/c16 to d3/c19 0 2024-04-22T23:45:30.613 INFO:tasks.workunit.client.0.smithi031.stdout:4/61: dwrite db/fc [0,4194304] 0 2024-04-22T23:45:30.617 INFO:tasks.workunit.client.0.smithi031.stdout:4/62: mknod db/c16 0 2024-04-22T23:45:30.622 INFO:tasks.workunit.client.0.smithi031.stdout:4/63: chown l5 52636026 1 2024-04-22T23:45:30.624 INFO:tasks.workunit.client.0.smithi031.stdout:4/64: symlink db/d10/l17 0 2024-04-22T23:45:30.627 INFO:tasks.workunit.client.0.smithi031.stdout:6/58: truncate d3/fb 121583 0 2024-04-22T23:45:30.630 INFO:tasks.workunit.client.0.smithi031.stdout:4/65: fdatasync db/fc 0 2024-04-22T23:45:30.631 INFO:tasks.workunit.client.0.smithi031.stdout:4/66: write f9 [4698274,118807] 0 2024-04-22T23:45:30.636 INFO:tasks.workunit.client.0.smithi031.stdout:4/67: dread db/f11 [0,4194304] 0 2024-04-22T23:45:30.644 INFO:tasks.workunit.client.0.smithi031.stdout:4/68: mknod db/d14/c18 0 2024-04-22T23:45:30.645 INFO:tasks.workunit.client.0.smithi031.stdout:4/69: chown db/d14/c18 15317512 1 2024-04-22T23:45:30.646 INFO:tasks.workunit.client.0.smithi031.stdout:6/59: dwrite d3/fb [0,4194304] 0 2024-04-22T23:45:30.648 INFO:tasks.workunit.client.0.smithi031.stdout:4/70: mknod db/c19 0 2024-04-22T23:45:30.649 INFO:tasks.workunit.client.0.smithi031.stdout:4/71: chown db/f11 78 1 2024-04-22T23:45:30.658 INFO:tasks.workunit.client.0.smithi031.stdout:6/60: dwrite d3/f17 [4194304,4194304] 0 2024-04-22T23:45:30.660 INFO:tasks.workunit.client.0.smithi031.stdout:4/72: dwrite db/f11 [0,4194304] 0 2024-04-22T23:45:30.662 INFO:tasks.workunit.client.0.smithi031.stdout:4/73: chown db/le 2229588 1 2024-04-22T23:45:30.663 INFO:tasks.workunit.client.0.smithi031.stdout:4/74: write db/d10/f12 [4737016,86432] 0 2024-04-22T23:45:30.665 INFO:tasks.workunit.client.0.smithi031.stdout:6/61: dread d3/fb [0,4194304] 0 2024-04-22T23:45:30.672 INFO:tasks.workunit.client.0.smithi031.stdout:4/75: dwrite db/f11 [0,4194304] 0 2024-04-22T23:45:30.676 INFO:tasks.workunit.client.0.smithi031.stdout:6/62: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:45:30.682 INFO:tasks.workunit.client.0.smithi031.stdout:6/63: rename d3/fb to d3/f1a 0 2024-04-22T23:45:30.683 INFO:tasks.workunit.client.0.smithi031.stdout:6/64: chown d3/f17 0 1 2024-04-22T23:45:30.688 INFO:tasks.workunit.client.0.smithi031.stdout:6/65: dread d3/f8 [0,4194304] 0 2024-04-22T23:45:30.690 INFO:tasks.workunit.client.0.smithi031.stdout:4/76: link db/d14/c18 db/d14/c1a 0 2024-04-22T23:45:30.690 INFO:tasks.workunit.client.0.smithi031.stdout:4/77: truncate f1 4303665 0 2024-04-22T23:45:30.690 INFO:tasks.workunit.client.0.smithi031.stdout:4/78: stat db/cd 0 2024-04-22T23:45:30.691 INFO:tasks.workunit.client.0.smithi031.stdout:4/79: stat db/cd 0 2024-04-22T23:45:30.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/80: dread f9 [0,4194304] 0 2024-04-22T23:45:30.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/81: chown db/d14 54 1 2024-04-22T23:45:30.697 INFO:tasks.workunit.client.0.smithi031.stdout:4/82: dread f4 [0,4194304] 0 2024-04-22T23:45:30.704 INFO:tasks.workunit.client.0.smithi031.stdout:4/83: dwrite db/fc [4194304,4194304] 0 2024-04-22T23:45:30.708 INFO:tasks.workunit.client.0.smithi031.stdout:4/84: dread f9 [0,4194304] 0 2024-04-22T23:45:30.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/66: getdents d3 0 2024-04-22T23:45:30.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/67: write d3/f11 [350139,104430] 0 2024-04-22T23:45:30.719 INFO:tasks.workunit.client.0.smithi031.stdout:6/68: symlink d3/l1b 0 2024-04-22T23:45:30.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/69: symlink d3/l1c 0 2024-04-22T23:45:30.722 INFO:tasks.workunit.client.0.smithi031.stdout:6/70: fdatasync d3/f18 0 2024-04-22T23:45:30.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.874+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75c01a2e70 con 0x7f75a806d950 2024-04-22T23:45:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.876+0000 7f75bd7fa700 1 -- 172.21.15.31:0/2956712589 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f75c01a2e70 con 0x7f75a806d950 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f75a806d950 msgr2=0x7f75a806fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f75a806d950 0x7f75a806fe10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f75b0006010 tx=0x7f75b0005280 comp rx=0 tx=0).stop 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 msgr2=0x7f75c019d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c019d020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b4b0 tx=0x7f75b400b7c0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 shutdown_connections 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7f75a806d950 0x7f75a806fe10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75c01072a0 0x7f75c019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 --2- 172.21.15.31:0/2956712589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75c01084a0 0x7f75c019d020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 >> 172.21.15.31:0/2956712589 conn(0x7f75c0076290 msgr2=0x7f75c010b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 shutdown_connections 2024-04-22T23:45:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:30.878+0000 7f75c69f6700 1 -- 172.21.15.31:0/2956712589 wait complete. 2024-04-22T23:45:30.882 INFO:tasks.workunit.client.0.smithi031.stdout:6/71: rmdir d3 39 2024-04-22T23:45:30.891 INFO:tasks.workunit.client.0.smithi031.stdout:6/72: dwrite d3/f11 [0,4194304] 0 2024-04-22T23:45:30.894 INFO:tasks.workunit.client.0.smithi031.stdout:6/73: mkdir d3/d1d 0 2024-04-22T23:45:30.896 INFO:tasks.workunit.client.0.smithi031.stdout:6/74: unlink d3/cd 0 2024-04-22T23:45:30.897 INFO:tasks.workunit.client.0.smithi031.stdout:6/75: chown d3/c5 62484804 1 2024-04-22T23:45:30.899 INFO:tasks.workunit.client.0.smithi031.stdout:6/76: symlink d3/d1d/l1e 0 2024-04-22T23:45:30.912 INFO:tasks.workunit.client.0.smithi031.stdout:6/77: fdatasync d3/f8 0 2024-04-22T23:45:30.913 INFO:tasks.workunit.client.0.smithi031.stdout:6/78: write d3/f14 [290314,97939] 0 2024-04-22T23:45:30.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 -- 172.21.15.31:0/3742205225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300ffbb0 msgr2=0x7ff1300fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 --2- 172.21.15.31:0/3742205225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300ffbb0 0x7ff1300fffd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff120009910 tx=0x7ff120009c20 comp rx=0 tx=0).stop 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 -- 172.21.15.31:0/3742205225 shutdown_connections 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 --2- 172.21.15.31:0/3742205225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff130100db0 0x7ff130101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 --2- 172.21.15.31:0/3742205225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300ffbb0 0x7ff1300fffd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 -- 172.21.15.31:0/3742205225 >> 172.21.15.31:0/3742205225 conn(0x7ff1300fb150 msgr2=0x7ff1300fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 -- 172.21.15.31:0/3742205225 shutdown_connections 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.000+0000 7ff135a9c700 1 -- 172.21.15.31:0/3742205225 wait complete. 2024-04-22T23:45:30.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff135a9c700 1 Processor -- start 2024-04-22T23:45:30.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff135a9c700 1 -- start start 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff135a9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300ffbb0 0x7ff1301953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff135a9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 0x7ff130195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff135a9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130195f40 con 0x7ff130100db0 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff135a9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130196080 con 0x7ff1300ffbb0 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff12effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 0x7ff130195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff12effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 0x7ff130195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57316/0 (socket says 172.21.15.31:57316) 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff12effd700 1 -- 172.21.15.31:0/2489853297 learned_addr learned my addr 172.21.15.31:0/2489853297 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.001+0000 7ff12f7fe700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300ffbb0 0x7ff1301953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12effd700 1 -- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300ffbb0 msgr2=0x7ff1301953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12effd700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300ffbb0 0x7ff1301953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12effd700 1 -- 172.21.15.31:0/2489853297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200095f0 con 0x7ff130100db0 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12f7fe700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300ffbb0 0x7ff1301953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12effd700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 0x7ff130195920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff12400eab0 tx=0x7ff12400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:30.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff124013070 con 0x7ff130100db0 2024-04-22T23:45:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff124017410 con 0x7ff130100db0 2024-04-22T23:45:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1240165e0 con 0x7ff130100db0 2024-04-22T23:45:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130068e70 con 0x7ff130100db0 2024-04-22T23:45:30.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.002+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1300693c0 con 0x7ff130100db0 2024-04-22T23:45:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.003+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130108710 con 0x7ff130100db0 2024-04-22T23:45:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.004+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 91557+0+0 (secure 0 0 0) 0x7ff124017580 con 0x7ff130100db0 2024-04-22T23:45:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.004+0000 7ff12cff9700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ff11806d850 0x7ff11806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.005+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff12408ea80 con 0x7ff130100db0 2024-04-22T23:45:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.005+0000 7ff12f7fe700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ff11806d850 0x7ff11806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:30.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.005+0000 7ff12f7fe700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ff11806d850 0x7ff11806fd10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff12000b6f0 tx=0x7ff120005940 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:31.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.010+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7ff12405b8a0 con 0x7ff130100db0 2024-04-22T23:45:31.082 INFO:tasks.workunit.client.0.smithi031.stdout:6/79: rename d3/d1d to d3/d1f 0 2024-04-22T23:45:31.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/80: write d3/f11 [4365962,95138] 0 2024-04-22T23:45:31.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/81: readlink d3/lc 0 2024-04-22T23:45:31.118 INFO:tasks.workunit.client.0.smithi031.stdout:6/82: creat d3/d1f/f20 x:0 0 0 2024-04-22T23:45:31.124 INFO:tasks.workunit.client.0.smithi031.stdout:6/83: dwrite d3/f1a [0,4194304] 0 2024-04-22T23:45:31.129 INFO:tasks.workunit.client.0.smithi031.stdout:6/84: link d3/lf d3/l21 0 2024-04-22T23:45:31.133 INFO:tasks.workunit.client.0.smithi031.stdout:6/85: dread d3/fe [0,4194304] 0 2024-04-22T23:45:31.136 INFO:tasks.workunit.client.0.smithi031.stdout:6/86: unlink l2 0 2024-04-22T23:45:31.143 INFO:tasks.workunit.client.0.smithi031.stdout:6/87: dwrite d3/f15 [0,4194304] 0 2024-04-22T23:45:31.145 INFO:tasks.workunit.client.0.smithi031.stdout:6/88: write d3/f11 [3106351,121337] 0 2024-04-22T23:45:31.151 INFO:tasks.workunit.client.0.smithi031.stdout:6/89: rename d3/fa to d3/d1f/f22 0 2024-04-22T23:45:31.170 INFO:tasks.workunit.client.0.smithi031.stdout:6/90: getdents d3 0 2024-04-22T23:45:31.172 INFO:tasks.workunit.client.0.smithi031.stdout:6/91: write d3/d1f/f22 [3010371,71703] 0 2024-04-22T23:45:31.172 INFO:tasks.workunit.client.0.smithi031.stdout:6/92: dread - d3/f18 zero size 2024-04-22T23:45:31.174 INFO:tasks.workunit.client.0.smithi031.stdout:6/93: mknod d3/c23 0 2024-04-22T23:45:31.182 INFO:tasks.workunit.client.0.smithi031.stdout:6/94: truncate d3/f1a 2768678 0 2024-04-22T23:45:31.186 INFO:tasks.workunit.client.0.smithi031.stdout:6/95: dread d3/f8 [0,4194304] 0 2024-04-22T23:45:31.189 INFO:tasks.workunit.client.0.smithi031.stdout:6/96: symlink d3/d1f/l24 0 2024-04-22T23:45:31.190 INFO:tasks.workunit.client.0.smithi031.stdout:6/97: getdents d3 0 2024-04-22T23:45:31.194 INFO:tasks.workunit.client.0.smithi031.stdout:6/98: truncate d3/fe 1987144 0 2024-04-22T23:45:31.200 INFO:tasks.workunit.client.0.smithi031.stdout:6/99: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:45:31.203 INFO:tasks.workunit.client.0.smithi031.stdout:6/100: creat d3/f25 x:0 0 0 2024-04-22T23:45:31.204 INFO:tasks.workunit.client.0.smithi031.stdout:6/101: chown d3/c6 2384 1 2024-04-22T23:45:31.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/102: stat d3/c5 0 2024-04-22T23:45:31.212 INFO:tasks.workunit.client.0.smithi031.stdout:6/103: link d3/c23 d3/c26 0 2024-04-22T23:45:31.214 INFO:tasks.workunit.client.0.smithi031.stdout:6/104: symlink d3/d1f/l27 0 2024-04-22T23:45:31.215 INFO:tasks.workunit.client.0.smithi031.stdout:6/105: creat d3/d1f/f28 x:0 0 0 2024-04-22T23:45:31.221 INFO:tasks.workunit.client.0.smithi031.stdout:6/106: dwrite d3/f11 [0,4194304] 0 2024-04-22T23:45:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.255+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff130105700 con 0x7ff11806d850 2024-04-22T23:45:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.270+0000 7ff12cff9700 1 -- 172.21.15.31:0/2489853297 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff130105700 con 0x7ff11806d850 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 22s ago 8m 25.6M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (8m) 22s ago 8m 7650k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (2m) 25s ago 7m 7520k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 22s ago 8m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (7m) 25s ago 7m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 22s ago 8m 88.7M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (2m) 22s ago 2m 13.1M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (2m) 22s ago 2m 85.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (2m) 25s ago 2m 108M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (2m) 25s ago 2m 16.9M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 starting - - - - 2024-04-22T23:45:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (62s) 25s ago 7m 491M - 19.0.0-3252-g43be0201 a0434699c1d7 05ec5942bb78 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 22s ago 9m 49.5M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (7m) 25s ago 7m 40.1M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (8m) 22s ago 8m 16.7M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 25s ago 7m 16.5M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 22s ago 6m 69.3M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 22s ago 6m 69.3M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 22s ago 5m 60.8M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (5m) 25s ago 5m 71.1M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (4m) 25s ago 4m 66.8M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (4m) 25s ago 4m 65.7M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:45:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 starting - - - - 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ff11806d850 msgr2=0x7ff11806fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ff11806d850 0x7ff11806fd10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff12000b6f0 tx=0x7ff120005940 comp rx=0 tx=0).stop 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 msgr2=0x7ff130195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 0x7ff130195920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff12400eab0 tx=0x7ff12400edc0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 shutdown_connections 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ff11806d850 0x7ff11806fd10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1300ffbb0 0x7ff1301953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 --2- 172.21.15.31:0/2489853297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff130100db0 0x7ff130195920 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 >> 172.21.15.31:0/2489853297 conn(0x7ff1300fb150 msgr2=0x7ff130103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:31.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 shutdown_connections 2024-04-22T23:45:31.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.273+0000 7ff135a9c700 1 -- 172.21.15.31:0/2489853297 wait complete. 2024-04-22T23:45:31.285 INFO:tasks.workunit.client.0.smithi031.stdout:6/107: dwrite d3/f17 [0,4194304] 0 2024-04-22T23:45:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.395+0000 7fdfec2f8700 1 -- 172.21.15.31:0/678412256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe4101f40 msgr2=0x7fdfe41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.395+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/678412256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe4101f40 0x7fdfe41023a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009910 tx=0x7fdfd4009c20 comp rx=0 tx=0).stop 2024-04-22T23:45:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.395+0000 7fdfec2f8700 1 -- 172.21.15.31:0/678412256 shutdown_connections 2024-04-22T23:45:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.395+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/678412256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe4101f40 0x7fdfe41023a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.395+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/678412256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe40fef60 0x7fdfe40ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.395+0000 7fdfec2f8700 1 -- 172.21.15.31:0/678412256 >> 172.21.15.31:0/678412256 conn(0x7fdfe40fa8c0 msgr2=0x7fdfe40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:31.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.396+0000 7fdfec2f8700 1 -- 172.21.15.31:0/678412256 shutdown_connections 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.396+0000 7fdfec2f8700 1 -- 172.21.15.31:0/678412256 wait complete. 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.396+0000 7fdfec2f8700 1 Processor -- start 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfec2f8700 1 -- start start 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfec2f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 0x7fdfe41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfec2f8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe4101f40 0x7fdfe4196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfec2f8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe4197040 con 0x7fdfe40fef60 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfec2f8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe4197180 con 0x7fdfe4101f40 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfea094700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 0x7fdfe41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfea094700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 0x7fdfe41964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57334/0 (socket says 172.21.15.31:57334) 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfea094700 1 -- 172.21.15.31:0/3693427375 learned_addr learned my addr 172.21.15.31:0/3693427375 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.397+0000 7fdfe9893700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe4101f40 0x7fdfe4196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfea094700 1 -- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe4101f40 msgr2=0x7fdfe4196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfea094700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe4101f40 0x7fdfe4196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfea094700 1 -- 172.21.15.31:0/3693427375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40095f0 con 0x7fdfe40fef60 2024-04-22T23:45:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfea094700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 0x7fdfe41964e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000e8f0 tx=0x7fdfe000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:31.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe0009790 con 0x7fdfe40fef60 2024-04-22T23:45:31.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdfe00044d0 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfe0003da0 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe419bc30 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.398+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe419c130 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.400+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe4109800 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.401+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 91557+0+0 (secure 0 0 0) 0x7fdfe000cba0 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.401+0000 7fdfdb7fe700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fdfd006d850 0x7fdfd006fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.401+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdfe0014070 con 0x7fdfe40fef60 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.402+0000 7fdfe9893700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fdfd006d850 0x7fdfd006fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.402+0000 7fdfe9893700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fdfd006d850 0x7fdfd006fd10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40059d0 tx=0x7fdfd4005940 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:31.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.406+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fdfe005a440 con 0x7fdfe40fef60 2024-04-22T23:45:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.706+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdfe419c7a0 con 0x7fdfe40fef60 2024-04-22T23:45:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.706+0000 7fdfdb7fe700 1 -- 172.21.15.31:0/3693427375 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fdfe0059fd0 con 0x7fdfe40fef60 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:45:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:45:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:45:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:45:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-22T23:45:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:45:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:45:31.700 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fdfd006d850 msgr2=0x7fdfd006fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fdfd006d850 0x7fdfd006fd10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40059d0 tx=0x7fdfd4005940 comp rx=0 tx=0).stop 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 msgr2=0x7fdfe41964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 0x7fdfe41964e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000e8f0 tx=0x7fdfe000ecb0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 shutdown_connections 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7fdfd006d850 0x7fdfd006fd10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfe40fef60 0x7fdfe41964e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.708+0000 7fdfec2f8700 1 --2- 172.21.15.31:0/3693427375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdfe4101f40 0x7fdfe4196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.709+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 >> 172.21.15.31:0/3693427375 conn(0x7fdfe40fa8c0 msgr2=0x7fdfe4105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.709+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 shutdown_connections 2024-04-22T23:45:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.709+0000 7fdfec2f8700 1 -- 172.21.15.31:0/3693427375 wait complete. 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/2472765876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4100e10 msgr2=0x7ffba4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/2472765876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4100e10 0x7ffba4101270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009910 tx=0x7ffb98009c20 comp rx=0 tx=0).stop 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/2472765876 shutdown_connections 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/2472765876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4100e10 0x7ffba4101270 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/2472765876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba40ffc10 0x7ffba4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/2472765876 >> 172.21.15.31:0/2472765876 conn(0x7ffba40fb190 msgr2=0x7ffba40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/2472765876 shutdown_connections 2024-04-22T23:45:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.822+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/2472765876 wait complete. 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffbaa9b8700 1 Processor -- start 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffbaa9b8700 1 -- start start 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffbaa9b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 0x7ffba4103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffbaa9b8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4100e10 0x7ffba41021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffbaa9b8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4102730 con 0x7ffba40ffc10 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffbaa9b8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4102870 con 0x7ffba4100e10 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.823+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 0x7ffba4103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 0x7ffba4103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57362/0 (socket says 172.21.15.31:57362) 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba37fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4100e10 0x7ffba41021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba3fff700 1 -- 172.21.15.31:0/4062631816 learned_addr learned my addr 172.21.15.31:0/4062631816 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:45:31.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba37fe700 1 -- 172.21.15.31:0/4062631816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 msgr2=0x7ffba4103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba37fe700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 0x7ffba4103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba37fe700 1 -- 172.21.15.31:0/4062631816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb980095f0 con 0x7ffba4100e10 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba3fff700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 0x7ffba4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba37fe700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4100e10 0x7ffba41021f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb98000c00 tx=0x7ffb9800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb9801d070 con 0x7ffba4100e10 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4102af0 con 0x7ffba4100e10 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb98005dc0 con 0x7ffba4100e10 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba4102fe0 con 0x7ffba4100e10 2024-04-22T23:45:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.824+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb98017400 con 0x7ffba4100e10 2024-04-22T23:45:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.825+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb90005320 con 0x7ffba4100e10 2024-04-22T23:45:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.826+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) v1 ==== 91557+0+0 (secure 0 0 0) 0x7ffb98003680 con 0x7ffba4100e10 2024-04-22T23:45:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.826+0000 7ffba17fa700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ffb8c06d850 0x7ffb8c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.826+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ffb9808e670 con 0x7ffba4100e10 2024-04-22T23:45:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.827+0000 7ffba3fff700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ffb8c06d850 0x7ffb8c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.827+0000 7ffba3fff700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ffb8c06d850 0x7ffb8c06fd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb94005950 tx=0x7ffb94009260 comp rx=0 tx=0).ready entity=mgr.24391 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:31.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:31.831+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7ffb9805b2c0 con 0x7ffba4100e10 2024-04-22T23:45:32.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.075+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 --> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb90000bf0 con 0x7ffb8c06d850 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.077+0000 7ffba17fa700 1 -- 172.21.15.31:0/4062631816 <== mgr.24391 v2:172.21.15.44:6828/170975257 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7ffb90000bf0 con 0x7ffb8c06d850 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "", 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:45:32.069 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:45:32.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ffb8c06d850 msgr2=0x7ffb8c06fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:32.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ffb8c06d850 0x7ffb8c06fd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb94005950 tx=0x7ffb94009260 comp rx=0 tx=0).stop 2024-04-22T23:45:32.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4100e10 msgr2=0x7ffba41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:32.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4100e10 0x7ffba41021f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb98000c00 tx=0x7ffb9800f690 comp rx=0 tx=0).stop 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 shutdown_connections 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:6828/170975257,v1:172.21.15.44:6829/170975257] conn(0x7ffb8c06d850 0x7ffb8c06fd10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba40ffc10 0x7ffba4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 --2- 172.21.15.31:0/4062631816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffba4100e10 0x7ffba41021f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 >> 172.21.15.31:0/4062631816 conn(0x7ffba40fb190 msgr2=0x7ffba4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 shutdown_connections 2024-04-22T23:45:32.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:45:32.079+0000 7ffbaa9b8700 1 -- 172.21.15.31:0/4062631816 wait complete. 2024-04-22T23:45:32.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:31 smithi031 ceph-mon[26721]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:32.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:31 smithi031 ceph-mon[26721]: pgmap v22: 65 pgs: 65 active+clean; 185 MiB data, 768 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 364 op/s 2024-04-22T23:45:32.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:31 smithi031 ceph-mon[26721]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:32.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:31 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3693427375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:32.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:31 smithi044 ceph-mon[34463]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:32.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:31 smithi044 ceph-mon[34463]: pgmap v22: 65 pgs: 65 active+clean; 185 MiB data, 768 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 364 op/s 2024-04-22T23:45:32.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:31 smithi044 ceph-mon[34463]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:32.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:31 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3693427375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:33.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:32 smithi031 ceph-mon[26721]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:33.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:32 smithi031 ceph-mon[26721]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:33.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:32 smithi044 ceph-mon[34463]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:33.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:32 smithi044 ceph-mon[34463]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:33.536 INFO:tasks.workunit.client.0.smithi031.stdout:2/53: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:8/7: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/16: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:4/85: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:5/23: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/2: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/3: link - no file 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/4: truncate - no filename 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/5: dread - no filename 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/6: truncate - no filename 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:1/7: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:1/8: write - no filename 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:1/9: write - no filename 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:6/108: sync 2024-04-22T23:45:33.537 INFO:tasks.workunit.client.0.smithi031.stdout:7/55: sync 2024-04-22T23:45:33.538 INFO:tasks.workunit.client.0.smithi031.stdout:9/20: sync 2024-04-22T23:45:33.538 INFO:tasks.workunit.client.0.smithi031.stdout:1/10: chown . 103439841 1 2024-04-22T23:45:33.538 INFO:tasks.workunit.client.0.smithi031.stdout:5/24: read - f7 zero size 2024-04-22T23:45:33.538 INFO:tasks.workunit.client.0.smithi031.stdout:9/21: fdatasync d2/f5 0 2024-04-22T23:45:33.539 INFO:tasks.workunit.client.0.smithi031.stdout:8/8: mknod c3 0 2024-04-22T23:45:33.540 INFO:tasks.workunit.client.0.smithi031.stdout:8/9: chown f0 97894 1 2024-04-22T23:45:33.540 INFO:tasks.workunit.client.0.smithi031.stdout:9/22: truncate d2/f5 1699124 0 2024-04-22T23:45:33.541 INFO:tasks.workunit.client.0.smithi031.stdout:8/10: truncate f1 1471875 0 2024-04-22T23:45:33.542 INFO:tasks.workunit.client.0.smithi031.stdout:2/54: mknod d6/cd 0 2024-04-22T23:45:33.544 INFO:tasks.workunit.client.0.smithi031.stdout:4/86: read f4 [739187,94985] 0 2024-04-22T23:45:33.547 INFO:tasks.workunit.client.0.smithi031.stdout:4/87: chown l8 107394 1 2024-04-22T23:45:33.548 INFO:tasks.workunit.client.0.smithi031.stdout:1/11: creat f0 x:0 0 0 2024-04-22T23:45:33.548 INFO:tasks.workunit.client.0.smithi031.stdout:5/25: creat f8 x:0 0 0 2024-04-22T23:45:33.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/23: mknod d2/c7 0 2024-04-22T23:45:33.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/56: dwrite d9/ff [0,4194304] 0 2024-04-22T23:45:33.551 INFO:tasks.workunit.client.0.smithi031.stdout:8/11: dread f0 [0,4194304] 0 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:4/88: chown db/cd 3160 1 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/7: getdents . 0 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/8: dread - no filename 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/9: truncate - no filename 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/10: fsync - no filename 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/11: rename - no filename 2024-04-22T23:45:33.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/12: rename - no filename 2024-04-22T23:45:33.557 INFO:tasks.workunit.client.0.smithi031.stdout:4/89: write db/d10/f12 [1079611,68750] 0 2024-04-22T23:45:33.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/17: rmdir d0/d1/d2 39 2024-04-22T23:45:33.559 INFO:tasks.workunit.client.0.smithi031.stdout:1/12: mkdir d1 0 2024-04-22T23:45:33.560 INFO:tasks.workunit.client.0.smithi031.stdout:1/13: chown d1 177765 1 2024-04-22T23:45:33.561 INFO:tasks.workunit.client.0.smithi031.stdout:2/55: dwrite d6/f9 [0,4194304] 0 2024-04-22T23:45:33.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/24: dwrite d2/f5 [0,4194304] 0 2024-04-22T23:45:33.568 INFO:tasks.workunit.client.0.smithi031.stdout:8/12: write f0 [1457959,43623] 0 2024-04-22T23:45:33.570 INFO:tasks.workunit.client.0.smithi031.stdout:8/13: chown c2 17 1 2024-04-22T23:45:33.574 INFO:tasks.workunit.client.0.smithi031.stdout:4/90: rmdir db/d14 39 2024-04-22T23:45:33.574 INFO:tasks.workunit.client.0.smithi031.stdout:7/57: dwrite f6 [0,4194304] 0 2024-04-22T23:45:33.579 INFO:tasks.workunit.client.0.smithi031.stdout:0/18: rmdir d0 39 2024-04-22T23:45:33.579 INFO:tasks.workunit.client.0.smithi031.stdout:0/19: write - no filename 2024-04-22T23:45:33.581 INFO:tasks.workunit.client.0.smithi031.stdout:7/58: dread d9/fa [0,4194304] 0 2024-04-22T23:45:33.582 INFO:tasks.workunit.client.0.smithi031.stdout:2/56: rename l2 to d6/dc/le 0 2024-04-22T23:45:33.583 INFO:tasks.workunit.client.0.smithi031.stdout:2/57: write f4 [4402161,105992] 0 2024-04-22T23:45:33.584 INFO:tasks.workunit.client.0.smithi031.stdout:2/58: chown l1 942544047 1 2024-04-22T23:45:33.585 INFO:tasks.workunit.client.0.smithi031.stdout:3/13: creat f0 x:0 0 0 2024-04-22T23:45:33.586 INFO:tasks.workunit.client.0.smithi031.stdout:3/14: dread - f0 zero size 2024-04-22T23:45:33.587 INFO:tasks.workunit.client.0.smithi031.stdout:3/15: write f0 [1040957,12709] 0 2024-04-22T23:45:33.591 INFO:tasks.workunit.client.0.smithi031.stdout:6/109: getdents d3 0 2024-04-22T23:45:33.592 INFO:tasks.workunit.client.0.smithi031.stdout:8/14: rename f1 to f4 0 2024-04-22T23:45:33.593 INFO:tasks.workunit.client.0.smithi031.stdout:8/15: write f0 [1195150,109912] 0 2024-04-22T23:45:33.593 INFO:tasks.workunit.client.0.smithi031.stdout:8/16: truncate f0 2524311 0 2024-04-22T23:45:33.595 INFO:tasks.workunit.client.0.smithi031.stdout:1/14: creat d1/f2 x:0 0 0 2024-04-22T23:45:33.599 INFO:tasks.workunit.client.0.smithi031.stdout:7/59: mkdir d9/d15 0 2024-04-22T23:45:33.599 INFO:tasks.workunit.client.0.smithi031.stdout:7/60: fdatasync d9/ff 0 2024-04-22T23:45:33.601 INFO:tasks.workunit.client.0.smithi031.stdout:2/59: creat d6/ff x:0 0 0 2024-04-22T23:45:33.603 INFO:tasks.workunit.client.0.smithi031.stdout:3/16: symlink l1 0 2024-04-22T23:45:33.604 INFO:tasks.workunit.client.0.smithi031.stdout:9/25: fsync d2/f5 0 2024-04-22T23:45:33.608 INFO:tasks.workunit.client.0.smithi031.stdout:6/110: creat d3/d1f/f29 x:0 0 0 2024-04-22T23:45:33.609 INFO:tasks.workunit.client.0.smithi031.stdout:4/91: symlink db/l1b 0 2024-04-22T23:45:33.612 INFO:tasks.workunit.client.0.smithi031.stdout:1/15: rename d1/f2 to d1/f3 0 2024-04-22T23:45:33.613 INFO:tasks.workunit.client.0.smithi031.stdout:3/17: dwrite f0 [0,4194304] 0 2024-04-22T23:45:33.614 INFO:tasks.workunit.client.0.smithi031.stdout:3/18: stat f0 0 2024-04-22T23:45:33.615 INFO:tasks.workunit.client.0.smithi031.stdout:2/60: rename l1 to d6/l10 0 2024-04-22T23:45:33.619 INFO:tasks.workunit.client.0.smithi031.stdout:9/26: creat d2/f8 x:0 0 0 2024-04-22T23:45:33.623 INFO:tasks.workunit.client.0.smithi031.stdout:3/19: dwrite f0 [0,4194304] 0 2024-04-22T23:45:33.624 INFO:tasks.workunit.client.0.smithi031.stdout:6/111: mkdir d3/d2a 0 2024-04-22T23:45:33.625 INFO:tasks.workunit.client.0.smithi031.stdout:6/112: fsync d3/f8 0 2024-04-22T23:45:33.628 INFO:tasks.workunit.client.0.smithi031.stdout:4/92: unlink db/d10/l13 0 2024-04-22T23:45:33.631 INFO:tasks.workunit.client.0.smithi031.stdout:7/61: symlink d9/d14/l16 0 2024-04-22T23:45:33.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/16: symlink d1/l4 0 2024-04-22T23:45:33.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/17: chown f0 31460 1 2024-04-22T23:45:33.633 INFO:tasks.workunit.client.0.smithi031.stdout:1/18: chown d1 331621167 1 2024-04-22T23:45:33.633 INFO:tasks.workunit.client.0.smithi031.stdout:1/19: rename d1 to d1/d5 22 2024-04-22T23:45:33.637 INFO:tasks.workunit.client.0.smithi031.stdout:9/27: creat d2/f9 x:0 0 0 2024-04-22T23:45:33.639 INFO:tasks.workunit.client.0.smithi031.stdout:0/20: creat d0/d1/f7 x:0 0 0 2024-04-22T23:45:33.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/28: creat d2/fa x:0 0 0 2024-04-22T23:45:33.649 INFO:tasks.workunit.client.0.smithi031.stdout:9/29: dread - d2/f8 zero size 2024-04-22T23:45:33.652 INFO:tasks.workunit.client.0.smithi031.stdout:9/30: dread d2/f5 [0,4194304] 0 2024-04-22T23:45:33.653 INFO:tasks.workunit.client.0.smithi031.stdout:9/31: truncate d2/f8 46535 0 2024-04-22T23:45:33.653 INFO:tasks.workunit.client.0.smithi031.stdout:9/32: chown d2 2890 1 2024-04-22T23:45:33.654 INFO:tasks.workunit.client.0.smithi031.stdout:4/93: symlink db/d14/l1c 0 2024-04-22T23:45:33.655 INFO:tasks.workunit.client.0.smithi031.stdout:9/33: truncate d2/fa 28145 0 2024-04-22T23:45:33.655 INFO:tasks.workunit.client.0.smithi031.stdout:9/34: fdatasync d2/f5 0 2024-04-22T23:45:33.655 INFO:tasks.workunit.client.0.smithi031.stdout:0/21: mknod d0/d1/d2/c8 0 2024-04-22T23:45:33.656 INFO:tasks.workunit.client.0.smithi031.stdout:0/22: dread - d0/d1/f7 zero size 2024-04-22T23:45:33.661 INFO:tasks.workunit.client.0.smithi031.stdout:9/35: creat d2/fb x:0 0 0 2024-04-22T23:45:33.661 INFO:tasks.workunit.client.0.smithi031.stdout:4/94: dwrite db/f11 [0,4194304] 0 2024-04-22T23:45:33.663 INFO:tasks.workunit.client.0.smithi031.stdout:0/23: creat d0/d1/d2/f9 x:0 0 0 2024-04-22T23:45:33.663 INFO:tasks.workunit.client.0.smithi031.stdout:6/113: link d3/l10 d3/d2a/l2b 0 2024-04-22T23:45:33.664 INFO:tasks.workunit.client.0.smithi031.stdout:0/24: dread - d0/d1/d2/f9 zero size 2024-04-22T23:45:33.664 INFO:tasks.workunit.client.0.smithi031.stdout:0/25: dread - d0/d1/f7 zero size 2024-04-22T23:45:33.667 INFO:tasks.workunit.client.0.smithi031.stdout:9/36: mkdir d2/dc 0 2024-04-22T23:45:33.667 INFO:tasks.workunit.client.0.smithi031.stdout:9/37: readlink d2/l4 0 2024-04-22T23:45:33.668 INFO:tasks.workunit.client.0.smithi031.stdout:9/38: readlink d2/l4 0 2024-04-22T23:45:33.669 INFO:tasks.workunit.client.0.smithi031.stdout:4/95: unlink db/f11 0 2024-04-22T23:45:33.670 INFO:tasks.workunit.client.0.smithi031.stdout:9/39: write d2/f5 [2812007,37619] 0 2024-04-22T23:45:33.671 INFO:tasks.workunit.client.0.smithi031.stdout:4/96: truncate db/fc 9396202 0 2024-04-22T23:45:33.673 INFO:tasks.workunit.client.0.smithi031.stdout:0/26: dwrite d0/d1/d2/f9 [0,4194304] 0 2024-04-22T23:45:33.676 INFO:tasks.workunit.client.0.smithi031.stdout:6/114: rename d3/d2a/l2b to d3/l2c 0 2024-04-22T23:45:33.677 INFO:tasks.workunit.client.0.smithi031.stdout:9/40: mkdir d2/dd 0 2024-04-22T23:45:33.679 INFO:tasks.workunit.client.0.smithi031.stdout:0/27: creat d0/d1/d2/fa x:0 0 0 2024-04-22T23:45:33.686 INFO:tasks.workunit.client.0.smithi031.stdout:9/41: mknod d2/ce 0 2024-04-22T23:45:33.689 INFO:tasks.workunit.client.0.smithi031.stdout:9/42: dread d2/f5 [0,4194304] 0 2024-04-22T23:45:33.693 INFO:tasks.workunit.client.0.smithi031.stdout:9/43: dread d2/f5 [0,4194304] 0 2024-04-22T23:45:33.700 INFO:tasks.workunit.client.0.smithi031.stdout:9/44: dwrite d2/f9 [0,4194304] 0 2024-04-22T23:45:33.708 INFO:tasks.workunit.client.0.smithi031.stdout:0/28: creat d0/d1/fb x:0 0 0 2024-04-22T23:45:33.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/45: creat d2/dc/ff x:0 0 0 2024-04-22T23:45:33.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/29: rename d0/d1/f7 to d0/d1/fc 0 2024-04-22T23:45:33.721 INFO:tasks.workunit.client.0.smithi031.stdout:9/46: dwrite d2/fb [0,4194304] 0 2024-04-22T23:45:33.725 INFO:tasks.workunit.client.0.smithi031.stdout:0/30: creat d0/d1/d2/d3/fd x:0 0 0 2024-04-22T23:45:33.729 INFO:tasks.workunit.client.0.smithi031.stdout:0/31: write d0/d1/fc [386689,25838] 0 2024-04-22T23:45:33.734 INFO:tasks.workunit.client.0.smithi031.stdout:0/32: mknod d0/d1/d2/ce 0 2024-04-22T23:45:33.737 INFO:tasks.workunit.client.0.smithi031.stdout:0/33: creat d0/ff x:0 0 0 2024-04-22T23:45:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:0/34: dread - d0/d1/d2/fa zero size 2024-04-22T23:45:33.745 INFO:tasks.workunit.client.0.smithi031.stdout:0/35: dwrite d0/d1/fc [0,4194304] 0 2024-04-22T23:45:33.747 INFO:tasks.workunit.client.0.smithi031.stdout:0/36: write d0/d1/fc [2888348,112577] 0 2024-04-22T23:45:33.755 INFO:tasks.workunit.client.0.smithi031.stdout:5/26: sync 2024-04-22T23:45:33.762 INFO:tasks.workunit.client.0.smithi031.stdout:5/27: stat f8 0 2024-04-22T23:45:33.762 INFO:tasks.workunit.client.0.smithi031.stdout:5/28: chown f2 233996716 1 2024-04-22T23:45:33.763 INFO:tasks.workunit.client.0.smithi031.stdout:5/29: write f8 [195178,115502] 0 2024-04-22T23:45:33.766 INFO:tasks.workunit.client.0.smithi031.stdout:5/30: dread f8 [0,4194304] 0 2024-04-22T23:45:33.940 INFO:tasks.workunit.client.0.smithi031.stdout:0/37: creat d0/d1/f10 x:0 0 0 2024-04-22T23:45:33.943 INFO:tasks.workunit.client.0.smithi031.stdout:5/31: mknod c9 0 2024-04-22T23:45:33.946 INFO:tasks.workunit.client.0.smithi031.stdout:0/38: mkdir d0/d1/d2/d3/d11 0 2024-04-22T23:45:33.949 INFO:tasks.workunit.client.0.smithi031.stdout:5/32: dread f8 [0,4194304] 0 2024-04-22T23:45:33.958 INFO:tasks.workunit.client.0.smithi031.stdout:5/33: dwrite f7 [0,4194304] 0 2024-04-22T23:45:33.962 INFO:tasks.workunit.client.0.smithi031.stdout:5/34: write f3 [1063726,83436] 0 2024-04-22T23:45:33.963 INFO:tasks.workunit.client.0.smithi031.stdout:5/35: dread f8 [0,4194304] 0 2024-04-22T23:45:34.158 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:33 smithi031 ceph-mon[26721]: pgmap v23: 65 pgs: 65 active+clean; 185 MiB data, 768 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-04-22T23:45:34.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/47: dwrite d2/f5 [4194304,4194304] 0 2024-04-22T23:45:34.204 INFO:tasks.workunit.client.0.smithi031.stdout:1/20: rename d1/f3 to d1/f6 0 2024-04-22T23:45:34.207 INFO:tasks.workunit.client.0.smithi031.stdout:9/48: creat d2/dc/f10 x:0 0 0 2024-04-22T23:45:34.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/61: rmdir d6 39 2024-04-22T23:45:34.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:33 smithi044 ceph-mon[34463]: pgmap v23: 65 pgs: 65 active+clean; 185 MiB data, 768 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-04-22T23:45:34.212 INFO:tasks.workunit.client.0.smithi031.stdout:1/21: symlink d1/l7 0 2024-04-22T23:45:34.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/22: chown f0 17 1 2024-04-22T23:45:34.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/23: stat f0 0 2024-04-22T23:45:34.214 INFO:tasks.workunit.client.0.smithi031.stdout:8/17: truncate f4 687464 0 2024-04-22T23:45:34.215 INFO:tasks.workunit.client.0.smithi031.stdout:9/49: symlink d2/dd/l11 0 2024-04-22T23:45:34.216 INFO:tasks.workunit.client.0.smithi031.stdout:9/50: chown d2/fa 187588091 1 2024-04-22T23:45:34.217 INFO:tasks.workunit.client.0.smithi031.stdout:8/18: mkdir d5 0 2024-04-22T23:45:34.218 INFO:tasks.workunit.client.0.smithi031.stdout:1/24: fdatasync d1/f6 0 2024-04-22T23:45:34.219 INFO:tasks.workunit.client.0.smithi031.stdout:7/62: getdents d9/d14 0 2024-04-22T23:45:34.225 INFO:tasks.workunit.client.0.smithi031.stdout:8/19: creat d5/f6 x:0 0 0 2024-04-22T23:45:34.226 INFO:tasks.workunit.client.0.smithi031.stdout:3/20: truncate f0 2162931 0 2024-04-22T23:45:34.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/63: creat d9/d15/f17 x:0 0 0 2024-04-22T23:45:34.231 INFO:tasks.workunit.client.0.smithi031.stdout:8/20: mknod d5/c7 0 2024-04-22T23:45:34.238 INFO:tasks.workunit.client.0.smithi031.stdout:7/64: link f7 d9/d14/f18 0 2024-04-22T23:45:34.239 INFO:tasks.workunit.client.0.smithi031.stdout:7/65: write d9/ff [1598014,28415] 0 2024-04-22T23:45:34.247 INFO:tasks.workunit.client.0.smithi031.stdout:6/115: getdents d3 0 2024-04-22T23:45:34.248 INFO:tasks.workunit.client.0.smithi031.stdout:6/116: mknod d3/d1f/c2d 0 2024-04-22T23:45:34.253 INFO:tasks.workunit.client.0.smithi031.stdout:4/97: truncate f9 1270202 0 2024-04-22T23:45:34.272 INFO:tasks.workunit.client.0.smithi031.stdout:9/51: dwrite d2/fb [4194304,4194304] 0 2024-04-22T23:45:34.341 INFO:tasks.workunit.client.0.smithi031.stdout:2/62: sync 2024-04-22T23:45:34.343 INFO:tasks.workunit.client.0.smithi031.stdout:2/63: chown d6 1794 1 2024-04-22T23:45:34.385 INFO:tasks.workunit.client.0.smithi031.stdout:5/36: getdents . 0 2024-04-22T23:45:34.385 INFO:tasks.workunit.client.0.smithi031.stdout:0/39: rmdir d0/d1/d2 39 2024-04-22T23:45:34.385 INFO:tasks.workunit.client.0.smithi031.stdout:5/37: write f0 [1883004,56955] 0 2024-04-22T23:45:34.389 INFO:tasks.workunit.client.0.smithi031.stdout:0/40: stat d0/d1/d2/d3/d11 0 2024-04-22T23:45:34.390 INFO:tasks.workunit.client.0.smithi031.stdout:5/38: symlink la 0 2024-04-22T23:45:34.397 INFO:tasks.workunit.client.0.smithi031.stdout:0/41: dwrite d0/d1/d2/fa [0,4194304] 0 2024-04-22T23:45:34.397 INFO:tasks.workunit.client.0.smithi031.stdout:5/39: dread f3 [0,4194304] 0 2024-04-22T23:45:34.401 INFO:tasks.workunit.client.0.smithi031.stdout:5/40: mkdir db 0 2024-04-22T23:45:34.404 INFO:tasks.workunit.client.0.smithi031.stdout:0/42: rename d0/d1/d2/l4 to d0/d1/d2/d3/l12 0 2024-04-22T23:45:34.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/43: stat d0/d1/d2/l6 0 2024-04-22T23:45:34.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/44: truncate d0/d1/fb 127714 0 2024-04-22T23:45:34.407 INFO:tasks.workunit.client.0.smithi031.stdout:5/41: fdatasync f8 0 2024-04-22T23:45:34.411 INFO:tasks.workunit.client.0.smithi031.stdout:0/45: dread d0/d1/d2/fa [0,4194304] 0 2024-04-22T23:45:34.414 INFO:tasks.workunit.client.0.smithi031.stdout:5/42: getdents db 0 2024-04-22T23:45:34.417 INFO:tasks.workunit.client.0.smithi031.stdout:5/43: mknod db/cc 0 2024-04-22T23:45:34.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/46: dwrite d0/ff [0,4194304] 0 2024-04-22T23:45:34.422 INFO:tasks.workunit.client.0.smithi031.stdout:5/44: rename f2 to db/fd 0 2024-04-22T23:45:34.423 INFO:tasks.workunit.client.0.smithi031.stdout:0/47: mkdir d0/d13 0 2024-04-22T23:45:34.423 INFO:tasks.workunit.client.0.smithi031.stdout:0/48: chown d0/d1/d2/d3/d11 44008990 1 2024-04-22T23:45:34.430 INFO:tasks.workunit.client.0.smithi031.stdout:0/49: creat d0/d1/d2/d3/d11/f14 x:0 0 0 2024-04-22T23:45:34.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/50: creat d0/d1/f15 x:0 0 0 2024-04-22T23:45:34.436 INFO:tasks.workunit.client.0.smithi031.stdout:0/51: chown d0/d1/fb 716364 1 2024-04-22T23:45:34.439 INFO:tasks.workunit.client.0.smithi031.stdout:0/52: dread d0/d1/d2/f9 [0,4194304] 0 2024-04-22T23:45:34.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/52: getdents d2/dc 0 2024-04-22T23:45:34.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/53: fsync d2/f9 0 2024-04-22T23:45:34.461 INFO:tasks.workunit.client.0.smithi031.stdout:9/54: write d2/fa [554577,66647] 0 2024-04-22T23:45:34.461 INFO:tasks.workunit.client.0.smithi031.stdout:1/25: getdents d1 0 2024-04-22T23:45:34.462 INFO:tasks.workunit.client.0.smithi031.stdout:1/26: write d1/f6 [147976,108732] 0 2024-04-22T23:45:34.468 INFO:tasks.workunit.client.0.smithi031.stdout:1/27: creat d1/f8 x:0 0 0 2024-04-22T23:45:34.468 INFO:tasks.workunit.client.0.smithi031.stdout:8/21: dread f4 [0,4194304] 0 2024-04-22T23:45:34.469 INFO:tasks.workunit.client.0.smithi031.stdout:1/28: readlink d1/l4 0 2024-04-22T23:45:34.469 INFO:tasks.workunit.client.0.smithi031.stdout:8/22: chown c2 7583239 1 2024-04-22T23:45:34.469 INFO:tasks.workunit.client.0.smithi031.stdout:1/29: stat f0 0 2024-04-22T23:45:34.471 INFO:tasks.workunit.client.0.smithi031.stdout:9/55: dwrite d2/dc/f10 [0,4194304] 0 2024-04-22T23:45:34.473 INFO:tasks.workunit.client.0.smithi031.stdout:8/23: creat d5/f8 x:0 0 0 2024-04-22T23:45:34.474 INFO:tasks.workunit.client.0.smithi031.stdout:1/30: mknod d1/c9 0 2024-04-22T23:45:34.476 INFO:tasks.workunit.client.0.smithi031.stdout:8/24: chown f0 1169 1 2024-04-22T23:45:34.476 INFO:tasks.workunit.client.0.smithi031.stdout:1/31: read - f0 zero size 2024-04-22T23:45:34.480 INFO:tasks.workunit.client.0.smithi031.stdout:3/21: dwrite f0 [0,4194304] 0 2024-04-22T23:45:34.481 INFO:tasks.workunit.client.0.smithi031.stdout:3/22: stat f0 0 2024-04-22T23:45:34.482 INFO:tasks.workunit.client.0.smithi031.stdout:3/23: chown f0 1788254 1 2024-04-22T23:45:34.483 INFO:tasks.workunit.client.0.smithi031.stdout:8/25: dread f0 [0,4194304] 0 2024-04-22T23:45:34.483 INFO:tasks.workunit.client.0.smithi031.stdout:8/26: readlink - no filename 2024-04-22T23:45:34.484 INFO:tasks.workunit.client.0.smithi031.stdout:7/66: truncate f8 678914 0 2024-04-22T23:45:34.484 INFO:tasks.workunit.client.0.smithi031.stdout:8/27: chown c2 67271671 1 2024-04-22T23:45:34.485 INFO:tasks.workunit.client.0.smithi031.stdout:1/32: symlink d1/la 0 2024-04-22T23:45:34.486 INFO:tasks.workunit.client.0.smithi031.stdout:9/56: getdents d2/dd 0 2024-04-22T23:45:34.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/24: mknod c2 0 2024-04-22T23:45:34.489 INFO:tasks.workunit.client.0.smithi031.stdout:3/25: chown c2 3157544 1 2024-04-22T23:45:34.491 INFO:tasks.workunit.client.0.smithi031.stdout:4/98: truncate f9 1454115 0 2024-04-22T23:45:34.491 INFO:tasks.workunit.client.0.smithi031.stdout:8/28: fdatasync f4 0 2024-04-22T23:45:34.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/67: mknod d9/d15/c19 0 2024-04-22T23:45:34.493 INFO:tasks.workunit.client.0.smithi031.stdout:1/33: mkdir d1/db 0 2024-04-22T23:45:34.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/29: truncate d5/f6 165130 0 2024-04-22T23:45:34.496 INFO:tasks.workunit.client.0.smithi031.stdout:9/57: unlink d2/c6 0 2024-04-22T23:45:34.499 INFO:tasks.workunit.client.0.smithi031.stdout:4/99: mkdir db/d14/d1d 0 2024-04-22T23:45:34.500 INFO:tasks.workunit.client.0.smithi031.stdout:6/117: dwrite d3/f1a [0,4194304] 0 2024-04-22T23:45:34.500 INFO:tasks.workunit.client.0.smithi031.stdout:7/68: symlink d9/d14/l1a 0 2024-04-22T23:45:34.500 INFO:tasks.workunit.client.0.smithi031.stdout:3/26: dread f0 [0,4194304] 0 2024-04-22T23:45:34.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/34: mkdir d1/dc 0 2024-04-22T23:45:34.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/27: chown c2 0 1 2024-04-22T23:45:34.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/58: creat d2/dc/f12 x:0 0 0 2024-04-22T23:45:34.510 INFO:tasks.workunit.client.0.smithi031.stdout:6/118: dwrite d3/f14 [0,4194304] 0 2024-04-22T23:45:34.514 INFO:tasks.workunit.client.0.smithi031.stdout:3/28: write f0 [683123,76016] 0 2024-04-22T23:45:34.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/59: creat d2/dd/f13 x:0 0 0 2024-04-22T23:45:34.517 INFO:tasks.workunit.client.0.smithi031.stdout:4/100: truncate f1 1052537 0 2024-04-22T23:45:34.522 INFO:tasks.workunit.client.0.smithi031.stdout:6/119: mknod d3/d2a/c2e 0 2024-04-22T23:45:34.524 INFO:tasks.workunit.client.0.smithi031.stdout:6/120: fdatasync d3/d1f/f20 0 2024-04-22T23:45:34.524 INFO:tasks.workunit.client.0.smithi031.stdout:3/29: rename f0 to f3 0 2024-04-22T23:45:34.525 INFO:tasks.workunit.client.0.smithi031.stdout:9/60: dwrite d2/f8 [0,4194304] 0 2024-04-22T23:45:34.525 INFO:tasks.workunit.client.0.smithi031.stdout:6/121: read d3/f17 [2302395,42098] 0 2024-04-22T23:45:34.526 INFO:tasks.workunit.client.0.smithi031.stdout:6/122: fdatasync d3/f17 0 2024-04-22T23:45:34.526 INFO:tasks.workunit.client.0.smithi031.stdout:4/101: stat db/lf 0 2024-04-22T23:45:34.527 INFO:tasks.workunit.client.0.smithi031.stdout:6/123: dread - d3/f25 zero size 2024-04-22T23:45:34.527 INFO:tasks.workunit.client.0.smithi031.stdout:2/64: truncate f4 219058 0 2024-04-22T23:45:34.529 INFO:tasks.workunit.client.0.smithi031.stdout:2/65: fsync d6/f9 0 2024-04-22T23:45:34.531 INFO:tasks.workunit.client.0.smithi031.stdout:2/66: truncate d6/f9 5096482 0 2024-04-22T23:45:34.532 INFO:tasks.workunit.client.0.smithi031.stdout:9/61: mknod d2/c14 0 2024-04-22T23:45:34.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/67: chown c0 12307 1 2024-04-22T23:45:34.533 INFO:tasks.workunit.client.0.smithi031.stdout:2/68: read f3 [3293937,78414] 0 2024-04-22T23:45:34.535 INFO:tasks.workunit.client.0.smithi031.stdout:2/69: write d6/f9 [4763749,108654] 0 2024-04-22T23:45:34.535 INFO:tasks.workunit.client.0.smithi031.stdout:4/102: fdatasync f4 0 2024-04-22T23:45:34.537 INFO:tasks.workunit.client.0.smithi031.stdout:2/70: read d6/f9 [4029943,55506] 0 2024-04-22T23:45:34.537 INFO:tasks.workunit.client.0.smithi031.stdout:4/103: fsync db/d10/f12 0 2024-04-22T23:45:34.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/124: dwrite d3/f1a [0,4194304] 0 2024-04-22T23:45:34.543 INFO:tasks.workunit.client.0.smithi031.stdout:9/62: mkdir d2/d15 0 2024-04-22T23:45:34.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/63: dwrite d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:34.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/64: stat d2/d15 0 2024-04-22T23:45:34.551 INFO:tasks.workunit.client.0.smithi031.stdout:2/71: creat d6/f11 x:0 0 0 2024-04-22T23:45:34.552 INFO:tasks.workunit.client.0.smithi031.stdout:2/72: dread - d6/f11 zero size 2024-04-22T23:45:34.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/53: getdents d0/d1/d2/d3 0 2024-04-22T23:45:34.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/45: write f8 [199249,5333] 0 2024-04-22T23:45:34.562 INFO:tasks.workunit.client.0.smithi031.stdout:4/104: mkdir db/d14/d1d/d1e 0 2024-04-22T23:45:34.566 INFO:tasks.workunit.client.0.smithi031.stdout:6/125: unlink d3/lc 0 2024-04-22T23:45:34.566 INFO:tasks.workunit.client.0.smithi031.stdout:6/126: chown d3/f1a 72 1 2024-04-22T23:45:34.567 INFO:tasks.workunit.client.0.smithi031.stdout:0/54: symlink d0/d1/d2/d3/l16 0 2024-04-22T23:45:34.569 INFO:tasks.workunit.client.0.smithi031.stdout:9/65: mknod d2/d15/c16 0 2024-04-22T23:45:34.570 INFO:tasks.workunit.client.0.smithi031.stdout:9/66: write d2/dc/f12 [1498054,30255] 0 2024-04-22T23:45:34.572 INFO:tasks.workunit.client.0.smithi031.stdout:5/46: symlink db/le 0 2024-04-22T23:45:34.574 INFO:tasks.workunit.client.0.smithi031.stdout:4/105: creat db/d14/d1d/f1f x:0 0 0 2024-04-22T23:45:34.574 INFO:tasks.workunit.client.0.smithi031.stdout:4/106: write db/d10/f12 [551567,89952] 0 2024-04-22T23:45:34.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/127: creat d3/d1f/f2f x:0 0 0 2024-04-22T23:45:34.578 INFO:tasks.workunit.client.0.smithi031.stdout:0/55: mkdir d0/d1/d2/d17 0 2024-04-22T23:45:34.578 INFO:tasks.workunit.client.0.smithi031.stdout:0/56: chown d0/d13 1205927732 1 2024-04-22T23:45:34.584 INFO:tasks.workunit.client.0.smithi031.stdout:4/107: creat db/d14/d1d/f20 x:0 0 0 2024-04-22T23:45:34.585 INFO:tasks.workunit.client.0.smithi031.stdout:6/128: rmdir d3/d2a 39 2024-04-22T23:45:34.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/129: read d3/f8 [3343816,129435] 0 2024-04-22T23:45:34.590 INFO:tasks.workunit.client.0.smithi031.stdout:9/67: truncate d2/f9 692930 0 2024-04-22T23:45:34.593 INFO:tasks.workunit.client.0.smithi031.stdout:0/57: mkdir d0/d13/d18 0 2024-04-22T23:45:34.595 INFO:tasks.workunit.client.0.smithi031.stdout:6/130: write d3/d1f/f22 [9229789,7218] 0 2024-04-22T23:45:34.598 INFO:tasks.workunit.client.0.smithi031.stdout:9/68: creat d2/dc/f17 x:0 0 0 2024-04-22T23:45:34.598 INFO:tasks.workunit.client.0.smithi031.stdout:5/47: link l1 db/lf 0 2024-04-22T23:45:34.599 INFO:tasks.workunit.client.0.smithi031.stdout:9/69: write d2/dd/f13 [477143,5514] 0 2024-04-22T23:45:34.599 INFO:tasks.workunit.client.0.smithi031.stdout:5/48: read f0 [2762231,102685] 0 2024-04-22T23:45:34.600 INFO:tasks.workunit.client.0.smithi031.stdout:8/30: rmdir d5 39 2024-04-22T23:45:34.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/58: symlink d0/d1/d2/d3/d11/l19 0 2024-04-22T23:45:34.603 INFO:tasks.workunit.client.0.smithi031.stdout:0/59: chown d0/d1/d2/d3/d11 367 1 2024-04-22T23:45:34.604 INFO:tasks.workunit.client.0.smithi031.stdout:0/60: write d0/ff [3126066,82571] 0 2024-04-22T23:45:34.608 INFO:tasks.workunit.client.0.smithi031.stdout:9/70: mkdir d2/dc/d18 0 2024-04-22T23:45:34.611 INFO:tasks.workunit.client.0.smithi031.stdout:7/69: dread f8 [0,4194304] 0 2024-04-22T23:45:34.614 INFO:tasks.workunit.client.0.smithi031.stdout:7/70: stat d9/d14/l16 0 2024-04-22T23:45:34.614 INFO:tasks.workunit.client.0.smithi031.stdout:6/131: creat d3/d2a/f30 x:0 0 0 2024-04-22T23:45:34.618 INFO:tasks.workunit.client.0.smithi031.stdout:9/71: creat d2/dd/f19 x:0 0 0 2024-04-22T23:45:34.620 INFO:tasks.workunit.client.0.smithi031.stdout:0/61: dwrite d0/d1/f10 [0,4194304] 0 2024-04-22T23:45:34.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/132: write d3/d1f/f22 [5077026,96257] 0 2024-04-22T23:45:34.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/49: dwrite f3 [0,4194304] 0 2024-04-22T23:45:34.626 INFO:tasks.workunit.client.0.smithi031.stdout:9/72: mknod d2/c1a 0 2024-04-22T23:45:34.627 INFO:tasks.workunit.client.0.smithi031.stdout:5/50: write db/fd [2849750,911] 0 2024-04-22T23:45:34.627 INFO:tasks.workunit.client.0.smithi031.stdout:9/73: fsync d2/f5 0 2024-04-22T23:45:34.634 INFO:tasks.workunit.client.0.smithi031.stdout:0/62: dread d0/d1/f10 [0,4194304] 0 2024-04-22T23:45:34.640 INFO:tasks.workunit.client.0.smithi031.stdout:7/71: link d9/cb d9/d14/c1b 0 2024-04-22T23:45:34.643 INFO:tasks.workunit.client.0.smithi031.stdout:9/74: dread d2/f5 [0,4194304] 0 2024-04-22T23:45:34.643 INFO:tasks.workunit.client.0.smithi031.stdout:5/51: dwrite f0 [0,4194304] 0 2024-04-22T23:45:34.643 INFO:tasks.workunit.client.0.smithi031.stdout:7/72: readlink d9/d14/l16 0 2024-04-22T23:45:34.653 INFO:tasks.workunit.client.0.smithi031.stdout:7/73: creat d9/f1c x:0 0 0 2024-04-22T23:45:34.653 INFO:tasks.workunit.client.0.smithi031.stdout:6/133: rename d3/l7 to d3/l31 0 2024-04-22T23:45:34.658 INFO:tasks.workunit.client.0.smithi031.stdout:9/75: dwrite d2/dc/f10 [4194304,4194304] 0 2024-04-22T23:45:34.660 INFO:tasks.workunit.client.0.smithi031.stdout:5/52: dwrite f0 [0,4194304] 0 2024-04-22T23:45:34.664 INFO:tasks.workunit.client.0.smithi031.stdout:5/53: write f0 [1858018,55395] 0 2024-04-22T23:45:34.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/74: mknod d9/d15/c1d 0 2024-04-22T23:45:34.667 INFO:tasks.workunit.client.0.smithi031.stdout:9/76: mkdir d2/dd/d1b 0 2024-04-22T23:45:34.669 INFO:tasks.workunit.client.0.smithi031.stdout:0/63: getdents d0 0 2024-04-22T23:45:34.674 INFO:tasks.workunit.client.0.smithi031.stdout:5/54: dread f5 [0,4194304] 0 2024-04-22T23:45:34.676 INFO:tasks.workunit.client.0.smithi031.stdout:0/64: dread d0/d1/fc [0,4194304] 0 2024-04-22T23:45:34.678 INFO:tasks.workunit.client.0.smithi031.stdout:9/77: link d2/f8 d2/d15/f1c 0 2024-04-22T23:45:34.682 INFO:tasks.workunit.client.0.smithi031.stdout:0/65: unlink d0/d1/f15 0 2024-04-22T23:45:34.683 INFO:tasks.workunit.client.0.smithi031.stdout:8/31: dwrite f4 [0,4194304] 0 2024-04-22T23:45:34.691 INFO:tasks.workunit.client.0.smithi031.stdout:9/78: dwrite d2/fb [0,4194304] 0 2024-04-22T23:45:34.694 INFO:tasks.workunit.client.0.smithi031.stdout:7/75: getdents d9/d14 0 2024-04-22T23:45:34.698 INFO:tasks.workunit.client.0.smithi031.stdout:4/108: write f1 [1291940,116686] 0 2024-04-22T23:45:34.698 INFO:tasks.workunit.client.0.smithi031.stdout:9/79: mkdir d2/d1d 0 2024-04-22T23:45:34.700 INFO:tasks.workunit.client.0.smithi031.stdout:0/66: dwrite d0/d1/fb [0,4194304] 0 2024-04-22T23:45:34.700 INFO:tasks.workunit.client.0.smithi031.stdout:3/30: sync 2024-04-22T23:45:34.700 INFO:tasks.workunit.client.0.smithi031.stdout:1/35: sync 2024-04-22T23:45:34.702 INFO:tasks.workunit.client.0.smithi031.stdout:7/76: symlink d9/l1e 0 2024-04-22T23:45:34.703 INFO:tasks.workunit.client.0.smithi031.stdout:9/80: truncate d2/dc/f12 4308905 0 2024-04-22T23:45:34.704 INFO:tasks.workunit.client.0.smithi031.stdout:8/32: link d5/f6 d5/f9 0 2024-04-22T23:45:34.706 INFO:tasks.workunit.client.0.smithi031.stdout:2/73: write f4 [1142650,40850] 0 2024-04-22T23:45:34.706 INFO:tasks.workunit.client.0.smithi031.stdout:8/33: dread d5/f6 [0,4194304] 0 2024-04-22T23:45:34.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/134: rmdir d3 39 2024-04-22T23:45:34.713 INFO:tasks.workunit.client.0.smithi031.stdout:1/36: mknod d1/cd 0 2024-04-22T23:45:34.715 INFO:tasks.workunit.client.0.smithi031.stdout:1/37: write d1/f8 [640803,49896] 0 2024-04-22T23:45:34.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/77: rmdir d9/d14 39 2024-04-22T23:45:34.715 INFO:tasks.workunit.client.0.smithi031.stdout:3/31: dwrite f3 [0,4194304] 0 2024-04-22T23:45:34.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/38: write d1/f6 [1063862,12897] 0 2024-04-22T23:45:34.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/67: mkdir d0/d1/d2/d3/d11/d1a 0 2024-04-22T23:45:34.720 INFO:tasks.workunit.client.0.smithi031.stdout:9/81: creat d2/dc/f1e x:0 0 0 2024-04-22T23:45:34.724 INFO:tasks.workunit.client.0.smithi031.stdout:1/39: dwrite d1/f8 [0,4194304] 0 2024-04-22T23:45:34.821 INFO:tasks.workunit.client.0.smithi031.stdout:2/74: mknod d6/c12 0 2024-04-22T23:45:34.827 INFO:tasks.workunit.client.0.smithi031.stdout:8/34: mkdir d5/da 0 2024-04-22T23:45:34.827 INFO:tasks.workunit.client.0.smithi031.stdout:5/55: truncate f7 451387 0 2024-04-22T23:45:34.829 INFO:tasks.workunit.client.0.smithi031.stdout:9/82: symlink d2/d15/l1f 0 2024-04-22T23:45:34.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/40: mkdir d1/de 0 2024-04-22T23:45:34.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/109: truncate db/fc 476567 0 2024-04-22T23:45:34.832 INFO:tasks.workunit.client.0.smithi031.stdout:1/41: chown d1/f8 2075297643 1 2024-04-22T23:45:34.834 INFO:tasks.workunit.client.0.smithi031.stdout:2/75: symlink d6/dc/l13 0 2024-04-22T23:45:34.835 INFO:tasks.workunit.client.0.smithi031.stdout:1/42: fdatasync d1/f8 0 2024-04-22T23:45:34.836 INFO:tasks.workunit.client.0.smithi031.stdout:7/78: mkdir d9/d14/d1f 0 2024-04-22T23:45:34.836 INFO:tasks.workunit.client.0.smithi031.stdout:1/43: chown d1 1136611 1 2024-04-22T23:45:34.837 INFO:tasks.workunit.client.0.smithi031.stdout:6/135: fsync d3/d1f/f28 0 2024-04-22T23:45:34.838 INFO:tasks.workunit.client.0.smithi031.stdout:2/76: write f4 [326079,75991] 0 2024-04-22T23:45:34.839 INFO:tasks.workunit.client.0.smithi031.stdout:7/79: write f3 [2012599,128277] 0 2024-04-22T23:45:34.841 INFO:tasks.workunit.client.0.smithi031.stdout:5/56: dwrite f8 [0,4194304] 0 2024-04-22T23:45:34.841 INFO:tasks.workunit.client.0.smithi031.stdout:0/68: dwrite d0/d1/fc [0,4194304] 0 2024-04-22T23:45:34.842 INFO:tasks.workunit.client.0.smithi031.stdout:7/80: dread - d9/f1c zero size 2024-04-22T23:45:34.845 INFO:tasks.workunit.client.0.smithi031.stdout:2/77: truncate d6/ff 852598 0 2024-04-22T23:45:34.845 INFO:tasks.workunit.client.0.smithi031.stdout:7/81: write d9/fa [1276197,93771] 0 2024-04-22T23:45:34.851 INFO:tasks.workunit.client.0.smithi031.stdout:0/69: dwrite d0/d1/fb [0,4194304] 0 2024-04-22T23:45:34.853 INFO:tasks.workunit.client.0.smithi031.stdout:6/136: readlink d3/l31 0 2024-04-22T23:45:34.858 INFO:tasks.workunit.client.0.smithi031.stdout:5/57: readlink db/lf 0 2024-04-22T23:45:34.863 INFO:tasks.workunit.client.0.smithi031.stdout:9/83: rmdir d2/d1d 0 2024-04-22T23:45:34.869 INFO:tasks.workunit.client.0.smithi031.stdout:7/82: creat d9/d14/f20 x:0 0 0 2024-04-22T23:45:34.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/44: creat d1/dc/ff x:0 0 0 2024-04-22T23:45:34.878 INFO:tasks.workunit.client.0.smithi031.stdout:5/58: unlink f3 0 2024-04-22T23:45:34.882 INFO:tasks.workunit.client.0.smithi031.stdout:0/70: mknod d0/d1/d2/d17/c1b 0 2024-04-22T23:45:34.885 INFO:tasks.workunit.client.0.smithi031.stdout:1/45: mknod d1/dc/c10 0 2024-04-22T23:45:34.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/46: stat d1/l4 0 2024-04-22T23:45:34.891 INFO:tasks.workunit.client.0.smithi031.stdout:0/71: mkdir d0/d1/d2/d3/d11/d1c 0 2024-04-22T23:45:34.892 INFO:tasks.workunit.client.0.smithi031.stdout:1/47: dwrite d1/dc/ff [0,4194304] 0 2024-04-22T23:45:34.893 INFO:tasks.workunit.client.0.smithi031.stdout:6/137: creat d3/f32 x:0 0 0 2024-04-22T23:45:34.894 INFO:tasks.workunit.client.0.smithi031.stdout:5/59: fsync f0 0 2024-04-22T23:45:34.896 INFO:tasks.workunit.client.0.smithi031.stdout:2/78: rmdir d6 39 2024-04-22T23:45:34.899 INFO:tasks.workunit.client.0.smithi031.stdout:6/138: symlink d3/d1f/l33 0 2024-04-22T23:45:34.901 INFO:tasks.workunit.client.0.smithi031.stdout:8/35: write d5/f9 [717819,72780] 0 2024-04-22T23:45:34.901 INFO:tasks.workunit.client.0.smithi031.stdout:5/60: dread f5 [0,4194304] 0 2024-04-22T23:45:34.901 INFO:tasks.workunit.client.0.smithi031.stdout:6/139: fdatasync d3/d1f/f28 0 2024-04-22T23:45:34.902 INFO:tasks.workunit.client.0.smithi031.stdout:1/48: creat d1/de/f11 x:0 0 0 2024-04-22T23:45:34.904 INFO:tasks.workunit.client.0.smithi031.stdout:5/61: read f5 [4009035,73576] 0 2024-04-22T23:45:34.906 INFO:tasks.workunit.client.0.smithi031.stdout:1/49: write d1/de/f11 [938960,15083] 0 2024-04-22T23:45:34.907 INFO:tasks.workunit.client.0.smithi031.stdout:2/79: dwrite f4 [0,4194304] 0 2024-04-22T23:45:34.907 INFO:tasks.workunit.client.0.smithi031.stdout:1/50: readlink d1/l7 0 2024-04-22T23:45:34.911 INFO:tasks.workunit.client.0.smithi031.stdout:5/62: dread db/fd [0,4194304] 0 2024-04-22T23:45:34.914 INFO:tasks.workunit.client.0.smithi031.stdout:5/63: dread f5 [0,4194304] 0 2024-04-22T23:45:34.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/51: creat d1/de/f12 x:0 0 0 2024-04-22T23:45:34.918 INFO:tasks.workunit.client.0.smithi031.stdout:6/140: creat d3/f34 x:0 0 0 2024-04-22T23:45:34.921 INFO:tasks.workunit.client.0.smithi031.stdout:5/64: mkdir db/d10 0 2024-04-22T23:45:34.927 INFO:tasks.workunit.client.0.smithi031.stdout:5/65: mkdir db/d11 0 2024-04-22T23:45:34.927 INFO:tasks.workunit.client.0.smithi031.stdout:5/66: rename db to db/d10/d12 22 2024-04-22T23:45:34.932 INFO:tasks.workunit.client.0.smithi031.stdout:6/141: mkdir d3/d35 0 2024-04-22T23:45:34.935 INFO:tasks.workunit.client.0.smithi031.stdout:9/84: write d2/f8 [3342397,121760] 0 2024-04-22T23:45:34.936 INFO:tasks.workunit.client.0.smithi031.stdout:6/142: fsync d3/f17 0 2024-04-22T23:45:34.936 INFO:tasks.workunit.client.0.smithi031.stdout:7/83: write f8 [972055,86545] 0 2024-04-22T23:45:34.938 INFO:tasks.workunit.client.0.smithi031.stdout:2/80: getdents d6 0 2024-04-22T23:45:34.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/143: truncate d3/d1f/f29 562409 0 2024-04-22T23:45:34.939 INFO:tasks.workunit.client.0.smithi031.stdout:5/67: mknod db/d11/c13 0 2024-04-22T23:45:34.940 INFO:tasks.workunit.client.0.smithi031.stdout:5/68: truncate db/fd 4818471 0 2024-04-22T23:45:34.941 INFO:tasks.workunit.client.0.smithi031.stdout:9/85: creat d2/f20 x:0 0 0 2024-04-22T23:45:34.945 INFO:tasks.workunit.client.0.smithi031.stdout:7/84: rename d9/f10 to d9/d15/f21 0 2024-04-22T23:45:34.950 INFO:tasks.workunit.client.0.smithi031.stdout:9/86: mkdir d2/dc/d21 0 2024-04-22T23:45:34.953 INFO:tasks.workunit.client.0.smithi031.stdout:6/144: rename d3/d1f/l33 to d3/d35/l36 0 2024-04-22T23:45:34.958 INFO:tasks.workunit.client.0.smithi031.stdout:9/87: symlink d2/d15/l22 0 2024-04-22T23:45:34.958 INFO:tasks.workunit.client.0.smithi031.stdout:4/110: write db/fc [12057,73159] 0 2024-04-22T23:45:34.960 INFO:tasks.workunit.client.0.smithi031.stdout:4/111: truncate db/d14/d1d/f20 722631 0 2024-04-22T23:45:34.963 INFO:tasks.workunit.client.0.smithi031.stdout:5/69: rmdir db/d10 0 2024-04-22T23:45:34.964 INFO:tasks.workunit.client.0.smithi031.stdout:9/88: symlink d2/dc/l23 0 2024-04-22T23:45:34.966 INFO:tasks.workunit.client.0.smithi031.stdout:9/89: truncate d2/fa 1494950 0 2024-04-22T23:45:34.967 INFO:tasks.workunit.client.0.smithi031.stdout:6/145: dwrite d3/f14 [0,4194304] 0 2024-04-22T23:45:34.968 INFO:tasks.workunit.client.0.smithi031.stdout:4/112: symlink db/d14/l21 0 2024-04-22T23:45:34.970 INFO:tasks.workunit.client.0.smithi031.stdout:5/70: creat db/f14 x:0 0 0 2024-04-22T23:45:34.970 INFO:tasks.workunit.client.0.smithi031.stdout:5/71: chown db/f14 9741009 1 2024-04-22T23:45:34.971 INFO:tasks.workunit.client.0.smithi031.stdout:1/52: truncate d1/dc/ff 3946385 0 2024-04-22T23:45:34.971 INFO:tasks.workunit.client.0.smithi031.stdout:1/53: fdatasync d1/f8 0 2024-04-22T23:45:34.972 INFO:tasks.workunit.client.0.smithi031.stdout:9/90: mkdir d2/dc/d24 0 2024-04-22T23:45:34.972 INFO:tasks.workunit.client.0.smithi031.stdout:1/54: dread - d1/de/f12 zero size 2024-04-22T23:45:34.976 INFO:tasks.workunit.client.0.smithi031.stdout:4/113: rmdir db 39 2024-04-22T23:45:34.978 INFO:tasks.workunit.client.0.smithi031.stdout:6/146: dread d3/f11 [0,4194304] 0 2024-04-22T23:45:34.979 INFO:tasks.workunit.client.0.smithi031.stdout:6/147: chown d3/d35 118261896 1 2024-04-22T23:45:34.979 INFO:tasks.workunit.client.0.smithi031.stdout:9/91: creat d2/dc/f25 x:0 0 0 2024-04-22T23:45:34.984 INFO:tasks.workunit.client.0.smithi031.stdout:4/114: dwrite f1 [0,4194304] 0 2024-04-22T23:45:34.991 INFO:tasks.workunit.client.0.smithi031.stdout:9/92: symlink d2/l26 0 2024-04-22T23:45:34.993 INFO:tasks.workunit.client.0.smithi031.stdout:2/81: dwrite f3 [4194304,4194304] 0 2024-04-22T23:45:34.996 INFO:tasks.workunit.client.0.smithi031.stdout:2/82: write d6/f11 [503322,43033] 0 2024-04-22T23:45:34.999 INFO:tasks.workunit.client.0.smithi031.stdout:6/148: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:45:35.002 INFO:tasks.workunit.client.0.smithi031.stdout:9/93: dread d2/dc/f10 [0,4194304] 0 2024-04-22T23:45:35.006 INFO:tasks.workunit.client.0.smithi031.stdout:4/115: dwrite db/d14/d1d/f20 [0,4194304] 0 2024-04-22T23:45:35.013 INFO:tasks.workunit.client.0.smithi031.stdout:9/94: rmdir d2/dd 39 2024-04-22T23:45:35.028 INFO:tasks.workunit.client.0.smithi031.stdout:2/83: link d6/c12 d6/dc/c14 0 2024-04-22T23:45:35.029 INFO:tasks.workunit.client.0.smithi031.stdout:5/72: truncate f0 2708652 0 2024-04-22T23:45:35.032 INFO:tasks.workunit.client.0.smithi031.stdout:9/95: dwrite d2/dd/f13 [0,4194304] 0 2024-04-22T23:45:35.033 INFO:tasks.workunit.client.0.smithi031.stdout:4/116: rmdir db/d14/d1d/d1e 0 2024-04-22T23:45:35.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/84: creat d6/dc/f15 x:0 0 0 2024-04-22T23:45:35.037 INFO:tasks.workunit.client.0.smithi031.stdout:6/149: getdents d3/d35 0 2024-04-22T23:45:35.038 INFO:tasks.workunit.client.0.smithi031.stdout:5/73: creat db/d11/f15 x:0 0 0 2024-04-22T23:45:35.042 INFO:tasks.workunit.client.0.smithi031.stdout:4/117: readlink l5 0 2024-04-22T23:45:35.043 INFO:tasks.workunit.client.0.smithi031.stdout:9/96: dwrite d2/fb [4194304,4194304] 0 2024-04-22T23:45:35.044 INFO:tasks.workunit.client.0.smithi031.stdout:6/150: readlink d3/l1b 0 2024-04-22T23:45:35.046 INFO:tasks.workunit.client.0.smithi031.stdout:4/118: fsync db/d14/d1d/f20 0 2024-04-22T23:45:35.048 INFO:tasks.workunit.client.0.smithi031.stdout:2/85: dwrite d6/ff [0,4194304] 0 2024-04-22T23:45:35.051 INFO:tasks.workunit.client.0.smithi031.stdout:6/151: unlink d3/d1f/f29 0 2024-04-22T23:45:35.052 INFO:tasks.workunit.client.0.smithi031.stdout:9/97: creat d2/dd/d1b/f27 x:0 0 0 2024-04-22T23:45:35.052 INFO:tasks.workunit.client.0.smithi031.stdout:4/119: dread f4 [0,4194304] 0 2024-04-22T23:45:35.054 INFO:tasks.workunit.client.0.smithi031.stdout:9/98: write d2/dc/f17 [250498,23640] 0 2024-04-22T23:45:35.054 INFO:tasks.workunit.client.0.smithi031.stdout:2/86: creat d6/f16 x:0 0 0 2024-04-22T23:45:35.056 INFO:tasks.workunit.client.0.smithi031.stdout:6/152: dread - d3/f18 zero size 2024-04-22T23:45:35.058 INFO:tasks.workunit.client.0.smithi031.stdout:2/87: mkdir d6/dc/d17 0 2024-04-22T23:45:35.061 INFO:tasks.workunit.client.0.smithi031.stdout:9/99: creat d2/dd/f28 x:0 0 0 2024-04-22T23:45:35.063 INFO:tasks.workunit.client.0.smithi031.stdout:6/153: mknod d3/d35/c37 0 2024-04-22T23:45:35.099 INFO:tasks.workunit.client.0.smithi031.stdout:3/32: sync 2024-04-22T23:45:35.120 INFO:tasks.workunit.client.0.smithi031.stdout:9/100: truncate d2/f5 7985076 0 2024-04-22T23:45:35.122 INFO:tasks.workunit.client.0.smithi031.stdout:5/74: rename db/d11 to db/d16 0 2024-04-22T23:45:35.128 INFO:tasks.workunit.client.0.smithi031.stdout:6/154: creat d3/f38 x:0 0 0 2024-04-22T23:45:35.129 INFO:tasks.workunit.client.0.smithi031.stdout:9/101: write d2/dc/f10 [2315593,127135] 0 2024-04-22T23:45:35.131 INFO:tasks.workunit.client.0.smithi031.stdout:9/102: fdatasync d2/dc/f12 0 2024-04-22T23:45:35.132 INFO:tasks.workunit.client.0.smithi031.stdout:4/120: truncate db/d14/d1d/f20 2250387 0 2024-04-22T23:45:35.132 INFO:tasks.workunit.client.0.smithi031.stdout:5/75: dwrite db/f14 [0,4194304] 0 2024-04-22T23:45:35.137 INFO:tasks.workunit.client.0.smithi031.stdout:9/103: dread d2/fb [0,4194304] 0 2024-04-22T23:45:35.138 INFO:tasks.workunit.client.0.smithi031.stdout:4/121: dread db/d10/f12 [4194304,4194304] 0 2024-04-22T23:45:35.139 INFO:tasks.workunit.client.0.smithi031.stdout:4/122: dread - db/d14/d1d/f1f zero size 2024-04-22T23:45:35.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/76: symlink db/l17 0 2024-04-22T23:45:35.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/77: chown l1 3822740 1 2024-04-22T23:45:35.140 INFO:tasks.workunit.client.0.smithi031.stdout:4/123: stat db/d14/l21 0 2024-04-22T23:45:35.141 INFO:tasks.workunit.client.0.smithi031.stdout:5/78: write f8 [1567488,59495] 0 2024-04-22T23:45:35.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/104: read d2/f9 [404740,81873] 0 2024-04-22T23:45:35.145 INFO:tasks.workunit.client.0.smithi031.stdout:9/105: fdatasync d2/dc/f1e 0 2024-04-22T23:45:35.151 INFO:tasks.workunit.client.0.smithi031.stdout:4/124: write db/d10/f12 [3370695,108936] 0 2024-04-22T23:45:35.155 INFO:tasks.workunit.client.0.smithi031.stdout:5/79: unlink f8 0 2024-04-22T23:45:35.158 INFO:tasks.workunit.client.0.smithi031.stdout:4/125: symlink db/d10/l22 0 2024-04-22T23:45:35.162 INFO:tasks.workunit.client.0.smithi031.stdout:9/106: mknod d2/c29 0 2024-04-22T23:45:35.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/107: fdatasync d2/f5 0 2024-04-22T23:45:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/108: dread - d2/dc/f25 zero size 2024-04-22T23:45:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:7/85: sync 2024-04-22T23:45:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:0/72: sync 2024-04-22T23:45:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:8/36: sync 2024-04-22T23:45:35.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/55: sync 2024-04-22T23:45:35.168 INFO:tasks.workunit.client.0.smithi031.stdout:9/109: readlink d2/dd/l11 0 2024-04-22T23:45:35.168 INFO:tasks.workunit.client.0.smithi031.stdout:4/126: symlink db/l23 0 2024-04-22T23:45:35.169 INFO:tasks.workunit.client.0.smithi031.stdout:1/56: write d1/f8 [35885,2897] 0 2024-04-22T23:45:35.170 INFO:tasks.workunit.client.0.smithi031.stdout:8/37: rmdir d5 39 2024-04-22T23:45:35.171 INFO:tasks.workunit.client.0.smithi031.stdout:0/73: dread - d0/d1/d2/d3/d11/f14 zero size 2024-04-22T23:45:35.173 INFO:tasks.workunit.client.0.smithi031.stdout:4/127: unlink db/d14/d1d/f1f 0 2024-04-22T23:45:35.175 INFO:tasks.workunit.client.0.smithi031.stdout:1/57: fdatasync d1/f8 0 2024-04-22T23:45:35.175 INFO:tasks.workunit.client.0.smithi031.stdout:9/110: rename d2/dc/d18 to d2/dc/d24/d2a 0 2024-04-22T23:45:35.184 INFO:tasks.workunit.client.0.smithi031.stdout:0/74: rename d0/d1/fb to d0/d1/d2/d3/f1d 0 2024-04-22T23:45:35.186 INFO:tasks.workunit.client.0.smithi031.stdout:0/75: dread - d0/d1/d2/d3/d11/f14 zero size 2024-04-22T23:45:35.187 INFO:tasks.workunit.client.0.smithi031.stdout:9/111: creat d2/dd/f2b x:0 0 0 2024-04-22T23:45:35.187 INFO:tasks.workunit.client.0.smithi031.stdout:1/58: dread d1/dc/ff [0,4194304] 0 2024-04-22T23:45:35.188 INFO:tasks.workunit.client.0.smithi031.stdout:9/112: readlink d2/d15/l1f 0 2024-04-22T23:45:35.189 INFO:tasks.workunit.client.0.smithi031.stdout:8/38: creat d5/fb x:0 0 0 2024-04-22T23:45:35.192 INFO:tasks.workunit.client.0.smithi031.stdout:0/76: write d0/d1/d2/f9 [3560890,35445] 0 2024-04-22T23:45:35.194 INFO:tasks.workunit.client.0.smithi031.stdout:9/113: write d2/fb [7982465,9632] 0 2024-04-22T23:45:35.196 INFO:tasks.workunit.client.0.smithi031.stdout:1/59: mknod d1/dc/c13 0 2024-04-22T23:45:35.197 INFO:tasks.workunit.client.0.smithi031.stdout:8/39: truncate f0 755523 0 2024-04-22T23:45:35.197 INFO:tasks.workunit.client.0.smithi031.stdout:8/40: readlink - no filename 2024-04-22T23:45:35.202 INFO:tasks.workunit.client.0.smithi031.stdout:1/60: dread d1/de/f11 [0,4194304] 0 2024-04-22T23:45:35.204 INFO:tasks.workunit.client.0.smithi031.stdout:8/41: creat d5/fc x:0 0 0 2024-04-22T23:45:35.205 INFO:tasks.workunit.client.0.smithi031.stdout:8/42: dread - d5/fb zero size 2024-04-22T23:45:35.205 INFO:tasks.workunit.client.0.smithi031.stdout:8/43: dread - d5/fb zero size 2024-04-22T23:45:35.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/44: write d5/f6 [282692,19825] 0 2024-04-22T23:45:35.224 INFO:tasks.workunit.client.0.smithi031.stdout:3/33: truncate f3 3338453 0 2024-04-22T23:45:35.233 INFO:tasks.workunit.client.0.smithi031.stdout:6/155: write d3/fe [2295007,19194] 0 2024-04-22T23:45:35.235 INFO:tasks.workunit.client.0.smithi031.stdout:2/88: rmdir d6 39 2024-04-22T23:45:35.238 INFO:tasks.workunit.client.0.smithi031.stdout:2/89: stat d6/dc/f15 0 2024-04-22T23:45:35.243 INFO:tasks.workunit.client.0.smithi031.stdout:6/156: creat d3/f39 x:0 0 0 2024-04-22T23:45:35.244 INFO:tasks.workunit.client.0.smithi031.stdout:2/90: unlink d6/dc/c14 0 2024-04-22T23:45:35.245 INFO:tasks.workunit.client.0.smithi031.stdout:2/91: stat l5 0 2024-04-22T23:45:35.247 INFO:tasks.workunit.client.0.smithi031.stdout:6/157: mknod d3/d35/c3a 0 2024-04-22T23:45:35.248 INFO:tasks.workunit.client.0.smithi031.stdout:6/158: truncate d3/f39 767398 0 2024-04-22T23:45:35.250 INFO:tasks.workunit.client.0.smithi031.stdout:2/92: mknod d6/dc/d17/c18 0 2024-04-22T23:45:35.252 INFO:tasks.workunit.client.0.smithi031.stdout:2/93: creat d6/f19 x:0 0 0 2024-04-22T23:45:35.256 INFO:tasks.workunit.client.0.smithi031.stdout:2/94: creat d6/dc/f1a x:0 0 0 2024-04-22T23:45:35.268 INFO:tasks.workunit.client.0.smithi031.stdout:0/77: fsync d0/d1/d2/d3/f1d 0 2024-04-22T23:45:35.271 INFO:tasks.workunit.client.0.smithi031.stdout:0/78: mkdir d0/d13/d18/d1e 0 2024-04-22T23:45:35.274 INFO:tasks.workunit.client.0.smithi031.stdout:0/79: dread d0/d1/f10 [0,4194304] 0 2024-04-22T23:45:35.275 INFO:tasks.workunit.client.0.smithi031.stdout:1/61: rename d1/dc to d1/db/d14 0 2024-04-22T23:45:35.276 INFO:tasks.workunit.client.0.smithi031.stdout:8/45: dread f0 [0,4194304] 0 2024-04-22T23:45:35.277 INFO:tasks.workunit.client.0.smithi031.stdout:1/62: mknod d1/de/c15 0 2024-04-22T23:45:35.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/63: fdatasync d1/de/f12 0 2024-04-22T23:45:35.278 INFO:tasks.workunit.client.0.smithi031.stdout:0/80: symlink d0/d1/d2/l1f 0 2024-04-22T23:45:35.279 INFO:tasks.workunit.client.0.smithi031.stdout:0/81: read d0/d1/fc [2518429,19503] 0 2024-04-22T23:45:35.279 INFO:tasks.workunit.client.0.smithi031.stdout:0/82: fsync d0/ff 0 2024-04-22T23:45:35.280 INFO:tasks.workunit.client.0.smithi031.stdout:0/83: chown d0/d13 1763261 1 2024-04-22T23:45:35.282 INFO:tasks.workunit.client.0.smithi031.stdout:1/64: mkdir d1/db/d16 0 2024-04-22T23:45:35.286 INFO:tasks.workunit.client.0.smithi031.stdout:8/46: creat d5/da/fd x:0 0 0 2024-04-22T23:45:35.286 INFO:tasks.workunit.client.0.smithi031.stdout:8/47: chown c2 16 1 2024-04-22T23:45:35.287 INFO:tasks.workunit.client.0.smithi031.stdout:8/48: truncate d5/f8 450848 0 2024-04-22T23:45:35.287 INFO:tasks.workunit.client.0.smithi031.stdout:0/84: unlink d0/d1/d2/l1f 0 2024-04-22T23:45:35.287 INFO:tasks.workunit.client.0.smithi031.stdout:8/49: chown d5/f8 11820 1 2024-04-22T23:45:35.289 INFO:tasks.workunit.client.0.smithi031.stdout:8/50: dread f0 [0,4194304] 0 2024-04-22T23:45:35.293 INFO:tasks.workunit.client.0.smithi031.stdout:5/80: chown f5 1 1 2024-04-22T23:45:35.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/65: symlink d1/db/d16/l17 0 2024-04-22T23:45:35.297 INFO:tasks.workunit.client.0.smithi031.stdout:8/51: dwrite d5/f9 [0,4194304] 0 2024-04-22T23:45:35.298 INFO:tasks.workunit.client.0.smithi031.stdout:5/81: rename db/f14 to db/d16/f18 0 2024-04-22T23:45:35.302 INFO:tasks.workunit.client.0.smithi031.stdout:9/114: dwrite d2/f5 [0,4194304] 0 2024-04-22T23:45:35.304 INFO:tasks.workunit.client.0.smithi031.stdout:8/52: symlink d5/le 0 2024-04-22T23:45:35.305 INFO:tasks.workunit.client.0.smithi031.stdout:1/66: rename d1/cd to d1/de/c18 0 2024-04-22T23:45:35.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/53: write d5/f9 [2229579,129310] 0 2024-04-22T23:45:35.307 INFO:tasks.workunit.client.0.smithi031.stdout:8/54: readlink d5/le 0 2024-04-22T23:45:35.308 INFO:tasks.workunit.client.0.smithi031.stdout:3/34: dread f3 [0,4194304] 0 2024-04-22T23:45:35.308 INFO:tasks.workunit.client.0.smithi031.stdout:9/115: unlink d2/d15/f1c 0 2024-04-22T23:45:35.308 INFO:tasks.workunit.client.0.smithi031.stdout:8/55: dread - d5/fb zero size 2024-04-22T23:45:35.309 INFO:tasks.workunit.client.0.smithi031.stdout:3/35: stat l1 0 2024-04-22T23:45:35.310 INFO:tasks.workunit.client.0.smithi031.stdout:8/56: write d5/fb [952454,95671] 0 2024-04-22T23:45:35.310 INFO:tasks.workunit.client.0.smithi031.stdout:4/128: truncate db/d14/d1d/f20 466525 0 2024-04-22T23:45:35.311 INFO:tasks.workunit.client.0.smithi031.stdout:1/67: rmdir d1/db 39 2024-04-22T23:45:35.311 INFO:tasks.workunit.client.0.smithi031.stdout:8/57: chown d5/le 2696162 1 2024-04-22T23:45:35.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/116: dread d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:35.315 INFO:tasks.workunit.client.0.smithi031.stdout:8/58: fsync d5/f6 0 2024-04-22T23:45:35.320 INFO:tasks.workunit.client.0.smithi031.stdout:9/117: unlink d2/c14 0 2024-04-22T23:45:35.323 INFO:tasks.workunit.client.0.smithi031.stdout:8/59: mkdir d5/da/df 0 2024-04-22T23:45:35.325 INFO:tasks.workunit.client.0.smithi031.stdout:1/68: creat d1/db/f19 x:0 0 0 2024-04-22T23:45:35.326 INFO:tasks.workunit.client.0.smithi031.stdout:1/69: write d1/de/f12 [741186,105877] 0 2024-04-22T23:45:35.327 INFO:tasks.workunit.client.0.smithi031.stdout:9/118: unlink d2/f9 0 2024-04-22T23:45:35.328 INFO:tasks.workunit.client.0.smithi031.stdout:9/119: dread - d2/dc/f25 zero size 2024-04-22T23:45:35.329 INFO:tasks.workunit.client.0.smithi031.stdout:9/120: chown d2/dc/f1e 42800221 1 2024-04-22T23:45:35.330 INFO:tasks.workunit.client.0.smithi031.stdout:2/95: rmdir d6/dc 39 2024-04-22T23:45:35.330 INFO:tasks.workunit.client.0.smithi031.stdout:4/129: creat db/f24 x:0 0 0 2024-04-22T23:45:35.330 INFO:tasks.workunit.client.0.smithi031.stdout:4/130: fsync db/fc 0 2024-04-22T23:45:35.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/96: write f3 [5414090,95481] 0 2024-04-22T23:45:35.331 INFO:tasks.workunit.client.0.smithi031.stdout:8/60: symlink d5/l10 0 2024-04-22T23:45:35.336 INFO:tasks.workunit.client.0.smithi031.stdout:9/121: rename d2/fa to d2/d15/f2c 0 2024-04-22T23:45:35.344 INFO:tasks.workunit.client.0.smithi031.stdout:8/61: mknod d5/da/c11 0 2024-04-22T23:45:35.344 INFO:tasks.workunit.client.0.smithi031.stdout:8/62: readlink d5/l10 0 2024-04-22T23:45:35.347 INFO:tasks.workunit.client.0.smithi031.stdout:2/97: rmdir d6/dc/d17 39 2024-04-22T23:45:35.348 INFO:tasks.workunit.client.0.smithi031.stdout:8/63: mkdir d5/da/d12 0 2024-04-22T23:45:35.349 INFO:tasks.workunit.client.0.smithi031.stdout:2/98: readlink d6/dc/le 0 2024-04-22T23:45:35.353 INFO:tasks.workunit.client.0.smithi031.stdout:0/85: rmdir d0/d1/d2 39 2024-04-22T23:45:35.353 INFO:tasks.workunit.client.0.smithi031.stdout:8/64: dread d5/fb [0,4194304] 0 2024-04-22T23:45:35.354 INFO:tasks.workunit.client.0.smithi031.stdout:0/86: fdatasync d0/ff 0 2024-04-22T23:45:35.357 INFO:tasks.workunit.client.0.smithi031.stdout:2/99: unlink d6/c12 0 2024-04-22T23:45:35.365 INFO:tasks.workunit.client.0.smithi031.stdout:0/87: dwrite d0/d1/d2/f9 [0,4194304] 0 2024-04-22T23:45:35.365 INFO:tasks.workunit.client.0.smithi031.stdout:2/100: mknod d6/dc/d17/c1b 0 2024-04-22T23:45:35.367 INFO:tasks.workunit.client.0.smithi031.stdout:2/101: write d6/dc/f1a [913349,85689] 0 2024-04-22T23:45:35.372 INFO:tasks.workunit.client.0.smithi031.stdout:0/88: symlink d0/d1/d2/d3/d11/d1c/l20 0 2024-04-22T23:45:35.372 INFO:tasks.workunit.client.0.smithi031.stdout:0/89: dread - d0/d1/d2/d3/d11/f14 zero size 2024-04-22T23:45:35.378 INFO:tasks.workunit.client.0.smithi031.stdout:2/102: creat d6/f1c x:0 0 0 2024-04-22T23:45:35.380 INFO:tasks.workunit.client.0.smithi031.stdout:1/70: rmdir d1 39 2024-04-22T23:45:35.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/103: mknod d6/dc/d17/c1d 0 2024-04-22T23:45:35.383 INFO:tasks.workunit.client.0.smithi031.stdout:0/90: link d0/d1/d2/ce d0/d13/c21 0 2024-04-22T23:45:35.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/131: rmdir db 39 2024-04-22T23:45:35.387 INFO:tasks.workunit.client.0.smithi031.stdout:8/65: getdents d5 0 2024-04-22T23:45:35.387 INFO:tasks.workunit.client.0.smithi031.stdout:0/91: rmdir d0 39 2024-04-22T23:45:35.388 INFO:tasks.workunit.client.0.smithi031.stdout:9/122: getdents d2/d15 0 2024-04-22T23:45:35.389 INFO:tasks.workunit.client.0.smithi031.stdout:9/123: chown d2/dc 13300045 1 2024-04-22T23:45:35.390 INFO:tasks.workunit.client.0.smithi031.stdout:9/124: fdatasync d2/f20 0 2024-04-22T23:45:35.390 INFO:tasks.workunit.client.0.smithi031.stdout:4/132: chown db/c16 390582482 1 2024-04-22T23:45:35.390 INFO:tasks.workunit.client.0.smithi031.stdout:2/104: dwrite d6/f19 [0,4194304] 0 2024-04-22T23:45:35.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/125: write d2/dc/f12 [1027185,100687] 0 2024-04-22T23:45:35.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/71: creat d1/db/d14/f1a x:0 0 0 2024-04-22T23:45:35.393 INFO:tasks.workunit.client.0.smithi031.stdout:2/105: write d6/dc/f15 [739613,436] 0 2024-04-22T23:45:35.395 INFO:tasks.workunit.client.0.smithi031.stdout:1/72: write d1/de/f12 [450442,43730] 0 2024-04-22T23:45:35.395 INFO:tasks.workunit.client.0.smithi031.stdout:4/133: creat db/d14/d1d/f25 x:0 0 0 2024-04-22T23:45:35.398 INFO:tasks.workunit.client.0.smithi031.stdout:8/66: creat d5/da/df/f13 x:0 0 0 2024-04-22T23:45:35.399 INFO:tasks.workunit.client.0.smithi031.stdout:8/67: write d5/f9 [1472143,124237] 0 2024-04-22T23:45:35.401 INFO:tasks.workunit.client.0.smithi031.stdout:8/68: dread f0 [0,4194304] 0 2024-04-22T23:45:35.402 INFO:tasks.workunit.client.0.smithi031.stdout:8/69: write d5/fc [116080,33443] 0 2024-04-22T23:45:35.403 INFO:tasks.workunit.client.0.smithi031.stdout:8/70: fsync d5/f8 0 2024-04-22T23:45:35.404 INFO:tasks.workunit.client.0.smithi031.stdout:9/126: rename d2/l26 to d2/dd/l2d 0 2024-04-22T23:45:35.405 INFO:tasks.workunit.client.0.smithi031.stdout:2/106: fdatasync d6/f9 0 2024-04-22T23:45:35.408 INFO:tasks.workunit.client.0.smithi031.stdout:1/73: creat d1/db/d14/f1b x:0 0 0 2024-04-22T23:45:35.409 INFO:tasks.workunit.client.0.smithi031.stdout:0/92: mknod d0/d1/c22 0 2024-04-22T23:45:35.411 INFO:tasks.workunit.client.0.smithi031.stdout:1/74: truncate d1/f6 1417891 0 2024-04-22T23:45:35.412 INFO:tasks.workunit.client.0.smithi031.stdout:9/127: dwrite d2/dd/f13 [0,4194304] 0 2024-04-22T23:45:35.418 INFO:tasks.workunit.client.0.smithi031.stdout:4/134: mknod db/c26 0 2024-04-22T23:45:35.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/93: creat d0/d1/d2/f23 x:0 0 0 2024-04-22T23:45:35.420 INFO:tasks.workunit.client.0.smithi031.stdout:0/94: stat d0/d1/d2/d3/d11/d1c 0 2024-04-22T23:45:35.421 INFO:tasks.workunit.client.0.smithi031.stdout:8/71: link d5/da/df/f13 d5/da/d12/f14 0 2024-04-22T23:45:35.423 INFO:tasks.workunit.client.0.smithi031.stdout:9/128: unlink d2/dd/f19 0 2024-04-22T23:45:35.424 INFO:tasks.workunit.client.0.smithi031.stdout:2/107: mkdir d6/d1e 0 2024-04-22T23:45:35.425 INFO:tasks.workunit.client.0.smithi031.stdout:2/108: chown d6/d1e 12361243 1 2024-04-22T23:45:35.426 INFO:tasks.workunit.client.0.smithi031.stdout:4/135: creat db/d10/f27 x:0 0 0 2024-04-22T23:45:35.427 INFO:tasks.workunit.client.0.smithi031.stdout:8/72: rmdir d5 39 2024-04-22T23:45:35.433 INFO:tasks.workunit.client.0.smithi031.stdout:4/136: truncate f4 2471370 0 2024-04-22T23:45:35.434 INFO:tasks.workunit.client.0.smithi031.stdout:4/137: write db/d14/d1d/f25 [457773,87764] 0 2024-04-22T23:45:35.443 INFO:tasks.workunit.client.0.smithi031.stdout:9/129: creat d2/f2e x:0 0 0 2024-04-22T23:45:35.444 INFO:tasks.workunit.client.0.smithi031.stdout:9/130: dread - d2/dd/f28 zero size 2024-04-22T23:45:35.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/95: getdents d0/d1/d2/d3/d11/d1c 0 2024-04-22T23:45:35.446 INFO:tasks.workunit.client.0.smithi031.stdout:9/131: chown d2/fb 342352754 1 2024-04-22T23:45:35.448 INFO:tasks.workunit.client.0.smithi031.stdout:2/109: getdents d6/dc/d17 0 2024-04-22T23:45:35.448 INFO:tasks.workunit.client.0.smithi031.stdout:9/132: chown d2/dc/f17 23 1 2024-04-22T23:45:35.449 INFO:tasks.workunit.client.0.smithi031.stdout:8/73: dwrite d5/fb [0,4194304] 0 2024-04-22T23:45:35.449 INFO:tasks.workunit.client.0.smithi031.stdout:9/133: chown d2/d15/l1f 15231081 1 2024-04-22T23:45:35.450 INFO:tasks.workunit.client.0.smithi031.stdout:8/74: chown c3 4952 1 2024-04-22T23:45:35.450 INFO:tasks.workunit.client.0.smithi031.stdout:8/75: fsync d5/f9 0 2024-04-22T23:45:35.451 INFO:tasks.workunit.client.0.smithi031.stdout:0/96: symlink d0/d13/d18/l24 0 2024-04-22T23:45:35.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/97: write d0/d1/d2/f9 [1119780,101243] 0 2024-04-22T23:45:35.459 INFO:tasks.workunit.client.0.smithi031.stdout:9/134: rename d2/dc/d24 to d2/dd/d2f 0 2024-04-22T23:45:35.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/135: write d2/f2e [191693,104146] 0 2024-04-22T23:45:35.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/136: dread - d2/dc/ff zero size 2024-04-22T23:45:35.461 INFO:tasks.workunit.client.0.smithi031.stdout:8/76: unlink f0 0 2024-04-22T23:45:35.462 INFO:tasks.workunit.client.0.smithi031.stdout:8/77: fsync d5/da/fd 0 2024-04-22T23:45:35.463 INFO:tasks.workunit.client.0.smithi031.stdout:9/137: write d2/dd/f13 [1756421,87963] 0 2024-04-22T23:45:35.464 INFO:tasks.workunit.client.0.smithi031.stdout:9/138: dread - d2/f20 zero size 2024-04-22T23:45:35.468 INFO:tasks.workunit.client.0.smithi031.stdout:4/138: write f9 [1575291,78261] 0 2024-04-22T23:45:35.468 INFO:tasks.workunit.client.0.smithi031.stdout:8/78: dread d5/f6 [0,4194304] 0 2024-04-22T23:45:35.469 INFO:tasks.workunit.client.0.smithi031.stdout:8/79: chown d5/l10 993 1 2024-04-22T23:45:35.469 INFO:tasks.workunit.client.0.smithi031.stdout:9/139: dread d2/fb [0,4194304] 0 2024-04-22T23:45:35.470 INFO:tasks.workunit.client.0.smithi031.stdout:8/80: stat d5/da/fd 0 2024-04-22T23:45:35.470 INFO:tasks.workunit.client.0.smithi031.stdout:9/140: readlink d2/d15/l1f 0 2024-04-22T23:45:35.470 INFO:tasks.workunit.client.0.smithi031.stdout:4/139: symlink db/d14/l28 0 2024-04-22T23:45:35.471 INFO:tasks.workunit.client.0.smithi031.stdout:9/141: stat d2 0 2024-04-22T23:45:35.471 INFO:tasks.workunit.client.0.smithi031.stdout:8/81: read f4 [2388553,114622] 0 2024-04-22T23:45:35.472 INFO:tasks.workunit.client.0.smithi031.stdout:9/142: dread - d2/dc/f25 zero size 2024-04-22T23:45:35.473 INFO:tasks.workunit.client.0.smithi031.stdout:4/140: write db/d10/f27 [182819,61128] 0 2024-04-22T23:45:35.474 INFO:tasks.workunit.client.0.smithi031.stdout:4/141: readlink db/l1b 0 2024-04-22T23:45:35.475 INFO:tasks.workunit.client.0.smithi031.stdout:4/142: chown db/d10/l17 2984 1 2024-04-22T23:45:35.475 INFO:tasks.workunit.client.0.smithi031.stdout:8/82: unlink f4 0 2024-04-22T23:45:35.479 INFO:tasks.workunit.client.0.smithi031.stdout:4/143: symlink db/l29 0 2024-04-22T23:45:35.483 INFO:tasks.workunit.client.0.smithi031.stdout:8/83: link c2 d5/da/c15 0 2024-04-22T23:45:35.485 INFO:tasks.workunit.client.0.smithi031.stdout:8/84: dread - d5/da/df/f13 zero size 2024-04-22T23:45:35.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/144: dwrite f9 [0,4194304] 0 2024-04-22T23:45:35.486 INFO:tasks.workunit.client.0.smithi031.stdout:4/145: chown db/d10/f27 3005128 1 2024-04-22T23:45:35.490 INFO:tasks.workunit.client.0.smithi031.stdout:8/85: mknod d5/c16 0 2024-04-22T23:45:35.492 INFO:tasks.workunit.client.0.smithi031.stdout:4/146: dwrite f1 [0,4194304] 0 2024-04-22T23:45:35.493 INFO:tasks.workunit.client.0.smithi031.stdout:4/147: chown db/d10/f27 215 1 2024-04-22T23:45:35.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/86: mknod d5/da/d12/c17 0 2024-04-22T23:45:35.498 INFO:tasks.workunit.client.0.smithi031.stdout:8/87: dread d5/f8 [0,4194304] 0 2024-04-22T23:45:35.499 INFO:tasks.workunit.client.0.smithi031.stdout:8/88: read d5/f6 [4193862,101644] 0 2024-04-22T23:45:35.508 INFO:tasks.workunit.client.0.smithi031.stdout:8/89: rename d5/da/c15 to d5/da/df/c18 0 2024-04-22T23:45:35.512 INFO:tasks.workunit.client.0.smithi031.stdout:8/90: creat d5/f19 x:0 0 0 2024-04-22T23:45:35.513 INFO:tasks.workunit.client.0.smithi031.stdout:4/148: truncate f1 509235 0 2024-04-22T23:45:35.516 INFO:tasks.workunit.client.0.smithi031.stdout:4/149: fdatasync f4 0 2024-04-22T23:45:35.518 INFO:tasks.workunit.client.0.smithi031.stdout:8/91: chown d5/da/df/c18 168 1 2024-04-22T23:45:35.519 INFO:tasks.workunit.client.0.smithi031.stdout:4/150: mkdir db/d10/d2a 0 2024-04-22T23:45:35.520 INFO:tasks.workunit.client.0.smithi031.stdout:7/86: sync 2024-04-22T23:45:35.520 INFO:tasks.workunit.client.0.smithi031.stdout:6/159: sync 2024-04-22T23:45:35.521 INFO:tasks.workunit.client.0.smithi031.stdout:6/160: fdatasync d3/d1f/f22 0 2024-04-22T23:45:35.521 INFO:tasks.workunit.client.0.smithi031.stdout:8/92: mkdir d5/d1a 0 2024-04-22T23:45:35.523 INFO:tasks.workunit.client.0.smithi031.stdout:7/87: mkdir d9/d14/d22 0 2024-04-22T23:45:35.524 INFO:tasks.workunit.client.0.smithi031.stdout:8/93: write d5/f9 [955180,87220] 0 2024-04-22T23:45:35.528 INFO:tasks.workunit.client.0.smithi031.stdout:7/88: creat d9/d14/f23 x:0 0 0 2024-04-22T23:45:35.529 INFO:tasks.workunit.client.0.smithi031.stdout:8/94: creat d5/da/df/f1b x:0 0 0 2024-04-22T23:45:35.529 INFO:tasks.workunit.client.0.smithi031.stdout:4/151: dwrite db/fc [0,4194304] 0 2024-04-22T23:45:35.531 INFO:tasks.workunit.client.0.smithi031.stdout:4/152: readlink db/l1b 0 2024-04-22T23:45:35.532 INFO:tasks.workunit.client.0.smithi031.stdout:4/153: readlink db/l23 0 2024-04-22T23:45:35.532 INFO:tasks.workunit.client.0.smithi031.stdout:8/95: dread d5/f8 [0,4194304] 0 2024-04-22T23:45:35.534 INFO:tasks.workunit.client.0.smithi031.stdout:6/161: dwrite d3/f34 [0,4194304] 0 2024-04-22T23:45:35.535 INFO:tasks.workunit.client.0.smithi031.stdout:8/96: fsync d5/f9 0 2024-04-22T23:45:35.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/89: mknod d9/d14/d22/c24 0 2024-04-22T23:45:35.537 INFO:tasks.workunit.client.0.smithi031.stdout:4/154: mkdir db/d14/d1d/d2b 0 2024-04-22T23:45:35.541 INFO:tasks.workunit.client.0.smithi031.stdout:7/90: getdents d9/d14/d1f 0 2024-04-22T23:45:35.544 INFO:tasks.workunit.client.0.smithi031.stdout:4/155: rename db/le to db/d14/d1d/l2c 0 2024-04-22T23:45:35.546 INFO:tasks.workunit.client.0.smithi031.stdout:8/97: dwrite d5/f6 [4194304,4194304] 0 2024-04-22T23:45:35.551 INFO:tasks.workunit.client.0.smithi031.stdout:8/98: creat d5/da/d12/f1c x:0 0 0 2024-04-22T23:45:35.551 INFO:tasks.workunit.client.0.smithi031.stdout:8/99: readlink d5/le 0 2024-04-22T23:45:35.552 INFO:tasks.workunit.client.0.smithi031.stdout:8/100: truncate d5/f19 751472 0 2024-04-22T23:45:35.555 INFO:tasks.workunit.client.0.smithi031.stdout:6/162: fsync d3/f34 0 2024-04-22T23:45:35.557 INFO:tasks.workunit.client.0.smithi031.stdout:8/101: dread d5/fb [0,4194304] 0 2024-04-22T23:45:35.557 INFO:tasks.workunit.client.0.smithi031.stdout:4/156: rmdir db/d14/d1d/d2b 0 2024-04-22T23:45:35.558 INFO:tasks.workunit.client.0.smithi031.stdout:7/91: link d9/cb d9/d14/d1f/c25 0 2024-04-22T23:45:35.560 INFO:tasks.workunit.client.0.smithi031.stdout:4/157: fdatasync f4 0 2024-04-22T23:45:35.564 INFO:tasks.workunit.client.0.smithi031.stdout:4/158: mkdir db/d14/d2d 0 2024-04-22T23:45:35.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/159: chown db/lf 200920 1 2024-04-22T23:45:35.567 INFO:tasks.workunit.client.0.smithi031.stdout:8/102: dwrite d5/f8 [0,4194304] 0 2024-04-22T23:45:35.573 INFO:tasks.workunit.client.0.smithi031.stdout:4/160: dwrite f9 [0,4194304] 0 2024-04-22T23:45:35.575 INFO:tasks.workunit.client.0.smithi031.stdout:7/92: truncate f3 3325094 0 2024-04-22T23:45:35.582 INFO:tasks.workunit.client.0.smithi031.stdout:8/103: write d5/da/d12/f14 [496960,50955] 0 2024-04-22T23:45:35.584 INFO:tasks.workunit.client.0.smithi031.stdout:7/93: getdents d9/d14/d1f 0 2024-04-22T23:45:35.585 INFO:tasks.workunit.client.0.smithi031.stdout:7/94: stat d9/d14/d22 0 2024-04-22T23:45:35.585 INFO:tasks.workunit.client.0.smithi031.stdout:8/104: mkdir d5/da/d12/d1d 0 2024-04-22T23:45:35.586 INFO:tasks.workunit.client.0.smithi031.stdout:7/95: write d9/d14/f20 [977699,11350] 0 2024-04-22T23:45:35.587 INFO:tasks.workunit.client.0.smithi031.stdout:8/105: creat d5/da/f1e x:0 0 0 2024-04-22T23:45:35.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/106: mknod d5/da/d12/c1f 0 2024-04-22T23:45:35.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/107: chown d5/c7 133468010 1 2024-04-22T23:45:35.592 INFO:tasks.workunit.client.0.smithi031.stdout:7/96: unlink d9/l11 0 2024-04-22T23:45:35.593 INFO:tasks.workunit.client.0.smithi031.stdout:4/161: truncate db/fc 1409992 0 2024-04-22T23:45:35.595 INFO:tasks.workunit.client.0.smithi031.stdout:7/97: mknod d9/d14/d1f/c26 0 2024-04-22T23:45:35.596 INFO:tasks.workunit.client.0.smithi031.stdout:7/98: chown f7 1450 1 2024-04-22T23:45:35.596 INFO:tasks.workunit.client.0.smithi031.stdout:4/162: mknod db/d10/c2e 0 2024-04-22T23:45:35.599 INFO:tasks.workunit.client.0.smithi031.stdout:8/108: dwrite d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:35.600 INFO:tasks.workunit.client.0.smithi031.stdout:4/163: creat db/d10/f2f x:0 0 0 2024-04-22T23:45:35.601 INFO:tasks.workunit.client.0.smithi031.stdout:4/164: chown l0 9276695 1 2024-04-22T23:45:35.606 INFO:tasks.workunit.client.0.smithi031.stdout:7/99: dwrite d9/f1c [0,4194304] 0 2024-04-22T23:45:35.613 INFO:tasks.workunit.client.0.smithi031.stdout:8/109: link d5/f9 d5/da/f20 0 2024-04-22T23:45:35.614 INFO:tasks.workunit.client.0.smithi031.stdout:7/100: fsync f7 0 2024-04-22T23:45:35.633 INFO:tasks.workunit.client.0.smithi031.stdout:7/101: link d9/d14/d22/c24 d9/d15/c27 0 2024-04-22T23:45:35.633 INFO:tasks.workunit.client.0.smithi031.stdout:7/102: chown d9/d14/f23 1716777 1 2024-04-22T23:45:35.635 INFO:tasks.workunit.client.0.smithi031.stdout:7/103: chown d9/c13 1 1 2024-04-22T23:45:35.635 INFO:tasks.workunit.client.0.smithi031.stdout:7/104: chown d9/d15/c1d 3 1 2024-04-22T23:45:35.637 INFO:tasks.workunit.client.0.smithi031.stdout:7/105: mkdir d9/d14/d22/d28 0 2024-04-22T23:45:35.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/106: write f3 [2621341,22232] 0 2024-04-22T23:45:35.649 INFO:tasks.workunit.client.0.smithi031.stdout:7/107: link d9/d14/d1f/c26 d9/d14/d22/c29 0 2024-04-22T23:45:35.653 INFO:tasks.workunit.client.0.smithi031.stdout:7/108: link f3 d9/d14/d1f/f2a 0 2024-04-22T23:45:35.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/109: link d9/l12 d9/d14/l2b 0 2024-04-22T23:45:35.662 INFO:tasks.workunit.client.0.smithi031.stdout:7/110: rename d9/d14/d22/c24 to d9/d14/c2c 0 2024-04-22T23:45:35.664 INFO:tasks.workunit.client.0.smithi031.stdout:7/111: rmdir d9/d15 39 2024-04-22T23:45:35.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/112: dread f8 [0,4194304] 0 2024-04-22T23:45:35.669 INFO:tasks.workunit.client.0.smithi031.stdout:7/113: creat d9/d14/d22/f2d x:0 0 0 2024-04-22T23:45:35.670 INFO:tasks.workunit.client.0.smithi031.stdout:7/114: truncate f8 1979998 0 2024-04-22T23:45:35.670 INFO:tasks.workunit.client.0.smithi031.stdout:7/115: chown f6 1517300 1 2024-04-22T23:45:35.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/116: truncate f6 549424 0 2024-04-22T23:45:35.684 INFO:tasks.workunit.client.0.smithi031.stdout:7/117: write f3 [1145816,108621] 0 2024-04-22T23:45:35.690 INFO:tasks.workunit.client.0.smithi031.stdout:7/118: getdents d9/d15 0 2024-04-22T23:45:35.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/119: mknod d9/d15/c2e 0 2024-04-22T23:45:35.704 INFO:tasks.workunit.client.0.smithi031.stdout:3/36: sync 2024-04-22T23:45:35.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/82: sync 2024-04-22T23:45:35.706 INFO:tasks.workunit.client.0.smithi031.stdout:5/83: mknod db/d16/c19 0 2024-04-22T23:45:35.706 INFO:tasks.workunit.client.0.smithi031.stdout:5/84: chown f5 109938 1 2024-04-22T23:45:35.709 INFO:tasks.workunit.client.0.smithi031.stdout:5/85: creat db/d16/f1a x:0 0 0 2024-04-22T23:45:35.712 INFO:tasks.workunit.client.0.smithi031.stdout:5/86: unlink la 0 2024-04-22T23:45:35.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/87: link db/l17 db/l1b 0 2024-04-22T23:45:35.723 INFO:tasks.workunit.client.0.smithi031.stdout:5/88: symlink db/l1c 0 2024-04-22T23:45:35.724 INFO:tasks.workunit.client.0.smithi031.stdout:3/37: write f3 [2409757,113277] 0 2024-04-22T23:45:35.743 INFO:tasks.workunit.client.0.smithi031.stdout:3/38: truncate f3 2348204 0 2024-04-22T23:45:35.746 INFO:tasks.workunit.client.0.smithi031.stdout:5/89: dwrite db/fd [0,4194304] 0 2024-04-22T23:45:35.748 INFO:tasks.workunit.client.0.smithi031.stdout:5/90: mkdir db/d16/d1d 0 2024-04-22T23:45:35.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/91: creat db/f1e x:0 0 0 2024-04-22T23:45:35.758 INFO:tasks.workunit.client.0.smithi031.stdout:5/92: creat db/d16/d1d/f1f x:0 0 0 2024-04-22T23:45:35.760 INFO:tasks.workunit.client.0.smithi031.stdout:5/93: creat db/d16/d1d/f20 x:0 0 0 2024-04-22T23:45:35.761 INFO:tasks.workunit.client.0.smithi031.stdout:5/94: chown db/f1e 8083 1 2024-04-22T23:45:35.765 INFO:tasks.workunit.client.0.smithi031.stdout:3/39: dread f3 [0,4194304] 0 2024-04-22T23:45:35.765 INFO:tasks.workunit.client.0.smithi031.stdout:5/95: creat db/d16/f21 x:0 0 0 2024-04-22T23:45:35.766 INFO:tasks.workunit.client.0.smithi031.stdout:5/96: read db/d16/f18 [822376,6160] 0 2024-04-22T23:45:35.769 INFO:tasks.workunit.client.0.smithi031.stdout:3/40: mkdir d4 0 2024-04-22T23:45:35.773 INFO:tasks.workunit.client.0.smithi031.stdout:5/97: dwrite db/d16/d1d/f20 [0,4194304] 0 2024-04-22T23:45:35.774 INFO:tasks.workunit.client.0.smithi031.stdout:3/41: dread f3 [0,4194304] 0 2024-04-22T23:45:35.774 INFO:tasks.workunit.client.0.smithi031.stdout:5/98: write db/f1e [175486,59536] 0 2024-04-22T23:45:35.778 INFO:tasks.workunit.client.0.smithi031.stdout:3/42: fsync f3 0 2024-04-22T23:45:35.780 INFO:tasks.workunit.client.0.smithi031.stdout:5/99: creat db/d16/d1d/f22 x:0 0 0 2024-04-22T23:45:35.781 INFO:tasks.workunit.client.0.smithi031.stdout:5/100: dread - db/d16/f21 zero size 2024-04-22T23:45:35.789 INFO:tasks.workunit.client.0.smithi031.stdout:3/43: creat d4/f5 x:0 0 0 2024-04-22T23:45:35.798 INFO:tasks.workunit.client.0.smithi031.stdout:5/101: write db/d16/d1d/f20 [4280626,80752] 0 2024-04-22T23:45:35.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/102: dwrite db/d16/f21 [0,4194304] 0 2024-04-22T23:45:35.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/44: fsync d4/f5 0 2024-04-22T23:45:35.810 INFO:tasks.workunit.client.0.smithi031.stdout:5/103: creat db/f23 x:0 0 0 2024-04-22T23:45:35.810 INFO:tasks.workunit.client.0.smithi031.stdout:5/104: fdatasync db/d16/f15 0 2024-04-22T23:45:35.815 INFO:tasks.workunit.client.0.smithi031.stdout:1/75: sync 2024-04-22T23:45:35.815 INFO:tasks.workunit.client.0.smithi031.stdout:3/45: rmdir d4 39 2024-04-22T23:45:35.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/105: unlink db/l1c 0 2024-04-22T23:45:35.819 INFO:tasks.workunit.client.0.smithi031.stdout:5/106: truncate db/f1e 1202157 0 2024-04-22T23:45:35.820 INFO:tasks.workunit.client.0.smithi031.stdout:5/107: truncate db/f1e 1623024 0 2024-04-22T23:45:35.820 INFO:tasks.workunit.client.0.smithi031.stdout:3/46: write d4/f5 [307356,63495] 0 2024-04-22T23:45:35.821 INFO:tasks.workunit.client.0.smithi031.stdout:1/76: fdatasync d1/db/d14/ff 0 2024-04-22T23:45:35.824 INFO:tasks.workunit.client.0.smithi031.stdout:3/47: dread f3 [0,4194304] 0 2024-04-22T23:45:35.824 INFO:tasks.workunit.client.0.smithi031.stdout:5/108: mkdir db/d16/d1d/d24 0 2024-04-22T23:45:35.828 INFO:tasks.workunit.client.0.smithi031.stdout:1/77: rename d1 to d1/d1c 22 2024-04-22T23:45:35.830 INFO:tasks.workunit.client.0.smithi031.stdout:3/48: mkdir d4/d6 0 2024-04-22T23:45:35.835 INFO:tasks.workunit.client.0.smithi031.stdout:1/78: dwrite f0 [0,4194304] 0 2024-04-22T23:45:35.837 INFO:tasks.workunit.client.0.smithi031.stdout:5/109: link db/d16/f18 db/f25 0 2024-04-22T23:45:35.837 INFO:tasks.workunit.client.0.smithi031.stdout:5/110: stat db/d16 0 2024-04-22T23:45:35.838 INFO:tasks.workunit.client.0.smithi031.stdout:5/111: fdatasync db/d16/d1d/f20 0 2024-04-22T23:45:35.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/49: creat d4/d6/f7 x:0 0 0 2024-04-22T23:45:35.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/112: symlink db/d16/d1d/d24/l26 0 2024-04-22T23:45:35.852 INFO:tasks.workunit.client.0.smithi031.stdout:1/79: creat d1/f1d x:0 0 0 2024-04-22T23:45:35.856 INFO:tasks.workunit.client.0.smithi031.stdout:1/80: mkdir d1/de/d1e 0 2024-04-22T23:45:35.856 INFO:tasks.workunit.client.0.smithi031.stdout:1/81: stat d1/db/f19 0 2024-04-22T23:45:35.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/50: rmdir d4/d6 39 2024-04-22T23:45:35.867 INFO:tasks.workunit.client.0.smithi031.stdout:3/51: dread d4/f5 [0,4194304] 0 2024-04-22T23:45:35.874 INFO:tasks.workunit.client.0.smithi031.stdout:3/52: rename l1 to d4/d6/l8 0 2024-04-22T23:45:35.877 INFO:tasks.workunit.client.0.smithi031.stdout:3/53: creat d4/d6/f9 x:0 0 0 2024-04-22T23:45:35.879 INFO:tasks.workunit.client.0.smithi031.stdout:5/113: truncate db/f1e 1546451 0 2024-04-22T23:45:35.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/54: mkdir d4/d6/da 0 2024-04-22T23:45:35.881 INFO:tasks.workunit.client.0.smithi031.stdout:5/114: mkdir db/d16/d1d/d24/d27 0 2024-04-22T23:45:35.882 INFO:tasks.workunit.client.0.smithi031.stdout:5/115: chown db/d16/d1d 18 1 2024-04-22T23:45:35.882 INFO:tasks.workunit.client.0.smithi031.stdout:5/116: readlink l1 0 2024-04-22T23:45:35.883 INFO:tasks.workunit.client.0.smithi031.stdout:3/55: mkdir d4/d6/da/db 0 2024-04-22T23:45:35.886 INFO:tasks.workunit.client.0.smithi031.stdout:3/56: symlink d4/lc 0 2024-04-22T23:45:35.888 INFO:tasks.workunit.client.0.smithi031.stdout:3/57: write d4/f5 [1133715,81778] 0 2024-04-22T23:45:35.890 INFO:tasks.workunit.client.0.smithi031.stdout:3/58: write d4/d6/f7 [103255,103565] 0 2024-04-22T23:45:35.891 INFO:tasks.workunit.client.0.smithi031.stdout:3/59: write d4/d6/f7 [307148,3725] 0 2024-04-22T23:45:35.894 INFO:tasks.workunit.client.0.smithi031.stdout:3/60: dread d4/d6/f7 [0,4194304] 0 2024-04-22T23:45:35.898 INFO:tasks.workunit.client.0.smithi031.stdout:3/61: unlink d4/lc 0 2024-04-22T23:45:35.901 INFO:tasks.workunit.client.0.smithi031.stdout:3/62: mknod d4/d6/cd 0 2024-04-22T23:45:35.911 INFO:tasks.workunit.client.0.smithi031.stdout:3/63: creat d4/fe x:0 0 0 2024-04-22T23:45:35.920 INFO:tasks.workunit.client.0.smithi031.stdout:3/64: getdents d4/d6/da 0 2024-04-22T23:45:35.926 INFO:tasks.workunit.client.0.smithi031.stdout:3/65: dwrite d4/fe [0,4194304] 0 2024-04-22T23:45:35.931 INFO:tasks.workunit.client.0.smithi031.stdout:3/66: creat d4/d6/da/ff x:0 0 0 2024-04-22T23:45:35.936 INFO:tasks.workunit.client.0.smithi031.stdout:3/67: dwrite d4/fe [0,4194304] 0 2024-04-22T23:45:35.953 INFO:tasks.workunit.client.0.smithi031.stdout:2/110: sync 2024-04-22T23:45:35.956 INFO:tasks.workunit.client.0.smithi031.stdout:2/111: creat d6/dc/d17/f1f x:0 0 0 2024-04-22T23:45:35.961 INFO:tasks.workunit.client.0.smithi031.stdout:2/112: getdents d6/d1e 0 2024-04-22T23:45:35.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/113: fdatasync d6/f11 0 2024-04-22T23:45:35.967 INFO:tasks.workunit.client.0.smithi031.stdout:3/68: rmdir d4/d6/da 39 2024-04-22T23:45:35.977 INFO:tasks.workunit.client.0.smithi031.stdout:3/69: rmdir d4/d6/da/db 0 2024-04-22T23:45:35.981 INFO:tasks.workunit.client.0.smithi031.stdout:3/70: mkdir d4/d10 0 2024-04-22T23:45:35.982 INFO:tasks.workunit.client.0.smithi031.stdout:3/71: fsync f3 0 2024-04-22T23:45:35.989 INFO:tasks.workunit.client.0.smithi031.stdout:3/72: unlink d4/d6/l8 0 2024-04-22T23:45:35.992 INFO:tasks.workunit.client.0.smithi031.stdout:3/73: dread d4/d6/f7 [0,4194304] 0 2024-04-22T23:45:35.998 INFO:tasks.workunit.client.0.smithi031.stdout:3/74: mkdir d4/d6/d11 0 2024-04-22T23:45:36.012 INFO:tasks.workunit.client.0.smithi031.stdout:3/75: getdents d4 0 2024-04-22T23:45:36.014 INFO:tasks.workunit.client.0.smithi031.stdout:3/76: symlink d4/l12 0 2024-04-22T23:45:36.069 INFO:tasks.workunit.client.0.smithi031.stdout:0/98: sync 2024-04-22T23:45:36.089 INFO:tasks.workunit.client.0.smithi031.stdout:0/99: truncate d0/d1/f10 1744503 0 2024-04-22T23:45:36.091 INFO:tasks.workunit.client.0.smithi031.stdout:0/100: mkdir d0/d13/d18/d1e/d25 0 2024-04-22T23:45:36.093 INFO:tasks.workunit.client.0.smithi031.stdout:0/101: mkdir d0/d13/d18/d26 0 2024-04-22T23:45:36.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/102: unlink d0/d13/c21 0 2024-04-22T23:45:36.097 INFO:tasks.workunit.client.0.smithi031.stdout:0/103: mknod d0/d1/d2/d3/d11/d1c/c27 0 2024-04-22T23:45:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:0/104: unlink d0/d1/d2/d3/d11/l19 0 2024-04-22T23:45:36.102 INFO:tasks.workunit.client.0.smithi031.stdout:0/105: unlink d0/d1/fc 0 2024-04-22T23:45:36.106 INFO:tasks.workunit.client.0.smithi031.stdout:0/106: dread d0/d1/d2/d3/f1d [0,4194304] 0 2024-04-22T23:45:36.122 INFO:tasks.workunit.client.0.smithi031.stdout:4/165: sync 2024-04-22T23:45:36.122 INFO:tasks.workunit.client.0.smithi031.stdout:6/163: sync 2024-04-22T23:45:36.122 INFO:tasks.workunit.client.0.smithi031.stdout:9/143: sync 2024-04-22T23:45:36.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/144: read - d2/dc/f1e zero size 2024-04-22T23:45:36.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/145: truncate d2/dc/f17 762139 0 2024-04-22T23:45:36.126 INFO:tasks.workunit.client.0.smithi031.stdout:4/166: unlink db/c16 0 2024-04-22T23:45:36.127 INFO:tasks.workunit.client.0.smithi031.stdout:6/164: unlink d3/d1f/f22 0 2024-04-22T23:45:36.130 INFO:tasks.workunit.client.0.smithi031.stdout:9/146: dwrite d2/f20 [0,4194304] 0 2024-04-22T23:45:36.132 INFO:tasks.workunit.client.0.smithi031.stdout:4/167: mkdir db/d10/d2a/d30 0 2024-04-22T23:45:36.134 INFO:tasks.workunit.client.0.smithi031.stdout:4/168: chown db/lf 0 1 2024-04-22T23:45:36.137 INFO:tasks.workunit.client.0.smithi031.stdout:6/165: dwrite d3/d1f/f28 [0,4194304] 0 2024-04-22T23:45:36.140 INFO:tasks.workunit.client.0.smithi031.stdout:9/147: dwrite d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:36.144 INFO:tasks.workunit.client.0.smithi031.stdout:4/169: mkdir db/d31 0 2024-04-22T23:45:36.145 INFO:tasks.workunit.client.0.smithi031.stdout:4/170: write db/f24 [390176,87494] 0 2024-04-22T23:45:36.147 INFO:tasks.workunit.client.0.smithi031.stdout:9/148: creat d2/d15/f30 x:0 0 0 2024-04-22T23:45:36.151 INFO:tasks.workunit.client.0.smithi031.stdout:4/171: creat db/d10/f32 x:0 0 0 2024-04-22T23:45:36.153 INFO:tasks.workunit.client.0.smithi031.stdout:9/149: mknod d2/dd/d1b/c31 0 2024-04-22T23:45:36.155 INFO:tasks.workunit.client.0.smithi031.stdout:6/166: rename d3/d35/l36 to d3/l3b 0 2024-04-22T23:45:36.159 INFO:tasks.workunit.client.0.smithi031.stdout:9/150: creat d2/dd/d1b/f32 x:0 0 0 2024-04-22T23:45:36.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/172: dread db/f24 [0,4194304] 0 2024-04-22T23:45:36.165 INFO:tasks.workunit.client.0.smithi031.stdout:6/167: mkdir d3/d2a/d3c 0 2024-04-22T23:45:36.169 INFO:tasks.workunit.client.0.smithi031.stdout:4/173: rename db/d10/f27 to db/d31/f33 0 2024-04-22T23:45:36.171 INFO:tasks.workunit.client.0.smithi031.stdout:6/168: chown d3/f1a 3596 1 2024-04-22T23:45:36.177 INFO:tasks.workunit.client.0.smithi031.stdout:4/174: rename db/lf to db/d10/d2a/d30/l34 0 2024-04-22T23:45:36.177 INFO:tasks.workunit.client.0.smithi031.stdout:6/169: dwrite d3/f39 [0,4194304] 0 2024-04-22T23:45:36.178 INFO:tasks.workunit.client.0.smithi031.stdout:4/175: read db/f24 [224485,982] 0 2024-04-22T23:45:36.182 INFO:tasks.workunit.client.0.smithi031.stdout:6/170: dread d3/f34 [0,4194304] 0 2024-04-22T23:45:36.185 INFO:tasks.workunit.client.0.smithi031.stdout:9/151: getdents d2/dd/d2f 0 2024-04-22T23:45:36.187 INFO:tasks.workunit.client.0.smithi031.stdout:6/171: symlink d3/d2a/l3d 0 2024-04-22T23:45:36.189 INFO:tasks.workunit.client.0.smithi031.stdout:4/176: creat db/d14/d2d/f35 x:0 0 0 2024-04-22T23:45:36.191 INFO:tasks.workunit.client.0.smithi031.stdout:9/152: dread d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:36.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/153: rename d2/dd/f28 to d2/dd/d1b/f33 0 2024-04-22T23:45:36.196 INFO:tasks.workunit.client.0.smithi031.stdout:9/154: truncate d2/dd/d1b/f32 143083 0 2024-04-22T23:45:36.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/177: getdents db/d10/d2a/d30 0 2024-04-22T23:45:36.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/155: dwrite d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:36.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/82: sync 2024-04-22T23:45:36.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/110: sync 2024-04-22T23:45:36.207 INFO:tasks.workunit.client.0.smithi031.stdout:7/120: sync 2024-04-22T23:45:36.208 INFO:tasks.workunit.client.0.smithi031.stdout:2/114: sync 2024-04-22T23:45:36.208 INFO:tasks.workunit.client.0.smithi031.stdout:5/117: sync 2024-04-22T23:45:36.208 INFO:tasks.workunit.client.0.smithi031.stdout:3/77: sync 2024-04-22T23:45:36.208 INFO:tasks.workunit.client.0.smithi031.stdout:4/178: mkdir db/d14/d2d/d36 0 2024-04-22T23:45:36.209 INFO:tasks.workunit.client.0.smithi031.stdout:9/156: dread d2/f20 [0,4194304] 0 2024-04-22T23:45:36.210 INFO:tasks.workunit.client.0.smithi031.stdout:2/115: write d6/f19 [1726298,92364] 0 2024-04-22T23:45:36.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:35 smithi044 ceph-mon[34463]: pgmap v24: 65 pgs: 65 active+clean; 293 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.9 MiB/s rd, 20 MiB/s wr, 359 op/s 2024-04-22T23:45:36.211 INFO:tasks.workunit.client.0.smithi031.stdout:6/172: dread d3/f17 [4194304,4194304] 0 2024-04-22T23:45:36.219 INFO:tasks.workunit.client.0.smithi031.stdout:1/83: mkdir d1/db/d16/d1f 0 2024-04-22T23:45:36.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/157: dwrite d2/dd/f13 [0,4194304] 0 2024-04-22T23:45:36.223 INFO:tasks.workunit.client.0.smithi031.stdout:2/116: mknod d6/dc/d17/c20 0 2024-04-22T23:45:36.226 INFO:tasks.workunit.client.0.smithi031.stdout:8/111: dwrite d5/fb [4194304,4194304] 0 2024-04-22T23:45:36.228 INFO:tasks.workunit.client.0.smithi031.stdout:3/78: creat d4/d6/da/f13 x:0 0 0 2024-04-22T23:45:36.229 INFO:tasks.workunit.client.0.smithi031.stdout:4/179: truncate db/d14/d1d/f20 1152827 0 2024-04-22T23:45:36.230 INFO:tasks.workunit.client.0.smithi031.stdout:1/84: unlink d1/de/c15 0 2024-04-22T23:45:36.232 INFO:tasks.workunit.client.0.smithi031.stdout:9/158: creat d2/dd/d1b/f34 x:0 0 0 2024-04-22T23:45:36.233 INFO:tasks.workunit.client.0.smithi031.stdout:9/159: write d2/d15/f30 [87548,114967] 0 2024-04-22T23:45:36.234 INFO:tasks.workunit.client.0.smithi031.stdout:9/160: chown d2/dd 489562469 1 2024-04-22T23:45:36.236 INFO:tasks.workunit.client.0.smithi031.stdout:7/121: write f6 [370258,99279] 0 2024-04-22T23:45:36.237 INFO:tasks.workunit.client.0.smithi031.stdout:7/122: write d9/d14/d1f/f2a [4166285,10818] 0 2024-04-22T23:45:36.239 INFO:tasks.workunit.client.0.smithi031.stdout:8/112: mknod d5/da/df/c21 0 2024-04-22T23:45:36.240 INFO:tasks.workunit.client.0.smithi031.stdout:4/180: creat db/d14/d2d/f37 x:0 0 0 2024-04-22T23:45:36.248 INFO:tasks.workunit.client.0.smithi031.stdout:4/181: dwrite db/d10/f12 [0,4194304] 0 2024-04-22T23:45:36.249 INFO:tasks.workunit.client.0.smithi031.stdout:6/173: truncate d3/f1a 4193374 0 2024-04-22T23:45:36.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/174: truncate d3/d1f/f28 4843876 0 2024-04-22T23:45:36.252 INFO:tasks.workunit.client.0.smithi031.stdout:7/123: mknod d9/d14/d22/d28/c2f 0 2024-04-22T23:45:36.254 INFO:tasks.workunit.client.0.smithi031.stdout:7/124: write d9/d14/d1f/f2a [1902857,59393] 0 2024-04-22T23:45:36.257 INFO:tasks.workunit.client.0.smithi031.stdout:8/113: symlink d5/da/d12/d1d/l22 0 2024-04-22T23:45:36.258 INFO:tasks.workunit.client.0.smithi031.stdout:6/175: dread d3/f8 [0,4194304] 0 2024-04-22T23:45:36.258 INFO:tasks.workunit.client.0.smithi031.stdout:4/182: write db/f24 [1197065,54904] 0 2024-04-22T23:45:36.259 INFO:tasks.workunit.client.0.smithi031.stdout:9/161: creat d2/f35 x:0 0 0 2024-04-22T23:45:36.260 INFO:tasks.workunit.client.0.smithi031.stdout:8/114: chown d5/le 198977 1 2024-04-22T23:45:36.262 INFO:tasks.workunit.client.0.smithi031.stdout:2/117: getdents d6/dc/d17 0 2024-04-22T23:45:36.263 INFO:tasks.workunit.client.0.smithi031.stdout:1/85: dwrite d1/de/f11 [0,4194304] 0 2024-04-22T23:45:36.265 INFO:tasks.workunit.client.0.smithi031.stdout:3/79: link d4/l12 d4/l14 0 2024-04-22T23:45:36.272 INFO:tasks.workunit.client.0.smithi031.stdout:4/183: symlink db/d14/d2d/l38 0 2024-04-22T23:45:36.273 INFO:tasks.workunit.client.0.smithi031.stdout:6/176: rename d3/d1f/l24 to d3/d1f/l3e 0 2024-04-22T23:45:36.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/184: truncate db/d10/f2f 327868 0 2024-04-22T23:45:36.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/185: chown db/d10 1592312 1 2024-04-22T23:45:36.276 INFO:tasks.workunit.client.0.smithi031.stdout:9/162: creat d2/dd/d1b/f36 x:0 0 0 2024-04-22T23:45:36.277 INFO:tasks.workunit.client.0.smithi031.stdout:8/115: symlink d5/da/d12/l23 0 2024-04-22T23:45:36.279 INFO:tasks.workunit.client.0.smithi031.stdout:2/118: unlink d6/dc/d17/f1f 0 2024-04-22T23:45:36.281 INFO:tasks.workunit.client.0.smithi031.stdout:1/86: unlink d1/db/d14/ff 0 2024-04-22T23:45:36.282 INFO:tasks.workunit.client.0.smithi031.stdout:4/186: dwrite db/d10/f2f [0,4194304] 0 2024-04-22T23:45:36.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/125: rename d9/d14/d22/c29 to d9/d14/d22/c30 0 2024-04-22T23:45:36.284 INFO:tasks.workunit.client.0.smithi031.stdout:6/177: dread - d3/f25 zero size 2024-04-22T23:45:36.287 INFO:tasks.workunit.client.0.smithi031.stdout:2/119: dread d6/dc/f1a [0,4194304] 0 2024-04-22T23:45:36.287 INFO:tasks.workunit.client.0.smithi031.stdout:8/116: symlink d5/da/d12/l24 0 2024-04-22T23:45:36.291 INFO:tasks.workunit.client.0.smithi031.stdout:3/80: mknod d4/d6/d11/c15 0 2024-04-22T23:45:36.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/87: rename d1/de/f12 to d1/de/f20 0 2024-04-22T23:45:36.295 INFO:tasks.workunit.client.0.smithi031.stdout:3/81: chown d4/d6/d11/c15 45047 1 2024-04-22T23:45:36.296 INFO:tasks.workunit.client.0.smithi031.stdout:7/126: dwrite d9/d15/f17 [0,4194304] 0 2024-04-22T23:45:36.297 INFO:tasks.workunit.client.0.smithi031.stdout:8/117: dread d5/da/df/f13 [0,4194304] 0 2024-04-22T23:45:36.299 INFO:tasks.workunit.client.0.smithi031.stdout:9/163: dwrite d2/dd/d1b/f33 [0,4194304] 0 2024-04-22T23:45:36.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/127: read f3 [1471158,42408] 0 2024-04-22T23:45:36.301 INFO:tasks.workunit.client.0.smithi031.stdout:9/164: chown d2/dd/d1b/f36 453349504 1 2024-04-22T23:45:36.306 INFO:tasks.workunit.client.0.smithi031.stdout:9/165: dread d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:36.307 INFO:tasks.workunit.client.0.smithi031.stdout:9/166: readlink d2/d15/l1f 0 2024-04-22T23:45:36.315 INFO:tasks.workunit.client.0.smithi031.stdout:1/88: mkdir d1/d21 0 2024-04-22T23:45:36.321 INFO:tasks.workunit.client.0.smithi031.stdout:4/187: dwrite f4 [0,4194304] 0 2024-04-22T23:45:36.323 INFO:tasks.workunit.client.0.smithi031.stdout:2/120: rename d6/dc/le to d6/l21 0 2024-04-22T23:45:36.327 INFO:tasks.workunit.client.0.smithi031.stdout:3/82: creat d4/d6/f16 x:0 0 0 2024-04-22T23:45:36.332 INFO:tasks.workunit.client.0.smithi031.stdout:7/128: rename d9/d14/d1f/f2a to d9/d14/f31 0 2024-04-22T23:45:36.333 INFO:tasks.workunit.client.0.smithi031.stdout:1/89: mknod d1/d21/c22 0 2024-04-22T23:45:36.334 INFO:tasks.workunit.client.0.smithi031.stdout:0/107: sync 2024-04-22T23:45:36.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/83: rename c2 to d4/d6/d11/c17 0 2024-04-22T23:45:36.338 INFO:tasks.workunit.client.0.smithi031.stdout:8/118: write d5/f9 [2914762,121031] 0 2024-04-22T23:45:36.339 INFO:tasks.workunit.client.0.smithi031.stdout:8/119: write d5/f8 [467924,34250] 0 2024-04-22T23:45:36.342 INFO:tasks.workunit.client.0.smithi031.stdout:1/90: creat d1/db/d14/f23 x:0 0 0 2024-04-22T23:45:36.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/188: link db/d10/c2e db/d14/c39 0 2024-04-22T23:45:36.346 INFO:tasks.workunit.client.0.smithi031.stdout:0/108: dread d0/d1/f10 [0,4194304] 0 2024-04-22T23:45:36.346 INFO:tasks.workunit.client.0.smithi031.stdout:4/189: stat db/d14/l1c 0 2024-04-22T23:45:36.346 INFO:tasks.workunit.client.0.smithi031.stdout:3/84: fsync d4/fe 0 2024-04-22T23:45:36.348 INFO:tasks.workunit.client.0.smithi031.stdout:9/167: truncate d2/dc/f17 540883 0 2024-04-22T23:45:36.350 INFO:tasks.workunit.client.0.smithi031.stdout:9/168: chown d2/d15/f2c 9 1 2024-04-22T23:45:36.351 INFO:tasks.workunit.client.0.smithi031.stdout:0/109: truncate d0/d1/d2/f9 4695740 0 2024-04-22T23:45:36.351 INFO:tasks.workunit.client.0.smithi031.stdout:1/91: dwrite d1/db/d14/f23 [0,4194304] 0 2024-04-22T23:45:36.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/169: dread - d2/dc/f25 zero size 2024-04-22T23:45:36.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/121: truncate d6/dc/f15 218375 0 2024-04-22T23:45:36.357 INFO:tasks.workunit.client.0.smithi031.stdout:1/92: write d1/f8 [5101183,47692] 0 2024-04-22T23:45:36.358 INFO:tasks.workunit.client.0.smithi031.stdout:2/122: fdatasync f4 0 2024-04-22T23:45:36.359 INFO:tasks.workunit.client.0.smithi031.stdout:7/129: getdents d9/d15 0 2024-04-22T23:45:36.361 INFO:tasks.workunit.client.0.smithi031.stdout:3/85: dwrite d4/d6/f9 [0,4194304] 0 2024-04-22T23:45:36.361 INFO:tasks.workunit.client.0.smithi031.stdout:2/123: fsync d6/f19 0 2024-04-22T23:45:36.363 INFO:tasks.workunit.client.0.smithi031.stdout:1/93: chown d1/de/f20 30065868 1 2024-04-22T23:45:36.364 INFO:tasks.workunit.client.0.smithi031.stdout:2/124: fsync f4 0 2024-04-22T23:45:36.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/110: dread d0/d1/d2/f9 [0,4194304] 0 2024-04-22T23:45:36.365 INFO:tasks.workunit.client.0.smithi031.stdout:1/94: dread - d1/db/d14/f1b zero size 2024-04-22T23:45:36.372 INFO:tasks.workunit.client.0.smithi031.stdout:3/86: write d4/f5 [407549,118853] 0 2024-04-22T23:45:36.373 INFO:tasks.workunit.client.0.smithi031.stdout:4/190: rmdir db/d14/d2d/d36 0 2024-04-22T23:45:36.378 INFO:tasks.workunit.client.0.smithi031.stdout:4/191: dread f1 [0,4194304] 0 2024-04-22T23:45:36.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/111: mkdir d0/d1/d28 0 2024-04-22T23:45:36.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/125: dwrite d6/f19 [0,4194304] 0 2024-04-22T23:45:36.383 INFO:tasks.workunit.client.0.smithi031.stdout:9/170: dwrite d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:36.387 INFO:tasks.workunit.client.0.smithi031.stdout:1/95: dwrite d1/db/d14/f1a [0,4194304] 0 2024-04-22T23:45:36.387 INFO:tasks.workunit.client.0.smithi031.stdout:7/130: truncate d9/d15/f21 139043 0 2024-04-22T23:45:36.388 INFO:tasks.workunit.client.0.smithi031.stdout:9/171: fsync d2/dd/d1b/f36 0 2024-04-22T23:45:36.389 INFO:tasks.workunit.client.0.smithi031.stdout:0/112: chown d0/d1/d2/d3 167 1 2024-04-22T23:45:36.390 INFO:tasks.workunit.client.0.smithi031.stdout:2/126: chown l5 7417543 1 2024-04-22T23:45:36.392 INFO:tasks.workunit.client.0.smithi031.stdout:3/87: mknod d4/c18 0 2024-04-22T23:45:36.392 INFO:tasks.workunit.client.0.smithi031.stdout:2/127: dread - d6/f16 zero size 2024-04-22T23:45:36.394 INFO:tasks.workunit.client.0.smithi031.stdout:7/131: truncate d9/d14/f18 327572 0 2024-04-22T23:45:36.396 INFO:tasks.workunit.client.0.smithi031.stdout:1/96: mknod d1/db/c24 0 2024-04-22T23:45:36.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/128: write d6/f1c [514182,68354] 0 2024-04-22T23:45:36.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/129: chown d6/d1e 1990 1 2024-04-22T23:45:36.401 INFO:tasks.workunit.client.0.smithi031.stdout:0/113: chown d0/d1/d2/ce 7 1 2024-04-22T23:45:36.402 INFO:tasks.workunit.client.0.smithi031.stdout:3/88: creat d4/d6/d11/f19 x:0 0 0 2024-04-22T23:45:36.405 INFO:tasks.workunit.client.0.smithi031.stdout:9/172: dwrite d2/dd/d1b/f34 [0,4194304] 0 2024-04-22T23:45:36.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/114: chown d0/d1/d2/d17 17575640 1 2024-04-22T23:45:36.405 INFO:tasks.workunit.client.0.smithi031.stdout:4/192: dread db/d10/f2f [0,4194304] 0 2024-04-22T23:45:36.407 INFO:tasks.workunit.client.0.smithi031.stdout:9/173: fdatasync d2/dc/f1e 0 2024-04-22T23:45:36.407 INFO:tasks.workunit.client.0.smithi031.stdout:1/97: rename d1/de/f20 to d1/db/d16/f25 0 2024-04-22T23:45:36.412 INFO:tasks.workunit.client.0.smithi031.stdout:3/89: dread f3 [0,4194304] 0 2024-04-22T23:45:36.414 INFO:tasks.workunit.client.0.smithi031.stdout:7/132: link d9/d15/c19 d9/d15/c32 0 2024-04-22T23:45:36.416 INFO:tasks.workunit.client.0.smithi031.stdout:0/115: symlink d0/d1/d2/d3/d11/d1a/l29 0 2024-04-22T23:45:36.417 INFO:tasks.workunit.client.0.smithi031.stdout:4/193: dwrite db/f24 [0,4194304] 0 2024-04-22T23:45:36.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/116: stat d0/d1/d2/d3/d11/d1a/l29 0 2024-04-22T23:45:36.422 INFO:tasks.workunit.client.0.smithi031.stdout:0/117: dread - d0/d1/d2/d3/d11/f14 zero size 2024-04-22T23:45:36.422 INFO:tasks.workunit.client.0.smithi031.stdout:9/174: rename d2/c7 to d2/c37 0 2024-04-22T23:45:36.423 INFO:tasks.workunit.client.0.smithi031.stdout:1/98: creat d1/db/d16/d1f/f26 x:0 0 0 2024-04-22T23:45:36.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/175: dread - d2/dd/d1b/f27 zero size 2024-04-22T23:45:36.425 INFO:tasks.workunit.client.0.smithi031.stdout:4/194: creat db/d14/f3a x:0 0 0 2024-04-22T23:45:36.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/176: fsync d2/f5 0 2024-04-22T23:45:36.426 INFO:tasks.workunit.client.0.smithi031.stdout:9/177: dread - d2/dd/f2b zero size 2024-04-22T23:45:36.427 INFO:tasks.workunit.client.0.smithi031.stdout:3/90: symlink d4/l1a 0 2024-04-22T23:45:36.428 INFO:tasks.workunit.client.0.smithi031.stdout:2/130: write d6/dc/f1a [1320334,5099] 0 2024-04-22T23:45:36.432 INFO:tasks.workunit.client.0.smithi031.stdout:4/195: mkdir db/d31/d3b 0 2024-04-22T23:45:36.433 INFO:tasks.workunit.client.0.smithi031.stdout:4/196: fsync db/f24 0 2024-04-22T23:45:36.433 INFO:tasks.workunit.client.0.smithi031.stdout:9/178: readlink d2/dd/l2d 0 2024-04-22T23:45:36.433 INFO:tasks.workunit.client.0.smithi031.stdout:4/197: chown db/d14 1100736527 1 2024-04-22T23:45:36.434 INFO:tasks.workunit.client.0.smithi031.stdout:9/179: fsync d2/dd/d1b/f34 0 2024-04-22T23:45:36.434 INFO:tasks.workunit.client.0.smithi031.stdout:3/91: truncate f3 2742848 0 2024-04-22T23:45:36.435 INFO:tasks.workunit.client.0.smithi031.stdout:7/133: link d9/d15/c1d d9/d15/c33 0 2024-04-22T23:45:36.436 INFO:tasks.workunit.client.0.smithi031.stdout:7/134: chown d9/d15/f17 2 1 2024-04-22T23:45:36.437 INFO:tasks.workunit.client.0.smithi031.stdout:0/118: mknod d0/d13/d18/d1e/d25/c2a 0 2024-04-22T23:45:36.437 INFO:tasks.workunit.client.0.smithi031.stdout:7/135: dread - d9/d14/f23 zero size 2024-04-22T23:45:36.439 INFO:tasks.workunit.client.0.smithi031.stdout:7/136: fdatasync d9/d15/f17 0 2024-04-22T23:45:36.440 INFO:tasks.workunit.client.0.smithi031.stdout:9/180: dwrite d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:36.441 INFO:tasks.workunit.client.0.smithi031.stdout:9/181: fdatasync d2/dc/ff 0 2024-04-22T23:45:36.442 INFO:tasks.workunit.client.0.smithi031.stdout:4/198: mkdir db/d14/d2d/d3c 0 2024-04-22T23:45:36.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:35 smithi031 ceph-mon[26721]: pgmap v24: 65 pgs: 65 active+clean; 293 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.9 MiB/s rd, 20 MiB/s wr, 359 op/s 2024-04-22T23:45:36.448 INFO:tasks.workunit.client.0.smithi031.stdout:1/99: link d1/db/d16/l17 d1/db/l27 0 2024-04-22T23:45:36.453 INFO:tasks.workunit.client.0.smithi031.stdout:9/182: rmdir d2/dc 39 2024-04-22T23:45:36.455 INFO:tasks.workunit.client.0.smithi031.stdout:4/199: write db/d10/f2f [201509,114073] 0 2024-04-22T23:45:36.456 INFO:tasks.workunit.client.0.smithi031.stdout:4/200: fdatasync db/d10/f32 0 2024-04-22T23:45:36.458 INFO:tasks.workunit.client.0.smithi031.stdout:1/100: mkdir d1/db/d14/d28 0 2024-04-22T23:45:36.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/183: stat d2/f8 0 2024-04-22T23:45:36.461 INFO:tasks.workunit.client.0.smithi031.stdout:2/131: getdents d6/dc/d17 0 2024-04-22T23:45:36.463 INFO:tasks.workunit.client.0.smithi031.stdout:4/201: dwrite f9 [4194304,4194304] 0 2024-04-22T23:45:36.493 INFO:tasks.workunit.client.0.smithi031.stdout:0/119: getdents d0/d1/d2 0 2024-04-22T23:45:36.493 INFO:tasks.workunit.client.0.smithi031.stdout:4/202: mkdir db/d10/d3d 0 2024-04-22T23:45:36.495 INFO:tasks.workunit.client.0.smithi031.stdout:3/92: write f3 [3640531,96150] 0 2024-04-22T23:45:36.495 INFO:tasks.workunit.client.0.smithi031.stdout:4/203: chown db/d31/d3b 870537 1 2024-04-22T23:45:36.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/132: link f4 d6/f22 0 2024-04-22T23:45:36.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/137: dread f7 [0,4194304] 0 2024-04-22T23:45:36.499 INFO:tasks.workunit.client.0.smithi031.stdout:3/93: chown d4/d6/cd 12050 1 2024-04-22T23:45:36.499 INFO:tasks.workunit.client.0.smithi031.stdout:0/120: rename d0/d13/d18/d26 to d0/d1/d28/d2b 0 2024-04-22T23:45:36.500 INFO:tasks.workunit.client.0.smithi031.stdout:9/184: dwrite d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:36.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/94: truncate d4/f5 1504665 0 2024-04-22T23:45:36.501 INFO:tasks.workunit.client.0.smithi031.stdout:4/204: mknod db/c3e 0 2024-04-22T23:45:36.502 INFO:tasks.workunit.client.0.smithi031.stdout:3/95: readlink d4/l1a 0 2024-04-22T23:45:36.502 INFO:tasks.workunit.client.0.smithi031.stdout:9/185: write d2/f35 [928810,51064] 0 2024-04-22T23:45:36.504 INFO:tasks.workunit.client.0.smithi031.stdout:0/121: readlink d0/d1/d2/d3/l12 0 2024-04-22T23:45:36.505 INFO:tasks.workunit.client.0.smithi031.stdout:0/122: chown d0/d1/d2/d3/l16 6080 1 2024-04-22T23:45:36.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/123: write d0/d1/d2/d3/d11/f14 [965560,91301] 0 2024-04-22T23:45:36.509 INFO:tasks.workunit.client.0.smithi031.stdout:0/124: dread - d0/d1/d2/d3/fd zero size 2024-04-22T23:45:36.512 INFO:tasks.workunit.client.0.smithi031.stdout:4/205: mknod db/d14/d2d/c3f 0 2024-04-22T23:45:36.512 INFO:tasks.workunit.client.0.smithi031.stdout:4/206: fdatasync db/f24 0 2024-04-22T23:45:36.513 INFO:tasks.workunit.client.0.smithi031.stdout:4/207: chown db/d10/c15 226689414 1 2024-04-22T23:45:36.517 INFO:tasks.workunit.client.0.smithi031.stdout:0/125: symlink d0/d13/l2c 0 2024-04-22T23:45:36.519 INFO:tasks.workunit.client.0.smithi031.stdout:3/96: rename d4/d6/f9 to d4/d10/f1b 0 2024-04-22T23:45:36.528 INFO:tasks.workunit.client.0.smithi031.stdout:0/126: write d0/d1/d2/fa [3043888,117317] 0 2024-04-22T23:45:36.530 INFO:tasks.workunit.client.0.smithi031.stdout:3/97: creat d4/d6/d11/f1c x:0 0 0 2024-04-22T23:45:36.531 INFO:tasks.workunit.client.0.smithi031.stdout:4/208: fdatasync db/fc 0 2024-04-22T23:45:36.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/133: write d6/dc/f15 [555057,9249] 0 2024-04-22T23:45:36.534 INFO:tasks.workunit.client.0.smithi031.stdout:9/186: getdents d2 0 2024-04-22T23:45:36.534 INFO:tasks.workunit.client.0.smithi031.stdout:0/127: rename d0 to d0/d1/d2d 22 2024-04-22T23:45:36.535 INFO:tasks.workunit.client.0.smithi031.stdout:3/98: write d4/d6/da/ff [822030,80070] 0 2024-04-22T23:45:36.536 INFO:tasks.workunit.client.0.smithi031.stdout:7/138: dwrite d9/d14/f31 [0,4194304] 0 2024-04-22T23:45:36.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/99: chown d4/d10/f1b 757573 1 2024-04-22T23:45:36.538 INFO:tasks.workunit.client.0.smithi031.stdout:2/134: chown d6/l21 241075 1 2024-04-22T23:45:36.538 INFO:tasks.workunit.client.0.smithi031.stdout:0/128: dread d0/d1/d2/f9 [4194304,4194304] 0 2024-04-22T23:45:36.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/100: fsync d4/d6/f16 0 2024-04-22T23:45:36.539 INFO:tasks.workunit.client.0.smithi031.stdout:4/209: mkdir db/d31/d40 0 2024-04-22T23:45:36.544 INFO:tasks.workunit.client.0.smithi031.stdout:7/139: mknod d9/d14/d22/c34 0 2024-04-22T23:45:36.545 INFO:tasks.workunit.client.0.smithi031.stdout:9/187: mkdir d2/dc/d21/d38 0 2024-04-22T23:45:36.546 INFO:tasks.workunit.client.0.smithi031.stdout:4/210: dread db/fc [0,4194304] 0 2024-04-22T23:45:36.549 INFO:tasks.workunit.client.0.smithi031.stdout:3/101: dread d4/f5 [0,4194304] 0 2024-04-22T23:45:36.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/188: write d2/d15/f30 [172549,41597] 0 2024-04-22T23:45:36.550 INFO:tasks.workunit.client.0.smithi031.stdout:7/140: creat d9/d15/f35 x:0 0 0 2024-04-22T23:45:36.553 INFO:tasks.workunit.client.0.smithi031.stdout:0/129: dwrite d0/d1/d2/fa [0,4194304] 0 2024-04-22T23:45:36.553 INFO:tasks.workunit.client.0.smithi031.stdout:2/135: dwrite d6/ff [0,4194304] 0 2024-04-22T23:45:36.554 INFO:tasks.workunit.client.0.smithi031.stdout:3/102: dread - d4/d6/da/f13 zero size 2024-04-22T23:45:36.556 INFO:tasks.workunit.client.0.smithi031.stdout:5/118: sync 2024-04-22T23:45:36.558 INFO:tasks.workunit.client.0.smithi031.stdout:3/103: write d4/d6/da/ff [1862898,71956] 0 2024-04-22T23:45:36.559 INFO:tasks.workunit.client.0.smithi031.stdout:4/211: dread db/f24 [0,4194304] 0 2024-04-22T23:45:36.561 INFO:tasks.workunit.client.0.smithi031.stdout:5/119: symlink db/d16/l28 0 2024-04-22T23:45:36.562 INFO:tasks.workunit.client.0.smithi031.stdout:5/120: write db/d16/d1d/f22 [163124,121914] 0 2024-04-22T23:45:36.563 INFO:tasks.workunit.client.0.smithi031.stdout:5/121: write f0 [1908830,85488] 0 2024-04-22T23:45:36.566 INFO:tasks.workunit.client.0.smithi031.stdout:7/141: link d9/fa d9/d14/d22/d28/f36 0 2024-04-22T23:45:36.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/142: write f3 [3022908,96326] 0 2024-04-22T23:45:36.567 INFO:tasks.workunit.client.0.smithi031.stdout:3/104: creat d4/d6/da/f1d x:0 0 0 2024-04-22T23:45:36.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/212: chown db/d14/c1a 2840630 1 2024-04-22T23:45:36.569 INFO:tasks.workunit.client.0.smithi031.stdout:3/105: chown d4/d6/f7 536517 1 2024-04-22T23:45:36.570 INFO:tasks.workunit.client.0.smithi031.stdout:2/136: link c0 d6/d1e/c23 0 2024-04-22T23:45:36.571 INFO:tasks.workunit.client.0.smithi031.stdout:4/213: dread db/d31/f33 [0,4194304] 0 2024-04-22T23:45:36.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/122: creat db/f29 x:0 0 0 2024-04-22T23:45:36.574 INFO:tasks.workunit.client.0.smithi031.stdout:5/123: write db/d16/d1d/f22 [1245360,34632] 0 2024-04-22T23:45:36.575 INFO:tasks.workunit.client.0.smithi031.stdout:5/124: chown db/d16/d1d/d24/d27 10520149 1 2024-04-22T23:45:36.579 INFO:tasks.workunit.client.0.smithi031.stdout:3/106: readlink d4/l14 0 2024-04-22T23:45:36.582 INFO:tasks.workunit.client.0.smithi031.stdout:9/189: truncate d2/dd/d1b/f33 3997863 0 2024-04-22T23:45:36.583 INFO:tasks.workunit.client.0.smithi031.stdout:5/125: mkdir db/d16/d2a 0 2024-04-22T23:45:36.584 INFO:tasks.workunit.client.0.smithi031.stdout:2/137: mkdir d6/d1e/d24 0 2024-04-22T23:45:36.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/214: truncate db/d14/d1d/f20 919745 0 2024-04-22T23:45:36.586 INFO:tasks.workunit.client.0.smithi031.stdout:9/190: creat d2/dd/d2f/f39 x:0 0 0 2024-04-22T23:45:36.591 INFO:tasks.workunit.client.0.smithi031.stdout:5/126: dwrite db/d16/f15 [0,4194304] 0 2024-04-22T23:45:36.593 INFO:tasks.workunit.client.0.smithi031.stdout:3/107: creat d4/f1e x:0 0 0 2024-04-22T23:45:36.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/215: mkdir db/d10/d3d/d41 0 2024-04-22T23:45:36.601 INFO:tasks.workunit.client.0.smithi031.stdout:7/143: dwrite d9/d14/f18 [0,4194304] 0 2024-04-22T23:45:36.602 INFO:tasks.workunit.client.0.smithi031.stdout:5/127: rmdir db 39 2024-04-22T23:45:36.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/128: fsync f5 0 2024-04-22T23:45:36.604 INFO:tasks.workunit.client.0.smithi031.stdout:3/108: stat d4/d6/d11/c17 0 2024-04-22T23:45:36.605 INFO:tasks.workunit.client.0.smithi031.stdout:7/144: read d9/fa [3340433,98736] 0 2024-04-22T23:45:36.613 INFO:tasks.workunit.client.0.smithi031.stdout:2/138: dwrite f4 [0,4194304] 0 2024-04-22T23:45:36.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/129: write db/d16/f21 [4061624,95906] 0 2024-04-22T23:45:36.619 INFO:tasks.workunit.client.0.smithi031.stdout:5/130: truncate f0 4578360 0 2024-04-22T23:45:36.626 INFO:tasks.workunit.client.0.smithi031.stdout:9/191: read d2/dd/d1b/f33 [2883852,80179] 0 2024-04-22T23:45:36.626 INFO:tasks.workunit.client.0.smithi031.stdout:7/145: dwrite f8 [0,4194304] 0 2024-04-22T23:45:36.629 INFO:tasks.workunit.client.0.smithi031.stdout:6/178: sync 2024-04-22T23:45:36.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/131: dwrite db/f23 [0,4194304] 0 2024-04-22T23:45:36.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/109: creat d4/d6/f1f x:0 0 0 2024-04-22T23:45:36.631 INFO:tasks.workunit.client.0.smithi031.stdout:6/179: dread - d3/f25 zero size 2024-04-22T23:45:36.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/110: chown d4/f5 29 1 2024-04-22T23:45:36.632 INFO:tasks.workunit.client.0.smithi031.stdout:6/180: write d3/f38 [953784,97508] 0 2024-04-22T23:45:36.633 INFO:tasks.workunit.client.0.smithi031.stdout:3/111: chown d4/d10 3142 1 2024-04-22T23:45:36.634 INFO:tasks.workunit.client.0.smithi031.stdout:3/112: read - d4/d6/d11/f1c zero size 2024-04-22T23:45:36.638 INFO:tasks.workunit.client.0.smithi031.stdout:4/216: truncate f4 4076751 0 2024-04-22T23:45:36.640 INFO:tasks.workunit.client.0.smithi031.stdout:4/217: write db/d10/f32 [282604,26213] 0 2024-04-22T23:45:36.648 INFO:tasks.workunit.client.0.smithi031.stdout:6/181: dwrite d3/f15 [0,4194304] 0 2024-04-22T23:45:36.649 INFO:tasks.workunit.client.0.smithi031.stdout:5/132: dread f7 [0,4194304] 0 2024-04-22T23:45:36.649 INFO:tasks.workunit.client.0.smithi031.stdout:4/218: creat db/d10/d2a/d30/f42 x:0 0 0 2024-04-22T23:45:36.650 INFO:tasks.workunit.client.0.smithi031.stdout:7/146: creat d9/f37 x:0 0 0 2024-04-22T23:45:36.651 INFO:tasks.workunit.client.0.smithi031.stdout:4/219: dread - db/d14/f3a zero size 2024-04-22T23:45:36.654 INFO:tasks.workunit.client.0.smithi031.stdout:4/220: write db/d10/f12 [1467064,83225] 0 2024-04-22T23:45:36.654 INFO:tasks.workunit.client.0.smithi031.stdout:3/113: getdents d4/d6 0 2024-04-22T23:45:36.656 INFO:tasks.workunit.client.0.smithi031.stdout:7/147: chown d9/f1c 2 1 2024-04-22T23:45:36.660 INFO:tasks.workunit.client.0.smithi031.stdout:6/182: dread d3/f15 [0,4194304] 0 2024-04-22T23:45:36.661 INFO:tasks.workunit.client.0.smithi031.stdout:3/114: unlink d4/l12 0 2024-04-22T23:45:36.661 INFO:tasks.workunit.client.0.smithi031.stdout:5/133: dwrite f5 [0,4194304] 0 2024-04-22T23:45:36.662 INFO:tasks.workunit.client.0.smithi031.stdout:9/192: truncate d2/dd/d1b/f32 138036 0 2024-04-22T23:45:36.664 INFO:tasks.workunit.client.0.smithi031.stdout:5/134: chown c9 72 1 2024-04-22T23:45:36.665 INFO:tasks.workunit.client.0.smithi031.stdout:7/148: dread f3 [0,4194304] 0 2024-04-22T23:45:36.669 INFO:tasks.workunit.client.0.smithi031.stdout:3/115: unlink d4/d6/f1f 0 2024-04-22T23:45:36.670 INFO:tasks.workunit.client.0.smithi031.stdout:6/183: dread d3/f38 [0,4194304] 0 2024-04-22T23:45:36.671 INFO:tasks.workunit.client.0.smithi031.stdout:9/193: symlink d2/dd/d1b/l3a 0 2024-04-22T23:45:36.673 INFO:tasks.workunit.client.0.smithi031.stdout:3/116: write d4/f1e [670642,1992] 0 2024-04-22T23:45:36.673 INFO:tasks.workunit.client.0.smithi031.stdout:4/221: dwrite db/d31/f33 [0,4194304] 0 2024-04-22T23:45:36.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/149: chown d9/d14/d1f/c26 764 1 2024-04-22T23:45:36.678 INFO:tasks.workunit.client.0.smithi031.stdout:4/222: write db/d14/d2d/f37 [726058,110737] 0 2024-04-22T23:45:36.678 INFO:tasks.workunit.client.0.smithi031.stdout:5/135: fdatasync f0 0 2024-04-22T23:45:36.679 INFO:tasks.workunit.client.0.smithi031.stdout:9/194: rename d2/dd/l11 to d2/dd/d2f/l3b 0 2024-04-22T23:45:36.681 INFO:tasks.workunit.client.0.smithi031.stdout:7/150: write d9/d15/f17 [2098446,121854] 0 2024-04-22T23:45:36.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/223: chown l5 530419838 1 2024-04-22T23:45:36.683 INFO:tasks.workunit.client.0.smithi031.stdout:9/195: chown d2/dd/d1b/l3a 59472 1 2024-04-22T23:45:36.684 INFO:tasks.workunit.client.0.smithi031.stdout:4/224: write db/f24 [3420491,106474] 0 2024-04-22T23:45:36.684 INFO:tasks.workunit.client.0.smithi031.stdout:7/151: truncate d9/d15/f35 152444 0 2024-04-22T23:45:36.685 INFO:tasks.workunit.client.0.smithi031.stdout:6/184: dwrite d3/f18 [0,4194304] 0 2024-04-22T23:45:36.685 INFO:tasks.workunit.client.0.smithi031.stdout:5/136: unlink db/d16/f1a 0 2024-04-22T23:45:36.687 INFO:tasks.workunit.client.0.smithi031.stdout:5/137: write db/d16/d1d/f22 [1565566,41899] 0 2024-04-22T23:45:36.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/225: dwrite db/d14/d2d/f35 [0,4194304] 0 2024-04-22T23:45:36.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/226: chown db 1367384 1 2024-04-22T23:45:36.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/152: mkdir d9/d38 0 2024-04-22T23:45:36.708 INFO:tasks.workunit.client.0.smithi031.stdout:7/153: fdatasync d9/d14/f31 0 2024-04-22T23:45:36.711 INFO:tasks.workunit.client.0.smithi031.stdout:6/185: rename d3/d2a/c2e to d3/c3f 0 2024-04-22T23:45:36.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/186: read d3/f18 [385803,88356] 0 2024-04-22T23:45:36.712 INFO:tasks.workunit.client.0.smithi031.stdout:8/120: sync 2024-04-22T23:45:36.713 INFO:tasks.workunit.client.0.smithi031.stdout:8/121: dread - d5/da/f1e zero size 2024-04-22T23:45:36.714 INFO:tasks.workunit.client.0.smithi031.stdout:8/122: fsync d5/f8 0 2024-04-22T23:45:36.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/196: fsync d2/dd/d1b/f32 0 2024-04-22T23:45:36.718 INFO:tasks.workunit.client.0.smithi031.stdout:3/117: truncate d4/d6/da/f13 347745 0 2024-04-22T23:45:36.718 INFO:tasks.workunit.client.0.smithi031.stdout:4/227: rename db/d31/d3b to db/d31/d40/d43 0 2024-04-22T23:45:36.719 INFO:tasks.workunit.client.0.smithi031.stdout:7/154: rename d9/d15 to d9/d15/d39 22 2024-04-22T23:45:36.720 INFO:tasks.workunit.client.0.smithi031.stdout:4/228: chown db/l23 47446 1 2024-04-22T23:45:36.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/187: dwrite d3/f32 [0,4194304] 0 2024-04-22T23:45:36.722 INFO:tasks.workunit.client.0.smithi031.stdout:5/138: getdents db/d16/d1d/d24 0 2024-04-22T23:45:36.725 INFO:tasks.workunit.client.0.smithi031.stdout:8/123: dwrite d5/da/d12/f1c [4194304,4194304] 0 2024-04-22T23:45:36.725 INFO:tasks.workunit.client.0.smithi031.stdout:5/139: dread f7 [0,4194304] 0 2024-04-22T23:45:36.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/124: readlink d5/da/d12/l23 0 2024-04-22T23:45:36.736 INFO:tasks.workunit.client.0.smithi031.stdout:5/140: rmdir db 39 2024-04-22T23:45:36.739 INFO:tasks.workunit.client.0.smithi031.stdout:8/125: unlink d5/da/c11 0 2024-04-22T23:45:36.743 INFO:tasks.workunit.client.0.smithi031.stdout:9/197: link d2/dc/f10 d2/f3c 0 2024-04-22T23:45:36.746 INFO:tasks.workunit.client.0.smithi031.stdout:3/118: chown d4/d6/d11/f19 293609 1 2024-04-22T23:45:36.747 INFO:tasks.workunit.client.0.smithi031.stdout:6/188: dwrite d3/f17 [0,4194304] 0 2024-04-22T23:45:36.751 INFO:tasks.workunit.client.0.smithi031.stdout:8/126: symlink d5/da/l25 0 2024-04-22T23:45:36.756 INFO:tasks.workunit.client.0.smithi031.stdout:6/189: dwrite d3/f17 [0,4194304] 0 2024-04-22T23:45:36.757 INFO:tasks.workunit.client.0.smithi031.stdout:5/141: creat db/d16/f2b x:0 0 0 2024-04-22T23:45:36.759 INFO:tasks.workunit.client.0.smithi031.stdout:5/142: write db/fd [78015,21714] 0 2024-04-22T23:45:36.760 INFO:tasks.workunit.client.0.smithi031.stdout:3/119: mknod d4/d10/c20 0 2024-04-22T23:45:36.761 INFO:tasks.workunit.client.0.smithi031.stdout:3/120: write d4/d6/d11/f19 [216665,91949] 0 2024-04-22T23:45:36.762 INFO:tasks.workunit.client.0.smithi031.stdout:4/229: rename db/d10/c2e to db/c44 0 2024-04-22T23:45:36.769 INFO:tasks.workunit.client.0.smithi031.stdout:3/121: dwrite d4/d6/d11/f19 [0,4194304] 0 2024-04-22T23:45:36.770 INFO:tasks.workunit.client.0.smithi031.stdout:4/230: creat db/d14/f45 x:0 0 0 2024-04-22T23:45:36.771 INFO:tasks.workunit.client.0.smithi031.stdout:6/190: mkdir d3/d2a/d3c/d40 0 2024-04-22T23:45:36.772 INFO:tasks.workunit.client.0.smithi031.stdout:9/198: mkdir d2/dc/d21/d3d 0 2024-04-22T23:45:36.773 INFO:tasks.workunit.client.0.smithi031.stdout:5/143: truncate db/d16/f18 1604486 0 2024-04-22T23:45:36.775 INFO:tasks.workunit.client.0.smithi031.stdout:4/231: creat db/d10/d3d/f46 x:0 0 0 2024-04-22T23:45:36.776 INFO:tasks.workunit.client.0.smithi031.stdout:6/191: mknod d3/d1f/c41 0 2024-04-22T23:45:36.778 INFO:tasks.workunit.client.0.smithi031.stdout:5/144: dread db/f23 [0,4194304] 0 2024-04-22T23:45:36.780 INFO:tasks.workunit.client.0.smithi031.stdout:4/232: mkdir db/d10/d2a/d47 0 2024-04-22T23:45:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:4/233: truncate db/d10/f32 812442 0 2024-04-22T23:45:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:6/192: symlink d3/d1f/l42 0 2024-04-22T23:45:36.782 INFO:tasks.workunit.client.0.smithi031.stdout:4/234: chown db/d14/d2d/d3c 93999536 1 2024-04-22T23:45:36.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/199: rmdir d2/dd/d2f 39 2024-04-22T23:45:36.792 INFO:tasks.workunit.client.0.smithi031.stdout:6/193: creat d3/d2a/d3c/f43 x:0 0 0 2024-04-22T23:45:36.795 INFO:tasks.workunit.client.0.smithi031.stdout:4/235: read db/d14/d1d/f20 [792582,24952] 0 2024-04-22T23:45:36.796 INFO:tasks.workunit.client.0.smithi031.stdout:9/200: unlink d2/dd/l2d 0 2024-04-22T23:45:36.798 INFO:tasks.workunit.client.0.smithi031.stdout:5/145: creat db/d16/d2a/f2c x:0 0 0 2024-04-22T23:45:36.799 INFO:tasks.workunit.client.0.smithi031.stdout:5/146: write db/d16/f21 [3471084,56437] 0 2024-04-22T23:45:36.799 INFO:tasks.workunit.client.0.smithi031.stdout:6/194: rmdir d3/d35 39 2024-04-22T23:45:36.802 INFO:tasks.workunit.client.0.smithi031.stdout:4/236: mkdir db/d10/d2a/d48 0 2024-04-22T23:45:36.803 INFO:tasks.workunit.client.0.smithi031.stdout:0/130: sync 2024-04-22T23:45:36.803 INFO:tasks.workunit.client.0.smithi031.stdout:1/101: sync 2024-04-22T23:45:36.803 INFO:tasks.workunit.client.0.smithi031.stdout:2/139: sync 2024-04-22T23:45:36.806 INFO:tasks.workunit.client.0.smithi031.stdout:5/147: dwrite db/d16/f2b [0,4194304] 0 2024-04-22T23:45:36.808 INFO:tasks.workunit.client.0.smithi031.stdout:5/148: write f0 [2994297,14288] 0 2024-04-22T23:45:36.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/237: mkdir db/d14/d2d/d49 0 2024-04-22T23:45:36.813 INFO:tasks.workunit.client.0.smithi031.stdout:9/201: dwrite d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:36.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/202: dread d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:36.819 INFO:tasks.workunit.client.0.smithi031.stdout:2/140: creat d6/dc/d17/f25 x:0 0 0 2024-04-22T23:45:36.819 INFO:tasks.workunit.client.0.smithi031.stdout:9/203: write d2/dc/f1e [322742,88859] 0 2024-04-22T23:45:36.819 INFO:tasks.workunit.client.0.smithi031.stdout:3/122: write d4/f5 [49387,110857] 0 2024-04-22T23:45:36.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/204: chown d2/dd 1011 1 2024-04-22T23:45:36.820 INFO:tasks.workunit.client.0.smithi031.stdout:2/141: fdatasync d6/f22 0 2024-04-22T23:45:36.822 INFO:tasks.workunit.client.0.smithi031.stdout:3/123: read d4/d6/d11/f19 [1932255,129998] 0 2024-04-22T23:45:36.823 INFO:tasks.workunit.client.0.smithi031.stdout:2/142: read d6/f11 [362906,87554] 0 2024-04-22T23:45:36.826 INFO:tasks.workunit.client.0.smithi031.stdout:1/102: rename d1/db/d14 to d1/d21/d29 0 2024-04-22T23:45:36.840 INFO:tasks.workunit.client.0.smithi031.stdout:9/205: dwrite d2/f5 [0,4194304] 0 2024-04-22T23:45:36.840 INFO:tasks.workunit.client.0.smithi031.stdout:9/206: chown d2/dc/l23 2 1 2024-04-22T23:45:36.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/124: symlink d4/d6/d11/l21 0 2024-04-22T23:45:36.844 INFO:tasks.workunit.client.0.smithi031.stdout:5/149: rename f0 to db/d16/d1d/d24/d27/f2d 0 2024-04-22T23:45:36.855 INFO:tasks.workunit.client.0.smithi031.stdout:7/155: sync 2024-04-22T23:45:36.857 INFO:tasks.workunit.client.0.smithi031.stdout:9/207: chown d2/c37 675 1 2024-04-22T23:45:36.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/125: rmdir d4/d6/d11 39 2024-04-22T23:45:36.869 INFO:tasks.workunit.client.0.smithi031.stdout:1/103: creat d1/d21/d29/d28/f2a x:0 0 0 2024-04-22T23:45:36.871 INFO:tasks.workunit.client.0.smithi031.stdout:6/195: getdents d3/d2a 0 2024-04-22T23:45:36.871 INFO:tasks.workunit.client.0.smithi031.stdout:7/156: dread d9/fa [0,4194304] 0 2024-04-22T23:45:36.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/196: readlink d3/d1f/l27 0 2024-04-22T23:45:36.872 INFO:tasks.workunit.client.0.smithi031.stdout:0/131: getdents d0/d13/d18/d1e/d25 0 2024-04-22T23:45:36.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/143: creat d6/dc/f26 x:0 0 0 2024-04-22T23:45:36.874 INFO:tasks.workunit.client.0.smithi031.stdout:5/150: dwrite f5 [0,4194304] 0 2024-04-22T23:45:36.875 INFO:tasks.workunit.client.0.smithi031.stdout:0/132: write d0/d1/d2/d3/fd [152822,23670] 0 2024-04-22T23:45:36.876 INFO:tasks.workunit.client.0.smithi031.stdout:9/208: mkdir d2/dc/d3e 0 2024-04-22T23:45:36.879 INFO:tasks.workunit.client.0.smithi031.stdout:4/238: rename db/d14/c39 to db/d31/c4a 0 2024-04-22T23:45:36.880 INFO:tasks.workunit.client.0.smithi031.stdout:1/104: mknod d1/d21/d29/d28/c2b 0 2024-04-22T23:45:36.883 INFO:tasks.workunit.client.0.smithi031.stdout:6/197: rmdir d3 39 2024-04-22T23:45:36.884 INFO:tasks.workunit.client.0.smithi031.stdout:5/151: dread db/d16/f2b [0,4194304] 0 2024-04-22T23:45:36.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/105: dread d1/de/f11 [0,4194304] 0 2024-04-22T23:45:36.888 INFO:tasks.workunit.client.0.smithi031.stdout:0/133: symlink d0/d13/d18/d1e/d25/l2e 0 2024-04-22T23:45:36.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/134: dwrite d0/d1/d2/d3/f1d [0,4194304] 0 2024-04-22T23:45:36.898 INFO:tasks.workunit.client.0.smithi031.stdout:6/198: write d3/d1f/f20 [439072,94688] 0 2024-04-22T23:45:36.899 INFO:tasks.workunit.client.0.smithi031.stdout:8/127: sync 2024-04-22T23:45:36.902 INFO:tasks.workunit.client.0.smithi031.stdout:0/135: dread d0/d1/d2/d3/f1d [0,4194304] 0 2024-04-22T23:45:36.904 INFO:tasks.workunit.client.0.smithi031.stdout:0/136: write d0/d1/d2/d3/f1d [218167,54040] 0 2024-04-22T23:45:36.905 INFO:tasks.workunit.client.0.smithi031.stdout:0/137: readlink d0/d13/d18/d1e/d25/l2e 0 2024-04-22T23:45:36.906 INFO:tasks.workunit.client.0.smithi031.stdout:5/152: dread db/f1e [0,4194304] 0 2024-04-22T23:45:36.908 INFO:tasks.workunit.client.0.smithi031.stdout:3/126: mkdir d4/d6/d22 0 2024-04-22T23:45:36.909 INFO:tasks.workunit.client.0.smithi031.stdout:4/239: creat db/d10/d3d/d41/f4b x:0 0 0 2024-04-22T23:45:36.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/144: dwrite d6/f9 [0,4194304] 0 2024-04-22T23:45:36.917 INFO:tasks.workunit.client.0.smithi031.stdout:8/128: rename d5/f6 to d5/f26 0 2024-04-22T23:45:36.918 INFO:tasks.workunit.client.0.smithi031.stdout:8/129: chown d5/d1a 56 1 2024-04-22T23:45:36.919 INFO:tasks.workunit.client.0.smithi031.stdout:8/130: write d5/fb [1481193,24759] 0 2024-04-22T23:45:36.921 INFO:tasks.workunit.client.0.smithi031.stdout:1/106: unlink d1/de/c18 0 2024-04-22T23:45:36.925 INFO:tasks.workunit.client.0.smithi031.stdout:2/145: dwrite d6/dc/f26 [0,4194304] 0 2024-04-22T23:45:36.931 INFO:tasks.workunit.client.0.smithi031.stdout:1/107: dread d1/d21/d29/f23 [0,4194304] 0 2024-04-22T23:45:36.931 INFO:tasks.workunit.client.0.smithi031.stdout:5/153: creat db/d16/d1d/d24/f2e x:0 0 0 2024-04-22T23:45:36.934 INFO:tasks.workunit.client.0.smithi031.stdout:6/199: creat d3/d2a/d3c/d40/f44 x:0 0 0 2024-04-22T23:45:36.935 INFO:tasks.workunit.client.0.smithi031.stdout:7/157: dwrite d9/d15/f21 [0,4194304] 0 2024-04-22T23:45:36.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/131: write d5/f9 [3694857,114120] 0 2024-04-22T23:45:36.939 INFO:tasks.workunit.client.0.smithi031.stdout:5/154: chown db/d16/d2a/f2c 23 1 2024-04-22T23:45:36.939 INFO:tasks.workunit.client.0.smithi031.stdout:9/209: link d2/d15/l1f d2/dd/d2f/d2a/l3f 0 2024-04-22T23:45:36.942 INFO:tasks.workunit.client.0.smithi031.stdout:2/146: dwrite d6/dc/f26 [0,4194304] 0 2024-04-22T23:45:36.948 INFO:tasks.workunit.client.0.smithi031.stdout:0/138: truncate d0/ff 1874220 0 2024-04-22T23:45:36.950 INFO:tasks.workunit.client.0.smithi031.stdout:1/108: write d1/de/f11 [1178462,93203] 0 2024-04-22T23:45:36.951 INFO:tasks.workunit.client.0.smithi031.stdout:6/200: readlink d3/l21 0 2024-04-22T23:45:36.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/240: fsync db/d10/d3d/d41/f4b 0 2024-04-22T23:45:36.957 INFO:tasks.workunit.client.0.smithi031.stdout:4/241: dread db/d10/f32 [0,4194304] 0 2024-04-22T23:45:36.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/132: rename d5/da/d12/c1f to d5/da/d12/d1d/c27 0 2024-04-22T23:45:36.963 INFO:tasks.workunit.client.0.smithi031.stdout:5/155: creat db/d16/d1d/d24/f2f x:0 0 0 2024-04-22T23:45:36.966 INFO:tasks.workunit.client.0.smithi031.stdout:1/109: creat d1/db/d16/d1f/f2c x:0 0 0 2024-04-22T23:45:36.967 INFO:tasks.workunit.client.0.smithi031.stdout:1/110: chown d1/d21/d29/c13 1292904 1 2024-04-22T23:45:36.968 INFO:tasks.workunit.client.0.smithi031.stdout:1/111: chown d1/d21/d29/d28/f2a 51587002 1 2024-04-22T23:45:36.974 INFO:tasks.workunit.client.0.smithi031.stdout:3/127: truncate d4/fe 3001486 0 2024-04-22T23:45:36.979 INFO:tasks.workunit.client.0.smithi031.stdout:4/242: mknod db/d14/c4c 0 2024-04-22T23:45:36.982 INFO:tasks.workunit.client.0.smithi031.stdout:9/210: rename d2/dd/d2f to d2/dc/d21/d3d/d40 0 2024-04-22T23:45:36.982 INFO:tasks.workunit.client.0.smithi031.stdout:9/211: write d2/dc/ff [713260,104469] 0 2024-04-22T23:45:36.985 INFO:tasks.workunit.client.0.smithi031.stdout:8/133: mknod d5/da/d12/c28 0 2024-04-22T23:45:36.985 INFO:tasks.workunit.client.0.smithi031.stdout:5/156: creat db/d16/d2a/f30 x:0 0 0 2024-04-22T23:45:36.987 INFO:tasks.workunit.client.0.smithi031.stdout:1/112: dread d1/f8 [0,4194304] 0 2024-04-22T23:45:36.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/113: dread - d1/d21/d29/f1b zero size 2024-04-22T23:45:36.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/114: fdatasync d1/de/f11 0 2024-04-22T23:45:36.991 INFO:tasks.workunit.client.0.smithi031.stdout:5/157: dwrite db/d16/f21 [0,4194304] 0 2024-04-22T23:45:37.003 INFO:tasks.workunit.client.0.smithi031.stdout:4/243: symlink db/d10/d3d/l4d 0 2024-04-22T23:45:37.013 INFO:tasks.workunit.client.0.smithi031.stdout:6/201: creat d3/f45 x:0 0 0 2024-04-22T23:45:37.014 INFO:tasks.workunit.client.0.smithi031.stdout:6/202: dread - d3/d2a/d3c/d40/f44 zero size 2024-04-22T23:45:37.015 INFO:tasks.workunit.client.0.smithi031.stdout:1/115: symlink d1/d21/l2d 0 2024-04-22T23:45:37.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/158: mknod db/d16/d1d/d24/d27/c31 0 2024-04-22T23:45:37.021 INFO:tasks.workunit.client.0.smithi031.stdout:2/147: truncate d6/dc/f1a 386481 0 2024-04-22T23:45:37.023 INFO:tasks.workunit.client.0.smithi031.stdout:7/158: getdents d9/d14 0 2024-04-22T23:45:37.029 INFO:tasks.workunit.client.0.smithi031.stdout:4/244: dwrite f1 [0,4194304] 0 2024-04-22T23:45:37.045 INFO:tasks.workunit.client.0.smithi031.stdout:5/159: rmdir db/d16/d1d/d24 39 2024-04-22T23:45:37.048 INFO:tasks.workunit.client.0.smithi031.stdout:4/245: dread db/d10/f32 [0,4194304] 0 2024-04-22T23:45:37.049 INFO:tasks.workunit.client.0.smithi031.stdout:1/116: mkdir d1/d2e 0 2024-04-22T23:45:37.050 INFO:tasks.workunit.client.0.smithi031.stdout:1/117: write d1/d21/d29/f1b [256919,106747] 0 2024-04-22T23:45:37.051 INFO:tasks.workunit.client.0.smithi031.stdout:3/128: creat d4/d6/f23 x:0 0 0 2024-04-22T23:45:37.054 INFO:tasks.workunit.client.0.smithi031.stdout:1/118: dread d1/de/f11 [0,4194304] 0 2024-04-22T23:45:37.057 INFO:tasks.workunit.client.0.smithi031.stdout:8/134: getdents d5 0 2024-04-22T23:45:37.058 INFO:tasks.workunit.client.0.smithi031.stdout:9/212: dwrite d2/f20 [0,4194304] 0 2024-04-22T23:45:37.066 INFO:tasks.workunit.client.0.smithi031.stdout:2/148: link d6/dc/d17/c1b d6/d1e/c27 0 2024-04-22T23:45:37.068 INFO:tasks.workunit.client.0.smithi031.stdout:8/135: rmdir d5 39 2024-04-22T23:45:37.069 INFO:tasks.workunit.client.0.smithi031.stdout:2/149: read f4 [3619529,59317] 0 2024-04-22T23:45:37.070 INFO:tasks.workunit.client.0.smithi031.stdout:4/246: symlink db/d14/d2d/d49/l4e 0 2024-04-22T23:45:37.072 INFO:tasks.workunit.client.0.smithi031.stdout:5/160: dwrite db/d16/d1d/d24/f2e [0,4194304] 0 2024-04-22T23:45:37.072 INFO:tasks.workunit.client.0.smithi031.stdout:9/213: dwrite d2/dd/d1b/f36 [0,4194304] 0 2024-04-22T23:45:37.075 INFO:tasks.workunit.client.0.smithi031.stdout:7/159: dwrite d9/fa [0,4194304] 0 2024-04-22T23:45:37.078 INFO:tasks.workunit.client.0.smithi031.stdout:5/161: fdatasync db/d16/d1d/d24/d27/f2d 0 2024-04-22T23:45:37.078 INFO:tasks.workunit.client.0.smithi031.stdout:4/247: mkdir db/d31/d4f 0 2024-04-22T23:45:37.079 INFO:tasks.workunit.client.0.smithi031.stdout:2/150: mkdir d6/d1e/d28 0 2024-04-22T23:45:37.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/248: rename db/d14 to db/d14/d2d/d50 22 2024-04-22T23:45:37.081 INFO:tasks.workunit.client.0.smithi031.stdout:2/151: rename d6/d1e/d24 to d6/d1e/d24/d29 22 2024-04-22T23:45:37.082 INFO:tasks.workunit.client.0.smithi031.stdout:8/136: symlink d5/da/d12/d1d/l29 0 2024-04-22T23:45:37.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/249: dread - db/d10/d2a/d30/f42 zero size 2024-04-22T23:45:37.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/137: read d5/fb [5414225,73873] 0 2024-04-22T23:45:37.084 INFO:tasks.workunit.client.0.smithi031.stdout:8/138: write d5/da/d12/f1c [6012275,90843] 0 2024-04-22T23:45:37.085 INFO:tasks.workunit.client.0.smithi031.stdout:8/139: fdatasync d5/da/f1e 0 2024-04-22T23:45:37.087 INFO:tasks.workunit.client.0.smithi031.stdout:5/162: mkdir db/d16/d1d/d24/d32 0 2024-04-22T23:45:37.096 INFO:tasks.workunit.client.0.smithi031.stdout:4/250: creat db/d31/d40/d43/f51 x:0 0 0 2024-04-22T23:45:37.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/251: fdatasync db/d14/f45 0 2024-04-22T23:45:37.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/252: dread - db/d14/f45 zero size 2024-04-22T23:45:37.108 INFO:tasks.workunit.client.0.smithi031.stdout:5/163: symlink db/d16/d2a/l33 0 2024-04-22T23:45:37.110 INFO:tasks.workunit.client.0.smithi031.stdout:2/152: creat d6/d1e/d28/f2a x:0 0 0 2024-04-22T23:45:37.112 INFO:tasks.workunit.client.0.smithi031.stdout:7/160: write f2 [2277285,10374] 0 2024-04-22T23:45:37.113 INFO:tasks.workunit.client.0.smithi031.stdout:7/161: read f7 [2975923,118304] 0 2024-04-22T23:45:37.115 INFO:tasks.workunit.client.0.smithi031.stdout:3/129: symlink d4/d6/d22/l24 0 2024-04-22T23:45:37.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/140: link d5/fb d5/da/d12/f2a 0 2024-04-22T23:45:37.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/141: dwrite d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:37.125 INFO:tasks.workunit.client.0.smithi031.stdout:8/142: chown d5/da/d12/c28 9 1 2024-04-22T23:45:37.125 INFO:tasks.workunit.client.0.smithi031.stdout:8/143: dread - d5/da/df/f1b zero size 2024-04-22T23:45:37.186 INFO:tasks.workunit.client.0.smithi031.stdout:9/214: truncate d2/dc/f12 302809 0 2024-04-22T23:45:37.186 INFO:tasks.workunit.client.0.smithi031.stdout:4/253: fdatasync f4 0 2024-04-22T23:45:37.187 INFO:tasks.workunit.client.0.smithi031.stdout:5/164: rename db/l17 to db/d16/d1d/d24/d27/l34 0 2024-04-22T23:45:37.187 INFO:tasks.workunit.client.0.smithi031.stdout:2/153: creat d6/d1e/d24/f2b x:0 0 0 2024-04-22T23:45:37.189 INFO:tasks.workunit.client.0.smithi031.stdout:2/154: write f4 [434664,126698] 0 2024-04-22T23:45:37.190 INFO:tasks.workunit.client.0.smithi031.stdout:8/144: dread d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:37.191 INFO:tasks.workunit.client.0.smithi031.stdout:8/145: chown d5/da/df/c18 11284 1 2024-04-22T23:45:37.192 INFO:tasks.workunit.client.0.smithi031.stdout:8/146: read - d5/da/f1e zero size 2024-04-22T23:45:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:36 smithi031 ceph-mon[26721]: Standby manager daemon smithi031.fazdcv restarted 2024-04-22T23:45:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:36 smithi031 ceph-mon[26721]: Standby manager daemon smithi031.fazdcv started 2024-04-22T23:45:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:36 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/crt"}]: dispatch 2024-04-22T23:45:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:36 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:45:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:36 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/key"}]: dispatch 2024-04-22T23:45:37.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:36 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:45:37.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/215: rename d2/d15/f30 to d2/dc/d3e/f41 0 2024-04-22T23:45:37.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/216: chown d2/d15/l22 0 1 2024-04-22T23:45:37.199 INFO:tasks.workunit.client.0.smithi031.stdout:4/254: creat db/d10/d3d/d41/f52 x:0 0 0 2024-04-22T23:45:37.201 INFO:tasks.workunit.client.0.smithi031.stdout:9/217: dwrite d2/dd/d1b/f32 [0,4194304] 0 2024-04-22T23:45:37.206 INFO:tasks.workunit.client.0.smithi031.stdout:7/162: getdents d9/d14/d1f 0 2024-04-22T23:45:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:36 smithi044 ceph-mon[34463]: Standby manager daemon smithi031.fazdcv restarted 2024-04-22T23:45:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:36 smithi044 ceph-mon[34463]: Standby manager daemon smithi031.fazdcv started 2024-04-22T23:45:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:36 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/crt"}]: dispatch 2024-04-22T23:45:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:36 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:45:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:36 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.fazdcv/key"}]: dispatch 2024-04-22T23:45:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:36 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:45:37.215 INFO:tasks.workunit.client.0.smithi031.stdout:5/165: truncate f5 15567 0 2024-04-22T23:45:37.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/166: write db/d16/d1d/f1f [192157,3909] 0 2024-04-22T23:45:37.220 INFO:tasks.workunit.client.0.smithi031.stdout:9/218: mknod d2/dd/d1b/c42 0 2024-04-22T23:45:37.221 INFO:tasks.workunit.client.0.smithi031.stdout:9/219: write d2/dc/f25 [252944,42229] 0 2024-04-22T23:45:37.222 INFO:tasks.workunit.client.0.smithi031.stdout:4/255: dread f4 [0,4194304] 0 2024-04-22T23:45:37.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/163: mkdir d9/d15/d3a 0 2024-04-22T23:45:37.224 INFO:tasks.workunit.client.0.smithi031.stdout:5/167: rmdir db/d16/d2a 39 2024-04-22T23:45:37.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/164: dread f8 [0,4194304] 0 2024-04-22T23:45:37.229 INFO:tasks.workunit.client.0.smithi031.stdout:9/220: creat d2/dc/f43 x:0 0 0 2024-04-22T23:45:37.229 INFO:tasks.workunit.client.0.smithi031.stdout:9/221: stat d2/c37 0 2024-04-22T23:45:37.232 INFO:tasks.workunit.client.0.smithi031.stdout:7/165: dwrite f2 [4194304,4194304] 0 2024-04-22T23:45:37.238 INFO:tasks.workunit.client.0.smithi031.stdout:9/222: creat d2/dc/d21/d3d/d40/f44 x:0 0 0 2024-04-22T23:45:37.241 INFO:tasks.workunit.client.0.smithi031.stdout:8/147: getdents d5/da 0 2024-04-22T23:45:37.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/148: write d5/f26 [1738817,80430] 0 2024-04-22T23:45:37.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/149: stat d5/f26 0 2024-04-22T23:45:37.244 INFO:tasks.workunit.client.0.smithi031.stdout:9/223: rmdir d2/dc/d21 39 2024-04-22T23:45:37.250 INFO:tasks.workunit.client.0.smithi031.stdout:7/166: creat d9/f3b x:0 0 0 2024-04-22T23:45:37.251 INFO:tasks.workunit.client.0.smithi031.stdout:8/150: mknod d5/d1a/c2b 0 2024-04-22T23:45:37.254 INFO:tasks.workunit.client.0.smithi031.stdout:7/167: symlink d9/d14/d1f/l3c 0 2024-04-22T23:45:37.255 INFO:tasks.workunit.client.0.smithi031.stdout:8/151: unlink d5/fc 0 2024-04-22T23:45:37.256 INFO:tasks.workunit.client.0.smithi031.stdout:8/152: write d5/da/f1e [693126,22078] 0 2024-04-22T23:45:37.257 INFO:tasks.workunit.client.0.smithi031.stdout:9/224: rename d2/l3 to d2/dc/d21/d3d/d40/l45 0 2024-04-22T23:45:37.261 INFO:tasks.workunit.client.0.smithi031.stdout:8/153: creat d5/da/f2c x:0 0 0 2024-04-22T23:45:37.264 INFO:tasks.workunit.client.0.smithi031.stdout:7/168: mknod d9/d15/d3a/c3d 0 2024-04-22T23:45:37.265 INFO:tasks.workunit.client.0.smithi031.stdout:8/154: fdatasync d5/da/d12/f14 0 2024-04-22T23:45:37.270 INFO:tasks.workunit.client.0.smithi031.stdout:3/130: truncate d4/d10/f1b 782918 0 2024-04-22T23:45:37.271 INFO:tasks.workunit.client.0.smithi031.stdout:8/155: symlink d5/da/df/l2d 0 2024-04-22T23:45:37.278 INFO:tasks.workunit.client.0.smithi031.stdout:8/156: mkdir d5/da/d12/d2e 0 2024-04-22T23:45:37.279 INFO:tasks.workunit.client.0.smithi031.stdout:3/131: mkdir d4/d6/d11/d25 0 2024-04-22T23:45:37.282 INFO:tasks.workunit.client.0.smithi031.stdout:8/157: mkdir d5/da/d12/d2f 0 2024-04-22T23:45:37.283 INFO:tasks.workunit.client.0.smithi031.stdout:8/158: readlink d5/da/d12/l23 0 2024-04-22T23:45:37.284 INFO:tasks.workunit.client.0.smithi031.stdout:3/132: symlink d4/d6/l26 0 2024-04-22T23:45:37.286 INFO:tasks.workunit.client.0.smithi031.stdout:8/159: mkdir d5/da/d12/d2e/d30 0 2024-04-22T23:45:37.292 INFO:tasks.workunit.client.0.smithi031.stdout:8/160: mkdir d5/da/d12/d2f/d31 0 2024-04-22T23:45:37.293 INFO:tasks.workunit.client.0.smithi031.stdout:8/161: truncate d5/da/fd 720791 0 2024-04-22T23:45:37.294 INFO:tasks.workunit.client.0.smithi031.stdout:8/162: chown d5/da/df/f1b 12838 1 2024-04-22T23:45:37.295 INFO:tasks.workunit.client.0.smithi031.stdout:8/163: readlink d5/da/d12/l23 0 2024-04-22T23:45:37.295 INFO:tasks.workunit.client.0.smithi031.stdout:6/203: sync 2024-04-22T23:45:37.296 INFO:tasks.workunit.client.0.smithi031.stdout:0/139: sync 2024-04-22T23:45:37.297 INFO:tasks.workunit.client.0.smithi031.stdout:6/204: chown d3/c12 6853 1 2024-04-22T23:45:37.299 INFO:tasks.workunit.client.0.smithi031.stdout:4/256: write db/d10/f32 [1452120,98929] 0 2024-04-22T23:45:37.300 INFO:tasks.workunit.client.0.smithi031.stdout:8/164: dread d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:37.302 INFO:tasks.workunit.client.0.smithi031.stdout:5/168: truncate db/f23 1206348 0 2024-04-22T23:45:37.303 INFO:tasks.workunit.client.0.smithi031.stdout:0/140: dwrite d0/d1/d2/fa [4194304,4194304] 0 2024-04-22T23:45:37.307 INFO:tasks.workunit.client.0.smithi031.stdout:0/141: dread d0/d1/d2/fa [0,4194304] 0 2024-04-22T23:45:37.308 INFO:tasks.workunit.client.0.smithi031.stdout:0/142: chown d0/d1/d2/d3/l12 41 1 2024-04-22T23:45:37.309 INFO:tasks.workunit.client.0.smithi031.stdout:4/257: rename db/d10/d3d/f46 to db/d14/d2d/d49/f53 0 2024-04-22T23:45:37.309 INFO:tasks.workunit.client.0.smithi031.stdout:4/258: chown db/l1b 3687 1 2024-04-22T23:45:37.310 INFO:tasks.workunit.client.0.smithi031.stdout:4/259: fdatasync db/d14/f3a 0 2024-04-22T23:45:37.312 INFO:tasks.workunit.client.0.smithi031.stdout:5/169: chown db/fd 29 1 2024-04-22T23:45:37.313 INFO:tasks.workunit.client.0.smithi031.stdout:5/170: stat db/d16/d1d/d24/l26 0 2024-04-22T23:45:37.320 INFO:tasks.workunit.client.0.smithi031.stdout:4/260: rename db/d10/d3d/d41/f52 to db/d10/d2a/d47/f54 0 2024-04-22T23:45:37.320 INFO:tasks.workunit.client.0.smithi031.stdout:8/165: write d5/da/df/f13 [568450,77492] 0 2024-04-22T23:45:37.321 INFO:tasks.workunit.client.0.smithi031.stdout:4/261: truncate db/d10/d2a/d30/f42 38973 0 2024-04-22T23:45:37.322 INFO:tasks.workunit.client.0.smithi031.stdout:8/166: write d5/f8 [1451675,109348] 0 2024-04-22T23:45:37.323 INFO:tasks.workunit.client.0.smithi031.stdout:9/225: readlink d2/l4 0 2024-04-22T23:45:37.328 INFO:tasks.workunit.client.0.smithi031.stdout:7/169: truncate d9/d14/d22/d28/f36 774605 0 2024-04-22T23:45:37.328 INFO:tasks.workunit.client.0.smithi031.stdout:7/170: chown d9/d14/d1f 6457 1 2024-04-22T23:45:37.330 INFO:tasks.workunit.client.0.smithi031.stdout:5/171: mkdir db/d16/d1d/d35 0 2024-04-22T23:45:37.341 INFO:tasks.workunit.client.0.smithi031.stdout:3/133: truncate d4/f5 562747 0 2024-04-22T23:45:37.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/262: link db/d14/d2d/f37 db/d14/d2d/d3c/f55 0 2024-04-22T23:45:37.344 INFO:tasks.workunit.client.0.smithi031.stdout:5/172: rename db/d16/d1d/d35 to db/d16/d2a/d36 0 2024-04-22T23:45:37.346 INFO:tasks.workunit.client.0.smithi031.stdout:9/226: dread d2/f8 [0,4194304] 0 2024-04-22T23:45:37.347 INFO:tasks.workunit.client.0.smithi031.stdout:7/171: creat d9/f3e x:0 0 0 2024-04-22T23:45:37.350 INFO:tasks.workunit.client.0.smithi031.stdout:6/205: truncate d3/fe 1292099 0 2024-04-22T23:45:37.353 INFO:tasks.workunit.client.0.smithi031.stdout:6/206: dread d3/d1f/f20 [0,4194304] 0 2024-04-22T23:45:37.357 INFO:tasks.workunit.client.0.smithi031.stdout:3/134: mknod d4/d6/d11/c27 0 2024-04-22T23:45:37.358 INFO:tasks.workunit.client.0.smithi031.stdout:7/172: stat d9/cc 0 2024-04-22T23:45:37.359 INFO:tasks.workunit.client.0.smithi031.stdout:4/263: mknod db/d31/d4f/c56 0 2024-04-22T23:45:37.361 INFO:tasks.workunit.client.0.smithi031.stdout:0/143: truncate d0/d1/d2/d3/fd 138331 0 2024-04-22T23:45:37.364 INFO:tasks.workunit.client.0.smithi031.stdout:3/135: symlink d4/d6/d22/l28 0 2024-04-22T23:45:37.366 INFO:tasks.workunit.client.0.smithi031.stdout:7/173: mkdir d9/d15/d3f 0 2024-04-22T23:45:37.367 INFO:tasks.workunit.client.0.smithi031.stdout:7/174: dread - d9/d14/d22/f2d zero size 2024-04-22T23:45:37.368 INFO:tasks.workunit.client.0.smithi031.stdout:4/264: mknod db/d10/d2a/c57 0 2024-04-22T23:45:37.371 INFO:tasks.workunit.client.0.smithi031.stdout:8/167: dwrite d5/fb [4194304,4194304] 0 2024-04-22T23:45:37.373 INFO:tasks.workunit.client.0.smithi031.stdout:0/144: creat d0/d1/d2/d3/f2f x:0 0 0 2024-04-22T23:45:37.375 INFO:tasks.workunit.client.0.smithi031.stdout:9/227: rename d2/dd/d1b/l3a to d2/dc/d21/d38/l46 0 2024-04-22T23:45:37.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/265: symlink db/d14/d1d/l58 0 2024-04-22T23:45:37.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/266: fsync db/d31/f33 0 2024-04-22T23:45:37.385 INFO:tasks.workunit.client.0.smithi031.stdout:5/173: write f5 [465641,56533] 0 2024-04-22T23:45:37.386 INFO:tasks.workunit.client.0.smithi031.stdout:0/145: creat d0/d13/d18/d1e/f30 x:0 0 0 2024-04-22T23:45:37.392 INFO:tasks.workunit.client.0.smithi031.stdout:5/174: dwrite db/d16/d1d/d24/d27/f2d [0,4194304] 0 2024-04-22T23:45:37.393 INFO:tasks.workunit.client.0.smithi031.stdout:4/267: truncate db/d14/d1d/f20 265115 0 2024-04-22T23:45:37.394 INFO:tasks.workunit.client.0.smithi031.stdout:4/268: dread - db/d14/d2d/d49/f53 zero size 2024-04-22T23:45:37.397 INFO:tasks.workunit.client.0.smithi031.stdout:9/228: mkdir d2/d47 0 2024-04-22T23:45:37.399 INFO:tasks.workunit.client.0.smithi031.stdout:0/146: mknod d0/d1/d2/d3/d11/d1c/c31 0 2024-04-22T23:45:37.403 INFO:tasks.workunit.client.0.smithi031.stdout:7/175: rmdir d9/d38 0 2024-04-22T23:45:37.403 INFO:tasks.workunit.client.0.smithi031.stdout:0/147: dread d0/d1/d2/d3/d11/f14 [0,4194304] 0 2024-04-22T23:45:37.404 INFO:tasks.workunit.client.0.smithi031.stdout:0/148: read - d0/d1/d2/d3/f2f zero size 2024-04-22T23:45:37.406 INFO:tasks.workunit.client.0.smithi031.stdout:5/175: creat db/d16/d2a/f37 x:0 0 0 2024-04-22T23:45:37.407 INFO:tasks.workunit.client.0.smithi031.stdout:4/269: mkdir db/d14/d2d/d49/d59 0 2024-04-22T23:45:37.408 INFO:tasks.workunit.client.0.smithi031.stdout:4/270: fsync db/d10/d2a/d30/f42 0 2024-04-22T23:45:37.411 INFO:tasks.workunit.client.0.smithi031.stdout:7/176: dwrite d9/d14/f31 [0,4194304] 0 2024-04-22T23:45:37.412 INFO:tasks.workunit.client.0.smithi031.stdout:7/177: truncate d9/d14/f23 1037057 0 2024-04-22T23:45:37.413 INFO:tasks.workunit.client.0.smithi031.stdout:3/136: link d4/d6/f16 d4/d6/da/f29 0 2024-04-22T23:45:37.413 INFO:tasks.workunit.client.0.smithi031.stdout:7/178: fsync d9/f1c 0 2024-04-22T23:45:37.414 INFO:tasks.workunit.client.0.smithi031.stdout:3/137: readlink d4/d6/d22/l28 0 2024-04-22T23:45:37.414 INFO:tasks.workunit.client.0.smithi031.stdout:0/149: symlink d0/d13/d18/l32 0 2024-04-22T23:45:37.415 INFO:tasks.workunit.client.0.smithi031.stdout:3/138: dread - d4/d6/da/f29 zero size 2024-04-22T23:45:37.417 INFO:tasks.workunit.client.0.smithi031.stdout:2/155: sync 2024-04-22T23:45:37.417 INFO:tasks.workunit.client.0.smithi031.stdout:1/119: sync 2024-04-22T23:45:37.418 INFO:tasks.workunit.client.0.smithi031.stdout:5/176: creat db/d16/d2a/d36/f38 x:0 0 0 2024-04-22T23:45:37.428 INFO:tasks.workunit.client.0.smithi031.stdout:7/179: mknod d9/d15/d3a/c40 0 2024-04-22T23:45:37.433 INFO:tasks.workunit.client.0.smithi031.stdout:7/180: dread d9/d14/f31 [0,4194304] 0 2024-04-22T23:45:37.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/156: rename d6/dc/f15 to d6/d1e/f2c 0 2024-04-22T23:45:37.435 INFO:tasks.workunit.client.0.smithi031.stdout:4/271: symlink db/d14/d2d/d3c/l5a 0 2024-04-22T23:45:37.437 INFO:tasks.workunit.client.0.smithi031.stdout:3/139: fdatasync d4/d10/f1b 0 2024-04-22T23:45:37.441 INFO:tasks.workunit.client.0.smithi031.stdout:5/177: link db/d16/d1d/d24/f2e db/d16/d1d/d24/d32/f39 0 2024-04-22T23:45:37.445 INFO:tasks.workunit.client.0.smithi031.stdout:3/140: write d4/f1e [47800,19229] 0 2024-04-22T23:45:37.446 INFO:tasks.workunit.client.0.smithi031.stdout:0/150: link d0/d1/d2/d17/c1b d0/d1/d2/d3/d11/d1a/c33 0 2024-04-22T23:45:37.447 INFO:tasks.workunit.client.0.smithi031.stdout:0/151: chown d0/d13/d18/d1e/d25/c2a 1229 1 2024-04-22T23:45:37.448 INFO:tasks.workunit.client.0.smithi031.stdout:9/229: link d2/f2e d2/dc/f48 0 2024-04-22T23:45:37.450 INFO:tasks.workunit.client.0.smithi031.stdout:4/272: mkdir db/d14/d2d/d49/d59/d5b 0 2024-04-22T23:45:37.450 INFO:tasks.workunit.client.0.smithi031.stdout:4/273: fsync db/d14/f45 0 2024-04-22T23:45:37.451 INFO:tasks.workunit.client.0.smithi031.stdout:3/141: mkdir d4/d10/d2a 0 2024-04-22T23:45:37.452 INFO:tasks.workunit.client.0.smithi031.stdout:3/142: stat d4/d6/d11/c17 0 2024-04-22T23:45:37.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/152: rename d0/d13/l2c to d0/d1/d2/d3/d11/d1a/l34 0 2024-04-22T23:45:37.455 INFO:tasks.workunit.client.0.smithi031.stdout:2/157: link d6/l10 d6/dc/d17/l2d 0 2024-04-22T23:45:37.457 INFO:tasks.workunit.client.0.smithi031.stdout:2/158: truncate d6/d1e/d24/f2b 383425 0 2024-04-22T23:45:37.457 INFO:tasks.workunit.client.0.smithi031.stdout:3/143: stat d4/c18 0 2024-04-22T23:45:37.458 INFO:tasks.workunit.client.0.smithi031.stdout:9/230: creat d2/dd/d1b/f49 x:0 0 0 2024-04-22T23:45:37.458 INFO:tasks.workunit.client.0.smithi031.stdout:4/274: dwrite db/d14/f3a [0,4194304] 0 2024-04-22T23:45:37.459 INFO:tasks.workunit.client.0.smithi031.stdout:0/153: read d0/d1/d2/d3/d11/f14 [418559,51704] 0 2024-04-22T23:45:37.460 INFO:tasks.workunit.client.0.smithi031.stdout:0/154: dread - d0/d13/d18/d1e/f30 zero size 2024-04-22T23:45:37.461 INFO:tasks.workunit.client.0.smithi031.stdout:5/178: getdents db/d16/d1d 0 2024-04-22T23:45:37.462 INFO:tasks.workunit.client.0.smithi031.stdout:2/159: write d6/d1e/f2c [716805,49568] 0 2024-04-22T23:45:37.462 INFO:tasks.workunit.client.0.smithi031.stdout:3/144: creat d4/d10/f2b x:0 0 0 2024-04-22T23:45:37.463 INFO:tasks.workunit.client.0.smithi031.stdout:4/275: creat db/d31/f5c x:0 0 0 2024-04-22T23:45:37.464 INFO:tasks.workunit.client.0.smithi031.stdout:0/155: readlink d0/d13/d18/d1e/d25/l2e 0 2024-04-22T23:45:37.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/179: symlink db/l3a 0 2024-04-22T23:45:37.468 INFO:tasks.workunit.client.0.smithi031.stdout:9/231: dwrite d2/dd/d1b/f36 [0,4194304] 0 2024-04-22T23:45:37.472 INFO:tasks.workunit.client.0.smithi031.stdout:2/160: dwrite d6/f9 [4194304,4194304] 0 2024-04-22T23:45:37.474 INFO:tasks.workunit.client.0.smithi031.stdout:2/161: truncate d6/dc/d17/f25 133863 0 2024-04-22T23:45:37.475 INFO:tasks.workunit.client.0.smithi031.stdout:3/145: dwrite d4/d6/da/f29 [0,4194304] 0 2024-04-22T23:45:37.476 INFO:tasks.workunit.client.0.smithi031.stdout:3/146: readlink d4/d6/d11/l21 0 2024-04-22T23:45:37.476 INFO:tasks.workunit.client.0.smithi031.stdout:3/147: chown f3 28449 1 2024-04-22T23:45:37.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/148: write d4/d6/da/f13 [62370,105071] 0 2024-04-22T23:45:37.483 INFO:tasks.workunit.client.0.smithi031.stdout:3/149: dread d4/d6/d11/f19 [0,4194304] 0 2024-04-22T23:45:37.486 INFO:tasks.workunit.client.0.smithi031.stdout:3/150: mkdir d4/d10/d2a/d2c 0 2024-04-22T23:45:37.494 INFO:tasks.workunit.client.0.smithi031.stdout:3/151: dwrite d4/f1e [0,4194304] 0 2024-04-22T23:45:37.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/152: link d4/d10/f1b d4/d10/f2d 0 2024-04-22T23:45:37.499 INFO:tasks.workunit.client.0.smithi031.stdout:4/276: dread f9 [0,4194304] 0 2024-04-22T23:45:37.499 INFO:tasks.workunit.client.0.smithi031.stdout:3/153: mkdir d4/d10/d2e 0 2024-04-22T23:45:37.564 INFO:tasks.workunit.client.0.smithi031.stdout:6/207: sync 2024-04-22T23:45:37.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/208: stat d3/f38 0 2024-04-22T23:45:37.569 INFO:tasks.workunit.client.0.smithi031.stdout:6/209: creat d3/d2a/d3c/d40/f46 x:0 0 0 2024-04-22T23:45:37.570 INFO:tasks.workunit.client.0.smithi031.stdout:6/210: chown d3/d1f/l1e 5 1 2024-04-22T23:45:37.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/211: mknod d3/d2a/d3c/c47 0 2024-04-22T23:45:37.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/212: unlink d3/f14 0 2024-04-22T23:45:37.584 INFO:tasks.workunit.client.0.smithi031.stdout:7/181: truncate d9/d14/d22/d28/f36 1037474 0 2024-04-22T23:45:37.587 INFO:tasks.workunit.client.0.smithi031.stdout:7/182: mkdir d9/d15/d3a/d41 0 2024-04-22T23:45:37.588 INFO:tasks.workunit.client.0.smithi031.stdout:7/183: stat d9/d15/c32 0 2024-04-22T23:45:37.594 INFO:tasks.workunit.client.0.smithi031.stdout:7/184: dwrite d9/d15/f35 [0,4194304] 0 2024-04-22T23:45:37.599 INFO:tasks.workunit.client.0.smithi031.stdout:7/185: symlink d9/d15/d3a/l42 0 2024-04-22T23:45:37.602 INFO:tasks.workunit.client.0.smithi031.stdout:2/162: getdents d6/d1e 0 2024-04-22T23:45:37.605 INFO:tasks.workunit.client.0.smithi031.stdout:2/163: creat d6/d1e/f2e x:0 0 0 2024-04-22T23:45:37.606 INFO:tasks.workunit.client.0.smithi031.stdout:2/164: stat d6/dc/d17/c20 0 2024-04-22T23:45:37.611 INFO:tasks.workunit.client.0.smithi031.stdout:7/186: getdents d9 0 2024-04-22T23:45:37.614 INFO:tasks.workunit.client.0.smithi031.stdout:7/187: truncate f7 3168633 0 2024-04-22T23:45:37.618 INFO:tasks.workunit.client.0.smithi031.stdout:7/188: rename l1 to d9/d15/l43 0 2024-04-22T23:45:37.622 INFO:tasks.workunit.client.0.smithi031.stdout:7/189: mknod d9/d14/d22/d28/c44 0 2024-04-22T23:45:37.624 INFO:tasks.workunit.client.0.smithi031.stdout:2/165: getdents d6/dc/d17 0 2024-04-22T23:45:37.624 INFO:tasks.workunit.client.0.smithi031.stdout:7/190: mkdir d9/d14/d1f/d45 0 2024-04-22T23:45:37.628 INFO:tasks.workunit.client.0.smithi031.stdout:7/191: unlink d9/d15/c2e 0 2024-04-22T23:45:37.631 INFO:tasks.workunit.client.0.smithi031.stdout:7/192: mknod d9/d15/d3a/d41/c46 0 2024-04-22T23:45:37.632 INFO:tasks.workunit.client.0.smithi031.stdout:7/193: readlink d9/l12 0 2024-04-22T23:45:37.639 INFO:tasks.workunit.client.0.smithi031.stdout:7/194: dwrite d9/d15/f21 [0,4194304] 0 2024-04-22T23:45:37.642 INFO:tasks.workunit.client.0.smithi031.stdout:5/180: dwrite db/f25 [0,4194304] 0 2024-04-22T23:45:37.643 INFO:tasks.workunit.client.0.smithi031.stdout:5/181: truncate db/d16/d1d/f22 2510153 0 2024-04-22T23:45:37.648 INFO:tasks.workunit.client.0.smithi031.stdout:5/182: symlink db/d16/d1d/d24/d27/l3b 0 2024-04-22T23:45:37.659 INFO:tasks.workunit.client.0.smithi031.stdout:7/195: fsync f7 0 2024-04-22T23:45:37.673 INFO:tasks.workunit.client.0.smithi031.stdout:8/168: sync 2024-04-22T23:45:37.674 INFO:tasks.workunit.client.0.smithi031.stdout:8/169: chown d5/c16 213913 1 2024-04-22T23:45:37.677 INFO:tasks.workunit.client.0.smithi031.stdout:4/277: dwrite db/d14/d1d/f20 [0,4194304] 0 2024-04-22T23:45:37.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/196: dwrite d9/d14/d22/d28/f36 [0,4194304] 0 2024-04-22T23:45:37.679 INFO:tasks.workunit.client.0.smithi031.stdout:8/170: symlink d5/da/d12/d2f/l32 0 2024-04-22T23:45:37.680 INFO:tasks.workunit.client.0.smithi031.stdout:8/171: fdatasync d5/fb 0 2024-04-22T23:45:37.681 INFO:tasks.workunit.client.0.smithi031.stdout:7/197: mkdir d9/d14/d1f/d47 0 2024-04-22T23:45:37.682 INFO:tasks.workunit.client.0.smithi031.stdout:8/172: truncate d5/da/df/f13 889919 0 2024-04-22T23:45:37.683 INFO:tasks.workunit.client.0.smithi031.stdout:8/173: fsync d5/da/d12/f14 0 2024-04-22T23:45:37.683 INFO:tasks.workunit.client.0.smithi031.stdout:4/278: creat db/d10/d2a/d48/f5d x:0 0 0 2024-04-22T23:45:37.684 INFO:tasks.workunit.client.0.smithi031.stdout:7/198: mknod d9/d15/d3a/c48 0 2024-04-22T23:45:37.684 INFO:tasks.workunit.client.0.smithi031.stdout:8/174: write d5/da/df/f13 [497514,87660] 0 2024-04-22T23:45:37.685 INFO:tasks.workunit.client.0.smithi031.stdout:4/279: write db/d14/f45 [777909,19947] 0 2024-04-22T23:45:37.685 INFO:tasks.workunit.client.0.smithi031.stdout:8/175: stat d5/f19 0 2024-04-22T23:45:37.686 INFO:tasks.workunit.client.0.smithi031.stdout:4/280: chown db/fc 223951 1 2024-04-22T23:45:37.687 INFO:tasks.workunit.client.0.smithi031.stdout:8/176: chown d5/da/d12/l23 757097 1 2024-04-22T23:45:37.687 INFO:tasks.workunit.client.0.smithi031.stdout:4/281: write db/f24 [981507,99919] 0 2024-04-22T23:45:37.688 INFO:tasks.workunit.client.0.smithi031.stdout:4/282: write db/f24 [3562686,108135] 0 2024-04-22T23:45:37.690 INFO:tasks.workunit.client.0.smithi031.stdout:7/199: rename f7 to d9/d15/f49 0 2024-04-22T23:45:37.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/283: dwrite db/d14/d1d/f20 [0,4194304] 0 2024-04-22T23:45:37.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/284: chown db/d31 1018159885 1 2024-04-22T23:45:37.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/177: rename d5/d1a to d5/da/df/d33 0 2024-04-22T23:45:37.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/200: creat d9/d14/d22/d28/f4a x:0 0 0 2024-04-22T23:45:37.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/201: chown d9 17 1 2024-04-22T23:45:37.700 INFO:tasks.workunit.client.0.smithi031.stdout:7/202: creat d9/f4b x:0 0 0 2024-04-22T23:45:37.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/203: mknod d9/c4c 0 2024-04-22T23:45:37.705 INFO:tasks.workunit.client.0.smithi031.stdout:7/204: stat d9/l12 0 2024-04-22T23:45:37.705 INFO:tasks.workunit.client.0.smithi031.stdout:4/285: getdents db/d31/d40 0 2024-04-22T23:45:37.706 INFO:tasks.workunit.client.0.smithi031.stdout:8/178: getdents d5/da/d12/d2e 0 2024-04-22T23:45:37.707 INFO:tasks.workunit.client.0.smithi031.stdout:8/179: fdatasync d5/da/fd 0 2024-04-22T23:45:37.710 INFO:tasks.workunit.client.0.smithi031.stdout:8/180: truncate d5/da/df/f1b 88739 0 2024-04-22T23:45:37.712 INFO:tasks.workunit.client.0.smithi031.stdout:4/286: dread db/f24 [0,4194304] 0 2024-04-22T23:45:37.712 INFO:tasks.workunit.client.0.smithi031.stdout:7/205: dwrite d9/ff [0,4194304] 0 2024-04-22T23:45:37.714 INFO:tasks.workunit.client.0.smithi031.stdout:8/181: mkdir d5/da/d12/d2f/d31/d34 0 2024-04-22T23:45:37.714 INFO:tasks.workunit.client.0.smithi031.stdout:4/287: dread - db/d10/d2a/d48/f5d zero size 2024-04-22T23:45:37.716 INFO:tasks.workunit.client.0.smithi031.stdout:8/182: dread d5/da/df/f1b [0,4194304] 0 2024-04-22T23:45:37.716 INFO:tasks.workunit.client.0.smithi031.stdout:7/206: rename d9/l12 to d9/d14/d1f/l4d 0 2024-04-22T23:45:37.717 INFO:tasks.workunit.client.0.smithi031.stdout:7/207: fsync d9/d15/f21 0 2024-04-22T23:45:37.722 INFO:tasks.workunit.client.0.smithi031.stdout:8/183: dwrite d5/da/d12/f2a [4194304,4194304] 0 2024-04-22T23:45:37.723 INFO:tasks.workunit.client.0.smithi031.stdout:4/288: link db/d10/f2f db/d10/d2a/d30/f5e 0 2024-04-22T23:45:37.726 INFO:tasks.workunit.client.0.smithi031.stdout:7/208: dread f3 [0,4194304] 0 2024-04-22T23:45:37.727 INFO:tasks.workunit.client.0.smithi031.stdout:8/184: rename d5/da/d12/l23 to d5/da/d12/d2f/d31/d34/l35 0 2024-04-22T23:45:37.729 INFO:tasks.workunit.client.0.smithi031.stdout:7/209: symlink d9/l4e 0 2024-04-22T23:45:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:4/289: creat db/f5f x:0 0 0 2024-04-22T23:45:37.731 INFO:tasks.workunit.client.0.smithi031.stdout:7/210: mkdir d9/d4f 0 2024-04-22T23:45:37.735 INFO:tasks.workunit.client.0.smithi031.stdout:7/211: rename d9/d15/c19 to d9/d4f/c50 0 2024-04-22T23:45:37.739 INFO:tasks.workunit.client.0.smithi031.stdout:8/185: fsync d5/da/d12/f2a 0 2024-04-22T23:45:37.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/212: dwrite d9/fa [0,4194304] 0 2024-04-22T23:45:37.742 INFO:tasks.workunit.client.0.smithi031.stdout:8/186: symlink d5/da/l36 0 2024-04-22T23:45:37.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/213: fdatasync d9/d14/d22/f2d 0 2024-04-22T23:45:37.743 INFO:tasks.workunit.client.0.smithi031.stdout:8/187: fdatasync d5/fb 0 2024-04-22T23:45:37.743 INFO:tasks.workunit.client.0.smithi031.stdout:8/188: readlink d5/le 0 2024-04-22T23:45:37.745 INFO:tasks.workunit.client.0.smithi031.stdout:7/214: chown d9/d14/c2c 0 1 2024-04-22T23:45:37.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/215: read d9/f1c [2298082,48226] 0 2024-04-22T23:45:37.751 INFO:tasks.workunit.client.0.smithi031.stdout:7/216: creat d9/d15/d3a/f51 x:0 0 0 2024-04-22T23:45:37.752 INFO:tasks.workunit.client.0.smithi031.stdout:7/217: chown d9/d14/d22 7301142 1 2024-04-22T23:45:37.755 INFO:tasks.workunit.client.0.smithi031.stdout:7/218: fdatasync f8 0 2024-04-22T23:45:37.756 INFO:tasks.workunit.client.0.smithi031.stdout:7/219: chown d9/d15/d3a/c3d 72760 1 2024-04-22T23:45:37.759 INFO:tasks.workunit.client.0.smithi031.stdout:7/220: dread d9/fa [0,4194304] 0 2024-04-22T23:45:37.763 INFO:tasks.workunit.client.0.smithi031.stdout:8/189: rename d5/da/d12/d2f/d31 to d5/da/d12/d37 0 2024-04-22T23:45:37.765 INFO:tasks.workunit.client.0.smithi031.stdout:7/221: creat d9/d14/d22/f52 x:0 0 0 2024-04-22T23:45:37.769 INFO:tasks.workunit.client.0.smithi031.stdout:8/190: mkdir d5/da/d12/d1d/d38 0 2024-04-22T23:45:37.770 INFO:tasks.workunit.client.0.smithi031.stdout:8/191: write d5/da/fd [622392,69835] 0 2024-04-22T23:45:37.771 INFO:tasks.workunit.client.0.smithi031.stdout:7/222: mkdir d9/d53 0 2024-04-22T23:45:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/290: dwrite f4 [0,4194304] 0 2024-04-22T23:45:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:8/192: creat d5/da/df/d33/f39 x:0 0 0 2024-04-22T23:45:37.776 INFO:tasks.workunit.client.0.smithi031.stdout:7/223: creat d9/d15/d3a/d41/f54 x:0 0 0 2024-04-22T23:45:37.777 INFO:tasks.workunit.client.0.smithi031.stdout:7/224: write d9/f4b [806567,21971] 0 2024-04-22T23:45:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:4/291: rename f1 to db/d10/d2a/d30/f60 0 2024-04-22T23:45:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:4/292: stat db/d14/d2d/d3c/l5a 0 2024-04-22T23:45:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:7/225: mknod d9/c55 0 2024-04-22T23:45:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:8/193: dwrite d5/f8 [0,4194304] 0 2024-04-22T23:45:37.784 INFO:tasks.workunit.client.0.smithi031.stdout:8/194: truncate d5/da/df/d33/f39 277734 0 2024-04-22T23:45:37.785 INFO:tasks.workunit.client.0.smithi031.stdout:7/226: creat d9/d15/f56 x:0 0 0 2024-04-22T23:45:37.790 INFO:tasks.workunit.client.0.smithi031.stdout:7/227: dread d9/d15/f49 [0,4194304] 0 2024-04-22T23:45:37.791 INFO:tasks.workunit.client.0.smithi031.stdout:8/195: symlink d5/da/d12/d1d/l3a 0 2024-04-22T23:45:37.792 INFO:tasks.workunit.client.0.smithi031.stdout:4/293: link db/c26 db/d10/d3d/c61 0 2024-04-22T23:45:37.795 INFO:tasks.workunit.client.0.smithi031.stdout:8/196: mkdir d5/da/d12/d37/d3b 0 2024-04-22T23:45:37.795 INFO:tasks.workunit.client.0.smithi031.stdout:8/197: chown d5/da/d12/d37/d34 11304247 1 2024-04-22T23:45:37.796 INFO:tasks.workunit.client.0.smithi031.stdout:4/294: mkdir db/d31/d40/d62 0 2024-04-22T23:45:37.797 INFO:tasks.workunit.client.0.smithi031.stdout:7/228: symlink d9/d53/l57 0 2024-04-22T23:45:37.798 INFO:tasks.workunit.client.0.smithi031.stdout:8/198: dread d5/da/df/f1b [0,4194304] 0 2024-04-22T23:45:37.800 INFO:tasks.workunit.client.0.smithi031.stdout:4/295: mkdir db/d10/d3d/d63 0 2024-04-22T23:45:37.800 INFO:tasks.workunit.client.0.smithi031.stdout:8/199: fsync d5/f26 0 2024-04-22T23:45:37.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/232: sync 2024-04-22T23:45:37.801 INFO:tasks.workunit.client.0.smithi031.stdout:1/120: sync 2024-04-22T23:45:37.801 INFO:tasks.workunit.client.0.smithi031.stdout:0/156: sync 2024-04-22T23:45:37.801 INFO:tasks.workunit.client.0.smithi031.stdout:3/154: sync 2024-04-22T23:45:37.802 INFO:tasks.workunit.client.0.smithi031.stdout:6/213: sync 2024-04-22T23:45:37.802 INFO:tasks.workunit.client.0.smithi031.stdout:5/183: sync 2024-04-22T23:45:37.802 INFO:tasks.workunit.client.0.smithi031.stdout:2/166: sync 2024-04-22T23:45:37.803 INFO:tasks.workunit.client.0.smithi031.stdout:4/296: write db/d31/f5c [984735,81417] 0 2024-04-22T23:45:37.803 INFO:tasks.workunit.client.0.smithi031.stdout:8/200: stat d5/da/d12/c28 0 2024-04-22T23:45:37.804 INFO:tasks.workunit.client.0.smithi031.stdout:6/214: chown d3/f18 182073554 1 2024-04-22T23:45:37.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/155: chown d4/l14 9222767 1 2024-04-22T23:45:37.806 INFO:tasks.workunit.client.0.smithi031.stdout:7/229: dwrite d9/d15/d3a/f51 [0,4194304] 0 2024-04-22T23:45:37.807 INFO:tasks.workunit.client.0.smithi031.stdout:6/215: write d3/f39 [4420875,25826] 0 2024-04-22T23:45:37.807 INFO:tasks.workunit.client.0.smithi031.stdout:3/156: chown d4 231 1 2024-04-22T23:45:37.807 INFO:tasks.workunit.client.0.smithi031.stdout:9/233: fsync d2/dc/f10 0 2024-04-22T23:45:37.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/230: chown d9/l1e 2 1 2024-04-22T23:45:37.810 INFO:tasks.workunit.client.0.smithi031.stdout:1/121: mkdir d1/db/d16/d2f 0 2024-04-22T23:45:37.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/297: stat db/d14/c1a 0 2024-04-22T23:45:37.813 INFO:tasks.workunit.client.0.smithi031.stdout:4/298: write db/d14/f3a [4812728,58966] 0 2024-04-22T23:45:37.813 INFO:tasks.workunit.client.0.smithi031.stdout:2/167: mknod d6/c2f 0 2024-04-22T23:45:37.814 INFO:tasks.workunit.client.0.smithi031.stdout:2/168: truncate d6/d1e/d24/f2b 958794 0 2024-04-22T23:45:37.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/201: unlink d5/da/d12/d1d/l3a 0 2024-04-22T23:45:37.821 INFO:tasks.workunit.client.0.smithi031.stdout:6/216: write d3/f38 [2055777,44149] 0 2024-04-22T23:45:37.821 INFO:tasks.workunit.client.0.smithi031.stdout:6/217: chown d3/f34 207299630 1 2024-04-22T23:45:37.822 INFO:tasks.workunit.client.0.smithi031.stdout:6/218: dread - d3/d2a/d3c/f43 zero size 2024-04-22T23:45:37.823 INFO:tasks.workunit.client.0.smithi031.stdout:6/219: dread - d3/f45 zero size 2024-04-22T23:45:37.828 INFO:tasks.workunit.client.0.smithi031.stdout:9/234: rename d2/dd/d1b to d2/d47/d4a 0 2024-04-22T23:45:37.835 INFO:tasks.workunit.client.0.smithi031.stdout:4/299: creat db/d10/d3d/f64 x:0 0 0 2024-04-22T23:45:37.837 INFO:tasks.workunit.client.0.smithi031.stdout:6/220: rmdir d3/d1f 39 2024-04-22T23:45:37.845 INFO:tasks.workunit.client.0.smithi031.stdout:9/235: stat d2/d15/l22 0 2024-04-22T23:45:37.847 INFO:tasks.workunit.client.0.smithi031.stdout:8/202: rename d5/da/fd to d5/da/d12/d37/f3c 0 2024-04-22T23:45:37.849 INFO:tasks.workunit.client.0.smithi031.stdout:1/122: mkdir d1/db/d16/d2f/d30 0 2024-04-22T23:45:37.857 INFO:tasks.workunit.client.0.smithi031.stdout:0/157: link d0/d13/d18/d1e/d25/l2e d0/d1/d2/d3/d11/d1c/l35 0 2024-04-22T23:45:37.861 INFO:tasks.workunit.client.0.smithi031.stdout:9/236: readlink d2/dc/d21/d38/l46 0 2024-04-22T23:45:37.864 INFO:tasks.workunit.client.0.smithi031.stdout:8/203: unlink c2 0 2024-04-22T23:45:37.866 INFO:tasks.workunit.client.0.smithi031.stdout:1/123: mknod d1/db/d16/c31 0 2024-04-22T23:45:37.867 INFO:tasks.workunit.client.0.smithi031.stdout:1/124: chown d1/la 415603 1 2024-04-22T23:45:37.870 INFO:tasks.workunit.client.0.smithi031.stdout:2/169: link d6/c2f d6/dc/d17/c30 0 2024-04-22T23:45:37.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/221: mknod d3/d1f/c48 0 2024-04-22T23:45:37.875 INFO:tasks.workunit.client.0.smithi031.stdout:3/157: dwrite d4/d10/f1b [0,4194304] 0 2024-04-22T23:45:37.876 INFO:tasks.workunit.client.0.smithi031.stdout:3/158: stat d4/l14 0 2024-04-22T23:45:37.876 INFO:tasks.workunit.client.0.smithi031.stdout:7/231: link d9/d15/c32 d9/d15/c58 0 2024-04-22T23:45:37.877 INFO:tasks.workunit.client.0.smithi031.stdout:3/159: stat d4/d6/d11 0 2024-04-22T23:45:37.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/160: read - d4/d6/d11/f1c zero size 2024-04-22T23:45:37.882 INFO:tasks.workunit.client.0.smithi031.stdout:6/222: dwrite d3/f45 [0,4194304] 0 2024-04-22T23:45:37.883 INFO:tasks.workunit.client.0.smithi031.stdout:5/184: getdents db/d16/d2a/d36 0 2024-04-22T23:45:37.884 INFO:tasks.workunit.client.0.smithi031.stdout:5/185: dread - db/f29 zero size 2024-04-22T23:45:37.886 INFO:tasks.workunit.client.0.smithi031.stdout:5/186: read db/d16/f18 [2088356,64636] 0 2024-04-22T23:45:37.886 INFO:tasks.workunit.client.0.smithi031.stdout:0/158: rename d0/d1/d2/d3 to d0/d1/d2/d36 0 2024-04-22T23:45:37.887 INFO:tasks.workunit.client.0.smithi031.stdout:5/187: fdatasync db/d16/d2a/d36/f38 0 2024-04-22T23:45:37.889 INFO:tasks.workunit.client.0.smithi031.stdout:2/170: mknod d6/d1e/d24/c31 0 2024-04-22T23:45:37.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/232: mknod d9/d15/d3a/c59 0 2024-04-22T23:45:37.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/233: fsync d9/d15/f21 0 2024-04-22T23:45:37.901 INFO:tasks.workunit.client.0.smithi031.stdout:8/204: truncate d5/da/f20 868253 0 2024-04-22T23:45:37.903 INFO:tasks.workunit.client.0.smithi031.stdout:1/125: mknod d1/db/d16/d2f/d30/c32 0 2024-04-22T23:45:37.903 INFO:tasks.workunit.client.0.smithi031.stdout:4/300: dwrite db/d14/d2d/d3c/f55 [0,4194304] 0 2024-04-22T23:45:37.904 INFO:tasks.workunit.client.0.smithi031.stdout:8/205: dread d5/da/df/d33/f39 [0,4194304] 0 2024-04-22T23:45:37.906 INFO:tasks.workunit.client.0.smithi031.stdout:1/126: write d1/d21/d29/f1b [387562,35869] 0 2024-04-22T23:45:37.908 INFO:tasks.workunit.client.0.smithi031.stdout:1/127: dread - d1/d21/d29/d28/f2a zero size 2024-04-22T23:45:37.908 INFO:tasks.workunit.client.0.smithi031.stdout:6/223: dread d3/f15 [0,4194304] 0 2024-04-22T23:45:37.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/159: creat d0/d1/d2/f37 x:0 0 0 2024-04-22T23:45:37.912 INFO:tasks.workunit.client.0.smithi031.stdout:5/188: rmdir db 39 2024-04-22T23:45:37.915 INFO:tasks.workunit.client.0.smithi031.stdout:8/206: dwrite d5/f8 [0,4194304] 0 2024-04-22T23:45:37.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/171: mkdir d6/d1e/d24/d32 0 2024-04-22T23:45:37.919 INFO:tasks.workunit.client.0.smithi031.stdout:9/237: rename d2/d47/d4a/f33 to d2/dc/f4b 0 2024-04-22T23:45:37.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/128: dwrite f0 [0,4194304] 0 2024-04-22T23:45:37.921 INFO:tasks.workunit.client.0.smithi031.stdout:8/207: truncate d5/da/d12/f14 1464918 0 2024-04-22T23:45:37.923 INFO:tasks.workunit.client.0.smithi031.stdout:6/224: dwrite d3/d2a/d3c/d40/f44 [0,4194304] 0 2024-04-22T23:45:37.923 INFO:tasks.workunit.client.0.smithi031.stdout:1/129: dread - d1/d21/d29/d28/f2a zero size 2024-04-22T23:45:37.929 INFO:tasks.workunit.client.0.smithi031.stdout:9/238: read d2/dc/d3e/f41 [83920,75930] 0 2024-04-22T23:45:37.944 INFO:tasks.workunit.client.0.smithi031.stdout:0/160: rmdir d0 39 2024-04-22T23:45:37.955 INFO:tasks.workunit.client.0.smithi031.stdout:2/172: rename d6/l21 to d6/d1e/d24/d32/l33 0 2024-04-22T23:45:37.957 INFO:tasks.workunit.client.0.smithi031.stdout:5/189: dread db/d16/d1d/f22 [0,4194304] 0 2024-04-22T23:45:37.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/225: mknod d3/d2a/d3c/d40/c49 0 2024-04-22T23:45:37.963 INFO:tasks.workunit.client.0.smithi031.stdout:3/161: truncate d4/d10/f2d 3881595 0 2024-04-22T23:45:37.965 INFO:tasks.workunit.client.0.smithi031.stdout:4/301: mknod db/d10/d3d/d63/c65 0 2024-04-22T23:45:37.966 INFO:tasks.workunit.client.0.smithi031.stdout:4/302: chown db/d14/d2d/d49/l4e 2 1 2024-04-22T23:45:37.974 INFO:tasks.workunit.client.0.smithi031.stdout:2/173: creat d6/d1e/d24/f34 x:0 0 0 2024-04-22T23:45:37.981 INFO:tasks.workunit.client.0.smithi031.stdout:5/190: symlink db/d16/d2a/d36/l3c 0 2024-04-22T23:45:37.986 INFO:tasks.workunit.client.0.smithi031.stdout:6/226: creat d3/d2a/d3c/d40/f4a x:0 0 0 2024-04-22T23:45:37.989 INFO:tasks.workunit.client.0.smithi031.stdout:6/227: truncate d3/d2a/f30 869439 0 2024-04-22T23:45:37.989 INFO:tasks.workunit.client.0.smithi031.stdout:5/191: dwrite db/d16/d2a/d36/f38 [0,4194304] 0 2024-04-22T23:45:37.996 INFO:tasks.workunit.client.0.smithi031.stdout:4/303: symlink db/d31/l66 0 2024-04-22T23:45:37.998 INFO:tasks.workunit.client.0.smithi031.stdout:0/161: creat d0/d13/d18/d1e/d25/f38 x:0 0 0 2024-04-22T23:45:38.002 INFO:tasks.workunit.client.0.smithi031.stdout:2/174: rmdir d6 39 2024-04-22T23:45:38.004 INFO:tasks.workunit.client.0.smithi031.stdout:7/234: getdents d9 0 2024-04-22T23:45:38.004 INFO:tasks.workunit.client.0.smithi031.stdout:8/208: link d5/c16 d5/da/d12/d2e/c3d 0 2024-04-22T23:45:38.007 INFO:tasks.workunit.client.0.smithi031.stdout:4/304: dwrite db/d14/f3a [0,4194304] 0 2024-04-22T23:45:38.010 INFO:tasks.workunit.client.0.smithi031.stdout:4/305: truncate f4 4215995 0 2024-04-22T23:45:38.012 INFO:tasks.workunit.client.0.smithi031.stdout:6/228: mknod d3/d1f/c4b 0 2024-04-22T23:45:38.013 INFO:tasks.workunit.client.0.smithi031.stdout:2/175: dwrite f4 [0,4194304] 0 2024-04-22T23:45:38.014 INFO:tasks.workunit.client.0.smithi031.stdout:3/162: symlink d4/d10/d2e/l2f 0 2024-04-22T23:45:38.015 INFO:tasks.workunit.client.0.smithi031.stdout:7/235: dread d9/d15/f35 [0,4194304] 0 2024-04-22T23:45:38.017 INFO:tasks.workunit.client.0.smithi031.stdout:4/306: dread db/f24 [0,4194304] 0 2024-04-22T23:45:38.019 INFO:tasks.workunit.client.0.smithi031.stdout:4/307: write db/d14/d2d/f37 [1079718,15970] 0 2024-04-22T23:45:38.023 INFO:tasks.workunit.client.0.smithi031.stdout:3/163: dwrite f3 [0,4194304] 0 2024-04-22T23:45:38.026 INFO:tasks.workunit.client.0.smithi031.stdout:1/130: getdents d1/db/d16 0 2024-04-22T23:45:38.029 INFO:tasks.workunit.client.0.smithi031.stdout:8/209: rmdir d5/da/d12 39 2024-04-22T23:45:38.030 INFO:tasks.workunit.client.0.smithi031.stdout:4/308: dwrite db/d14/f3a [4194304,4194304] 0 2024-04-22T23:45:38.030 INFO:tasks.workunit.client.0.smithi031.stdout:8/210: chown d5/le 164948160 1 2024-04-22T23:45:38.034 INFO:tasks.workunit.client.0.smithi031.stdout:9/239: getdents d2/d47 0 2024-04-22T23:45:38.035 INFO:tasks.workunit.client.0.smithi031.stdout:5/192: mknod db/d16/c3d 0 2024-04-22T23:45:38.037 INFO:tasks.workunit.client.0.smithi031.stdout:6/229: mkdir d3/d1f/d4c 0 2024-04-22T23:45:38.039 INFO:tasks.workunit.client.0.smithi031.stdout:6/230: write d3/d2a/d3c/d40/f44 [87925,23655] 0 2024-04-22T23:45:38.040 INFO:tasks.workunit.client.0.smithi031.stdout:6/231: write d3/d1f/f28 [5855938,114929] 0 2024-04-22T23:45:38.042 INFO:tasks.workunit.client.0.smithi031.stdout:7/236: rename d9/d14/f31 to d9/d15/d3a/d41/f5a 0 2024-04-22T23:45:38.046 INFO:tasks.workunit.client.0.smithi031.stdout:7/237: dread d9/d15/d3a/f51 [0,4194304] 0 2024-04-22T23:45:38.047 INFO:tasks.workunit.client.0.smithi031.stdout:7/238: chown d9/l1e 155566 1 2024-04-22T23:45:38.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/164: rmdir d4/d6/d11 39 2024-04-22T23:45:38.050 INFO:tasks.workunit.client.0.smithi031.stdout:1/131: symlink d1/db/d16/d1f/l33 0 2024-04-22T23:45:38.051 INFO:tasks.workunit.client.0.smithi031.stdout:7/239: dread f2 [4194304,4194304] 0 2024-04-22T23:45:38.052 INFO:tasks.workunit.client.0.smithi031.stdout:1/132: write d1/de/f11 [613546,43528] 0 2024-04-22T23:45:38.056 INFO:tasks.workunit.client.0.smithi031.stdout:4/309: mknod db/d14/d2d/d3c/c67 0 2024-04-22T23:45:38.059 INFO:tasks.workunit.client.0.smithi031.stdout:2/176: unlink c0 0 2024-04-22T23:45:38.059 INFO:tasks.workunit.client.0.smithi031.stdout:4/310: dread db/d14/d1d/f20 [0,4194304] 0 2024-04-22T23:45:38.064 INFO:tasks.workunit.client.0.smithi031.stdout:6/232: mkdir d3/d2a/d3c/d40/d4d 0 2024-04-22T23:45:38.078 INFO:tasks.workunit.client.0.smithi031.stdout:9/240: truncate d2/f2e 14065 0 2024-04-22T23:45:38.082 INFO:tasks.workunit.client.0.smithi031.stdout:9/241: stat d2/dc/ff 0 2024-04-22T23:45:38.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/311: creat db/d31/d40/f68 x:0 0 0 2024-04-22T23:45:38.087 INFO:tasks.workunit.client.0.smithi031.stdout:0/162: getdents d0/d1/d2/d36/d11/d1c 0 2024-04-22T23:45:38.096 INFO:tasks.workunit.client.0.smithi031.stdout:2/177: mkdir d6/d1e/d24/d32/d35 0 2024-04-22T23:45:38.098 INFO:tasks.workunit.client.0.smithi031.stdout:4/312: truncate db/fc 374723 0 2024-04-22T23:45:38.102 INFO:tasks.workunit.client.0.smithi031.stdout:6/233: link d3/f45 d3/d2a/d3c/d40/f4e 0 2024-04-22T23:45:38.103 INFO:tasks.workunit.client.0.smithi031.stdout:6/234: write d3/d2a/d3c/d40/f44 [3407625,52048] 0 2024-04-22T23:45:38.105 INFO:tasks.workunit.client.0.smithi031.stdout:5/193: rename db/d16/d1d/d24/f2f to db/d16/f3e 0 2024-04-22T23:45:38.117 INFO:tasks.workunit.client.0.smithi031.stdout:4/313: mkdir db/d14/d2d/d3c/d69 0 2024-04-22T23:45:38.122 INFO:tasks.workunit.client.0.smithi031.stdout:6/235: creat d3/d1f/f4f x:0 0 0 2024-04-22T23:45:38.124 INFO:tasks.workunit.client.0.smithi031.stdout:5/194: creat db/f3f x:0 0 0 2024-04-22T23:45:38.125 INFO:tasks.workunit.client.0.smithi031.stdout:5/195: write db/fd [1676293,21404] 0 2024-04-22T23:45:38.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/165: link d4/d6/d22/l24 d4/d6/l30 0 2024-04-22T23:45:38.130 INFO:tasks.workunit.client.0.smithi031.stdout:8/211: getdents d5/da/d12/d37 0 2024-04-22T23:45:38.135 INFO:tasks.workunit.client.0.smithi031.stdout:4/314: rename db/d10/d3d to db/d14/d2d/d49/d59/d6a 0 2024-04-22T23:45:38.136 INFO:tasks.workunit.client.0.smithi031.stdout:8/212: dwrite d5/f19 [0,4194304] 0 2024-04-22T23:45:38.137 INFO:tasks.workunit.client.0.smithi031.stdout:8/213: chown d5/da/l36 71619 1 2024-04-22T23:45:38.141 INFO:tasks.workunit.client.0.smithi031.stdout:0/163: dwrite d0/d1/d2/d36/d11/f14 [0,4194304] 0 2024-04-22T23:45:38.144 INFO:tasks.workunit.client.0.smithi031.stdout:0/164: truncate d0/d1/d2/f37 440782 0 2024-04-22T23:45:38.147 INFO:tasks.workunit.client.0.smithi031.stdout:5/196: symlink db/l40 0 2024-04-22T23:45:38.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/236: dwrite d3/f15 [0,4194304] 0 2024-04-22T23:45:38.149 INFO:tasks.workunit.client.0.smithi031.stdout:6/237: readlink d3/l1b 0 2024-04-22T23:45:38.151 INFO:tasks.workunit.client.0.smithi031.stdout:0/165: truncate d0/d1/d2/d36/f2f 860969 0 2024-04-22T23:45:38.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/242: dread d2/dc/f48 [0,4194304] 0 2024-04-22T23:45:38.157 INFO:tasks.workunit.client.0.smithi031.stdout:2/178: mknod d6/d1e/d24/c36 0 2024-04-22T23:45:38.163 INFO:tasks.workunit.client.0.smithi031.stdout:5/197: fdatasync db/d16/d1d/d24/f2e 0 2024-04-22T23:45:38.164 INFO:tasks.workunit.client.0.smithi031.stdout:2/179: dread d6/f11 [0,4194304] 0 2024-04-22T23:45:38.164 INFO:tasks.workunit.client.0.smithi031.stdout:6/238: mknod d3/d2a/c50 0 2024-04-22T23:45:38.165 INFO:tasks.workunit.client.0.smithi031.stdout:6/239: fsync d3/d2a/d3c/d40/f44 0 2024-04-22T23:45:38.165 INFO:tasks.workunit.client.0.smithi031.stdout:6/240: stat d3/d2a/c50 0 2024-04-22T23:45:38.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/166: creat d0/d13/d18/f39 x:0 0 0 2024-04-22T23:45:38.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/214: mknod d5/da/d12/d1d/d38/c3e 0 2024-04-22T23:45:38.180 INFO:tasks.workunit.client.0.smithi031.stdout:0/167: read d0/d1/f10 [981794,68063] 0 2024-04-22T23:45:38.181 INFO:tasks.workunit.client.0.smithi031.stdout:5/198: mkdir db/d16/d1d/d24/d32/d41 0 2024-04-22T23:45:38.181 INFO:tasks.workunit.client.0.smithi031.stdout:0/168: chown d0/d13/d18/d1e/d25/c2a 1614 1 2024-04-22T23:45:38.181 INFO:tasks.workunit.client.0.smithi031.stdout:9/243: dread d2/f20 [0,4194304] 0 2024-04-22T23:45:38.182 INFO:tasks.workunit.client.0.smithi031.stdout:5/199: truncate db/f3f 1033480 0 2024-04-22T23:45:38.182 INFO:tasks.workunit.client.0.smithi031.stdout:6/241: symlink d3/d1f/d4c/l51 0 2024-04-22T23:45:38.182 INFO:tasks.workunit.client.0.smithi031.stdout:5/200: stat db/f25 0 2024-04-22T23:45:38.183 INFO:tasks.workunit.client.0.smithi031.stdout:0/169: truncate d0/d1/d2/f37 770867 0 2024-04-22T23:45:38.183 INFO:tasks.workunit.client.0.smithi031.stdout:5/201: chown db/d16/d2a/d36 1282612999 1 2024-04-22T23:45:38.185 INFO:tasks.workunit.client.0.smithi031.stdout:5/202: chown db/lf 6420650 1 2024-04-22T23:45:38.187 INFO:tasks.workunit.client.0.smithi031.stdout:8/215: creat d5/da/d12/d1d/d38/f3f x:0 0 0 2024-04-22T23:45:38.188 INFO:tasks.workunit.client.0.smithi031.stdout:9/244: dwrite d2/d47/d4a/f49 [0,4194304] 0 2024-04-22T23:45:38.190 INFO:tasks.workunit.client.0.smithi031.stdout:6/242: creat d3/d2a/d3c/f52 x:0 0 0 2024-04-22T23:45:38.192 INFO:tasks.workunit.client.0.smithi031.stdout:2/180: creat d6/d1e/d28/f37 x:0 0 0 2024-04-22T23:45:38.198 INFO:tasks.workunit.client.0.smithi031.stdout:5/203: rename db/d16/f21 to db/f42 0 2024-04-22T23:45:38.198 INFO:tasks.workunit.client.0.smithi031.stdout:4/315: dwrite db/d10/d2a/d30/f5e [0,4194304] 0 2024-04-22T23:45:38.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/170: mknod d0/d1/c3a 0 2024-04-22T23:45:38.202 INFO:tasks.workunit.client.0.smithi031.stdout:3/166: write d4/fe [3282041,49085] 0 2024-04-22T23:45:38.203 INFO:tasks.workunit.client.0.smithi031.stdout:0/171: readlink d0/d1/d2/d36/d11/d1c/l20 0 2024-04-22T23:45:38.205 INFO:tasks.workunit.client.0.smithi031.stdout:4/316: dread db/d31/f5c [0,4194304] 0 2024-04-22T23:45:38.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/245: unlink d2/dc/d21/d38/l46 0 2024-04-22T23:45:38.207 INFO:tasks.workunit.client.0.smithi031.stdout:6/243: creat d3/d2a/f53 x:0 0 0 2024-04-22T23:45:38.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/204: symlink db/d16/d1d/d24/l43 0 2024-04-22T23:45:38.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/205: stat db/d16/c13 0 2024-04-22T23:45:38.217 INFO:tasks.workunit.client.0.smithi031.stdout:8/216: truncate d5/f26 998558 0 2024-04-22T23:45:38.218 INFO:tasks.workunit.client.0.smithi031.stdout:8/217: chown d5/da/d12/d2e/d30 29 1 2024-04-22T23:45:38.219 INFO:tasks.workunit.client.0.smithi031.stdout:0/172: mknod d0/d1/d2/d36/d11/c3b 0 2024-04-22T23:45:38.223 INFO:tasks.workunit.client.0.smithi031.stdout:4/317: rename db/d10/c15 to db/d14/d1d/c6b 0 2024-04-22T23:45:38.224 INFO:tasks.workunit.client.0.smithi031.stdout:4/318: write db/d14/f45 [527693,117434] 0 2024-04-22T23:45:38.225 INFO:tasks.workunit.client.0.smithi031.stdout:4/319: stat db/d31/f33 0 2024-04-22T23:45:38.228 INFO:tasks.workunit.client.0.smithi031.stdout:9/246: dread d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:38.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/206: mknod db/d16/d1d/c44 0 2024-04-22T23:45:38.230 INFO:tasks.workunit.client.0.smithi031.stdout:3/167: mknod d4/d6/c31 0 2024-04-22T23:45:38.230 INFO:tasks.workunit.client.0.smithi031.stdout:4/320: truncate db/d14/d2d/d49/d59/d6a/d41/f4b 463091 0 2024-04-22T23:45:38.230 INFO:tasks.workunit.client.0.smithi031.stdout:6/244: dwrite d3/f18 [0,4194304] 0 2024-04-22T23:45:38.231 INFO:tasks.workunit.client.0.smithi031.stdout:6/245: fsync d3/d2a/f30 0 2024-04-22T23:45:38.232 INFO:tasks.workunit.client.0.smithi031.stdout:4/321: chown db/d14/d2d/d3c/l5a 854938 1 2024-04-22T23:45:38.233 INFO:tasks.workunit.client.0.smithi031.stdout:0/173: mkdir d0/d1/d2/d36/d11/d1a/d3c 0 2024-04-22T23:45:38.235 INFO:tasks.workunit.client.0.smithi031.stdout:6/246: truncate d3/d1f/f4f 114851 0 2024-04-22T23:45:38.238 INFO:tasks.workunit.client.0.smithi031.stdout:9/247: read d2/f5 [3859082,125226] 0 2024-04-22T23:45:38.239 INFO:tasks.workunit.client.0.smithi031.stdout:8/218: creat d5/da/d12/d37/d3b/f40 x:0 0 0 2024-04-22T23:45:38.240 INFO:tasks.workunit.client.0.smithi031.stdout:3/168: dwrite d4/d6/f16 [0,4194304] 0 2024-04-22T23:45:38.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/219: write d5/f19 [4235231,126624] 0 2024-04-22T23:45:38.246 INFO:tasks.workunit.client.0.smithi031.stdout:6/247: dwrite d3/d2a/d3c/d40/f46 [0,4194304] 0 2024-04-22T23:45:38.250 INFO:tasks.workunit.client.0.smithi031.stdout:0/174: creat d0/f3d x:0 0 0 2024-04-22T23:45:38.251 INFO:tasks.workunit.client.0.smithi031.stdout:0/175: write d0/d1/d2/f37 [1660349,2813] 0 2024-04-22T23:45:38.256 INFO:tasks.workunit.client.0.smithi031.stdout:9/248: fdatasync d2/dc/d3e/f41 0 2024-04-22T23:45:38.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/248: symlink d3/d1f/l54 0 2024-04-22T23:45:38.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/249: dwrite d2/dc/f1e [0,4194304] 0 2024-04-22T23:45:38.266 INFO:tasks.workunit.client.0.smithi031.stdout:4/322: mknod db/d14/d2d/d49/d59/d5b/c6c 0 2024-04-22T23:45:38.269 INFO:tasks.workunit.client.0.smithi031.stdout:4/323: stat db/d14/d2d 0 2024-04-22T23:45:38.269 INFO:tasks.workunit.client.0.smithi031.stdout:7/240: sync 2024-04-22T23:45:38.269 INFO:tasks.workunit.client.0.smithi031.stdout:1/133: sync 2024-04-22T23:45:38.272 INFO:tasks.workunit.client.0.smithi031.stdout:8/220: unlink d5/c16 0 2024-04-22T23:45:38.277 INFO:tasks.workunit.client.0.smithi031.stdout:0/176: creat d0/d1/d2/d36/d11/d1a/d3c/f3e x:0 0 0 2024-04-22T23:45:38.277 INFO:tasks.workunit.client.0.smithi031.stdout:9/250: dwrite d2/d47/d4a/f36 [4194304,4194304] 0 2024-04-22T23:45:38.277 INFO:tasks.workunit.client.0.smithi031.stdout:6/249: dwrite d3/d2a/d3c/d40/f46 [0,4194304] 0 2024-04-22T23:45:38.279 INFO:tasks.workunit.client.0.smithi031.stdout:3/169: creat d4/f32 x:0 0 0 2024-04-22T23:45:38.280 INFO:tasks.workunit.client.0.smithi031.stdout:4/324: mknod db/d31/d4f/c6d 0 2024-04-22T23:45:38.283 INFO:tasks.workunit.client.0.smithi031.stdout:8/221: dread d5/f19 [0,4194304] 0 2024-04-22T23:45:38.287 INFO:tasks.workunit.client.0.smithi031.stdout:6/250: symlink d3/d1f/d4c/l55 0 2024-04-22T23:45:38.288 INFO:tasks.workunit.client.0.smithi031.stdout:1/134: mkdir d1/de/d1e/d34 0 2024-04-22T23:45:38.290 INFO:tasks.workunit.client.0.smithi031.stdout:3/170: rmdir d4/d10/d2e 39 2024-04-22T23:45:38.291 INFO:tasks.workunit.client.0.smithi031.stdout:2/181: dwrite d6/dc/f1a [0,4194304] 0 2024-04-22T23:45:38.292 INFO:tasks.workunit.client.0.smithi031.stdout:4/325: creat db/d14/f6e x:0 0 0 2024-04-22T23:45:38.296 INFO:tasks.workunit.client.0.smithi031.stdout:2/182: write d6/f19 [3075902,15362] 0 2024-04-22T23:45:38.297 INFO:tasks.workunit.client.0.smithi031.stdout:9/251: fsync d2/dc/f17 0 2024-04-22T23:45:38.302 INFO:tasks.workunit.client.0.smithi031.stdout:1/135: dwrite d1/d21/d29/f1b [0,4194304] 0 2024-04-22T23:45:38.304 INFO:tasks.workunit.client.0.smithi031.stdout:6/251: readlink d3/l2c 0 2024-04-22T23:45:38.304 INFO:tasks.workunit.client.0.smithi031.stdout:5/207: dwrite db/f25 [0,4194304] 0 2024-04-22T23:45:38.305 INFO:tasks.workunit.client.0.smithi031.stdout:1/136: write d1/de/f11 [3935253,78643] 0 2024-04-22T23:45:38.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/208: chown db/d16/f18 10410437 1 2024-04-22T23:45:38.309 INFO:tasks.workunit.client.0.smithi031.stdout:1/137: dread - d1/db/d16/d1f/f26 zero size 2024-04-22T23:45:38.310 INFO:tasks.workunit.client.0.smithi031.stdout:0/177: getdents d0/d1/d2/d36/d11/d1c 0 2024-04-22T23:45:38.311 INFO:tasks.workunit.client.0.smithi031.stdout:4/326: rename db/d10/f12 to db/d14/d2d/d3c/f6f 0 2024-04-22T23:45:38.313 INFO:tasks.workunit.client.0.smithi031.stdout:4/327: write f4 [3235117,93384] 0 2024-04-22T23:45:38.313 INFO:tasks.workunit.client.0.smithi031.stdout:6/252: dread d3/f18 [0,4194304] 0 2024-04-22T23:45:38.315 INFO:tasks.workunit.client.0.smithi031.stdout:6/253: chown d3/f45 1893803 1 2024-04-22T23:45:38.316 INFO:tasks.workunit.client.0.smithi031.stdout:9/252: truncate d2/f35 1678420 0 2024-04-22T23:45:38.321 INFO:tasks.workunit.client.0.smithi031.stdout:1/138: rmdir d1/db/d16/d1f 39 2024-04-22T23:45:38.327 INFO:tasks.workunit.client.0.smithi031.stdout:7/241: dwrite d9/d14/d22/d28/f36 [0,4194304] 0 2024-04-22T23:45:38.329 INFO:tasks.workunit.client.0.smithi031.stdout:8/222: mkdir d5/da/df/d41 0 2024-04-22T23:45:38.329 INFO:tasks.workunit.client.0.smithi031.stdout:7/242: chown d9/d14/d1f/l3c 14 1 2024-04-22T23:45:38.331 INFO:tasks.workunit.client.0.smithi031.stdout:7/243: truncate d9/d15/f56 596884 0 2024-04-22T23:45:38.335 INFO:tasks.workunit.client.0.smithi031.stdout:2/183: dread d6/f19 [0,4194304] 0 2024-04-22T23:45:38.341 INFO:tasks.workunit.client.0.smithi031.stdout:2/184: dwrite f4 [4194304,4194304] 0 2024-04-22T23:45:38.342 INFO:tasks.workunit.client.0.smithi031.stdout:2/185: stat d6/d1e/d24/f34 0 2024-04-22T23:45:38.342 INFO:tasks.workunit.client.0.smithi031.stdout:2/186: chown d6/ff 4 1 2024-04-22T23:45:38.344 INFO:tasks.workunit.client.0.smithi031.stdout:5/209: symlink db/d16/l45 0 2024-04-22T23:45:38.345 INFO:tasks.workunit.client.0.smithi031.stdout:6/254: dread d3/d1f/f28 [0,4194304] 0 2024-04-22T23:45:38.346 INFO:tasks.workunit.client.0.smithi031.stdout:6/255: fsync d3/d2a/d3c/d40/f46 0 2024-04-22T23:45:38.346 INFO:tasks.workunit.client.0.smithi031.stdout:3/171: mknod d4/d6/d11/d25/c33 0 2024-04-22T23:45:38.348 INFO:tasks.workunit.client.0.smithi031.stdout:3/172: read - d4/d6/f23 zero size 2024-04-22T23:45:38.352 INFO:tasks.workunit.client.0.smithi031.stdout:4/328: fsync db/fc 0 2024-04-22T23:45:38.354 INFO:tasks.workunit.client.0.smithi031.stdout:0/178: dwrite d0/d1/f10 [0,4194304] 0 2024-04-22T23:45:38.355 INFO:tasks.workunit.client.0.smithi031.stdout:7/244: unlink d9/d14/d1f/l3c 0 2024-04-22T23:45:38.363 INFO:tasks.workunit.client.0.smithi031.stdout:2/187: chown d6/d1e/f2c 33 1 2024-04-22T23:45:38.367 INFO:tasks.workunit.client.0.smithi031.stdout:1/139: mkdir d1/d35 0 2024-04-22T23:45:38.368 INFO:tasks.workunit.client.0.smithi031.stdout:3/173: rename d4/d6/d11/c15 to d4/d6/d11/c34 0 2024-04-22T23:45:38.371 INFO:tasks.workunit.client.0.smithi031.stdout:4/329: rmdir db/d31/d4f 39 2024-04-22T23:45:38.373 INFO:tasks.workunit.client.0.smithi031.stdout:4/330: write db/d31/d40/f68 [260356,2342] 0 2024-04-22T23:45:38.374 INFO:tasks.workunit.client.0.smithi031.stdout:0/179: readlink d0/d1/d2/d36/d11/d1c/l35 0 2024-04-22T23:45:38.375 INFO:tasks.workunit.client.0.smithi031.stdout:7/245: mkdir d9/d14/d22/d5b 0 2024-04-22T23:45:38.376 INFO:tasks.workunit.client.0.smithi031.stdout:3/174: dwrite d4/d6/d11/f1c [0,4194304] 0 2024-04-22T23:45:38.377 INFO:tasks.workunit.client.0.smithi031.stdout:0/180: readlink d0/d13/d18/d1e/d25/l2e 0 2024-04-22T23:45:38.377 INFO:tasks.workunit.client.0.smithi031.stdout:9/253: creat d2/dc/f4c x:0 0 0 2024-04-22T23:45:38.378 INFO:tasks.workunit.client.0.smithi031.stdout:8/223: write d5/da/f20 [1259699,73735] 0 2024-04-22T23:45:38.379 INFO:tasks.workunit.client.0.smithi031.stdout:2/188: mknod d6/d1e/d24/c38 0 2024-04-22T23:45:38.381 INFO:tasks.workunit.client.0.smithi031.stdout:9/254: fdatasync d2/dc/d3e/f41 0 2024-04-22T23:45:38.383 INFO:tasks.workunit.client.0.smithi031.stdout:7/246: dread d9/fa [0,4194304] 0 2024-04-22T23:45:38.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/247: dread - d9/d14/d22/f2d zero size 2024-04-22T23:45:38.387 INFO:tasks.workunit.client.0.smithi031.stdout:4/331: rename l5 to db/d14/d2d/d3c/l70 0 2024-04-22T23:45:38.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/248: dread d9/d15/f21 [0,4194304] 0 2024-04-22T23:45:38.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/249: dread - d9/d15/d3a/d41/f54 zero size 2024-04-22T23:45:38.393 INFO:tasks.workunit.client.0.smithi031.stdout:6/256: write d3/f1a [3918673,102289] 0 2024-04-22T23:45:38.394 INFO:tasks.workunit.client.0.smithi031.stdout:2/189: creat d6/d1e/d24/f39 x:0 0 0 2024-04-22T23:45:38.395 INFO:tasks.workunit.client.0.smithi031.stdout:1/140: mkdir d1/d35/d36 0 2024-04-22T23:45:38.395 INFO:tasks.workunit.client.0.smithi031.stdout:4/332: dwrite db/d14/d2d/d49/d59/d6a/d41/f4b [0,4194304] 0 2024-04-22T23:45:38.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/190: dread d6/dc/d17/f25 [0,4194304] 0 2024-04-22T23:45:38.397 INFO:tasks.workunit.client.0.smithi031.stdout:9/255: rmdir d2/dc/d21/d3d/d40/d2a 39 2024-04-22T23:45:38.401 INFO:tasks.workunit.client.0.smithi031.stdout:5/210: getdents db 0 2024-04-22T23:45:38.401 INFO:tasks.workunit.client.0.smithi031.stdout:3/175: write d4/f1e [5026614,63882] 0 2024-04-22T23:45:38.404 INFO:tasks.workunit.client.0.smithi031.stdout:7/250: mkdir d9/d14/d22/d5c 0 2024-04-22T23:45:38.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/181: mkdir d0/d1/d3f 0 2024-04-22T23:45:38.406 INFO:tasks.workunit.client.0.smithi031.stdout:9/256: dwrite d2/dc/d21/d3d/d40/f44 [0,4194304] 0 2024-04-22T23:45:38.409 INFO:tasks.workunit.client.0.smithi031.stdout:6/257: creat d3/d1f/d4c/f56 x:0 0 0 2024-04-22T23:45:38.410 INFO:tasks.workunit.client.0.smithi031.stdout:4/333: mknod db/d31/d40/d43/c71 0 2024-04-22T23:45:38.411 INFO:tasks.workunit.client.0.smithi031.stdout:7/251: dread d9/d14/f18 [0,4194304] 0 2024-04-22T23:45:38.412 INFO:tasks.workunit.client.0.smithi031.stdout:6/258: dread d3/d2a/f30 [0,4194304] 0 2024-04-22T23:45:38.413 INFO:tasks.workunit.client.0.smithi031.stdout:6/259: chown d3/d1f/c2d 35512394 1 2024-04-22T23:45:38.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/260: chown d3/d2a/d3c/f43 593 1 2024-04-22T23:45:38.417 INFO:tasks.workunit.client.0.smithi031.stdout:0/182: creat d0/d1/d2/d36/d11/d1a/d3c/f40 x:0 0 0 2024-04-22T23:45:38.419 INFO:tasks.workunit.client.0.smithi031.stdout:8/224: getdents d5/da/df/d41 0 2024-04-22T23:45:38.423 INFO:tasks.workunit.client.0.smithi031.stdout:4/334: mkdir db/d14/d2d/d3c/d72 0 2024-04-22T23:45:38.426 INFO:tasks.workunit.client.0.smithi031.stdout:4/335: read db/d14/d2d/d49/d59/d6a/d41/f4b [1064155,30613] 0 2024-04-22T23:45:38.426 INFO:tasks.workunit.client.0.smithi031.stdout:4/336: fsync db/d14/f3a 0 2024-04-22T23:45:38.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/183: dwrite d0/d1/d2/fa [0,4194304] 0 2024-04-22T23:45:38.436 INFO:tasks.workunit.client.0.smithi031.stdout:0/184: dread - d0/d1/d2/d36/d11/d1a/d3c/f3e zero size 2024-04-22T23:45:38.440 INFO:tasks.workunit.client.0.smithi031.stdout:4/337: rename db/d14/d2d/d3c/d69 to db/d31/d40/d73 0 2024-04-22T23:45:38.441 INFO:tasks.workunit.client.0.smithi031.stdout:8/225: truncate d5/da/df/f1b 909517 0 2024-04-22T23:45:38.442 INFO:tasks.workunit.client.0.smithi031.stdout:1/141: link d1/db/d16/c31 d1/db/d16/c37 0 2024-04-22T23:45:38.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:37 smithi031 ceph-mon[26721]: pgmap v25: 65 pgs: 65 active+clean; 293 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-22T23:45:38.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:37 smithi031 ceph-mon[26721]: mgrmap e25: smithi044.iglcti(active, since 44s), standbys: smithi031.fazdcv 2024-04-22T23:45:38.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:37 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:38.446 INFO:tasks.workunit.client.0.smithi031.stdout:6/261: dread d3/f34 [0,4194304] 0 2024-04-22T23:45:38.447 INFO:tasks.workunit.client.0.smithi031.stdout:4/338: creat db/d10/d2a/f74 x:0 0 0 2024-04-22T23:45:38.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/262: creat d3/d2a/d3c/d40/f57 x:0 0 0 2024-04-22T23:45:38.452 INFO:tasks.workunit.client.0.smithi031.stdout:1/142: truncate d1/db/d16/d1f/f26 833480 0 2024-04-22T23:45:38.453 INFO:tasks.workunit.client.0.smithi031.stdout:1/143: chown d1/d35/d36 1470 1 2024-04-22T23:45:38.454 INFO:tasks.workunit.client.0.smithi031.stdout:8/226: dwrite d5/da/d12/f1c [4194304,4194304] 0 2024-04-22T23:45:38.455 INFO:tasks.workunit.client.0.smithi031.stdout:8/227: fsync d5/f9 0 2024-04-22T23:45:38.456 INFO:tasks.workunit.client.0.smithi031.stdout:9/257: truncate d2/d47/d4a/f36 359973 0 2024-04-22T23:45:38.459 INFO:tasks.workunit.client.0.smithi031.stdout:4/339: getdents db/d14/d2d/d49/d59 0 2024-04-22T23:45:38.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:37 smithi044 ceph-mon[34463]: pgmap v25: 65 pgs: 65 active+clean; 293 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-22T23:45:38.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:37 smithi044 ceph-mon[34463]: mgrmap e25: smithi044.iglcti(active, since 44s), standbys: smithi031.fazdcv 2024-04-22T23:45:38.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:37 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:38.460 INFO:tasks.workunit.client.0.smithi031.stdout:4/340: write db/d14/d2d/f35 [2410101,67673] 0 2024-04-22T23:45:38.461 INFO:tasks.workunit.client.0.smithi031.stdout:5/211: write db/d16/d1d/d24/f2e [2455002,91304] 0 2024-04-22T23:45:38.462 INFO:tasks.workunit.client.0.smithi031.stdout:5/212: chown db/d16/d2a/l33 591475156 1 2024-04-22T23:45:38.462 INFO:tasks.workunit.client.0.smithi031.stdout:4/341: dread - db/d10/d2a/d47/f54 zero size 2024-04-22T23:45:38.465 INFO:tasks.workunit.client.0.smithi031.stdout:4/342: dread db/d31/d40/f68 [0,4194304] 0 2024-04-22T23:45:38.466 INFO:tasks.workunit.client.0.smithi031.stdout:4/343: readlink db/d14/l28 0 2024-04-22T23:45:38.468 INFO:tasks.workunit.client.0.smithi031.stdout:5/213: dread db/f1e [0,4194304] 0 2024-04-22T23:45:38.470 INFO:tasks.workunit.client.0.smithi031.stdout:5/214: dread - db/d16/d2a/f2c zero size 2024-04-22T23:45:38.470 INFO:tasks.workunit.client.0.smithi031.stdout:4/344: write db/d10/d2a/d30/f60 [2691036,112636] 0 2024-04-22T23:45:38.473 INFO:tasks.workunit.client.0.smithi031.stdout:1/144: write d1/d21/d29/f23 [1812027,58268] 0 2024-04-22T23:45:38.475 INFO:tasks.workunit.client.0.smithi031.stdout:4/345: unlink db/d31/f5c 0 2024-04-22T23:45:38.479 INFO:tasks.workunit.client.0.smithi031.stdout:6/263: dwrite d3/f11 [0,4194304] 0 2024-04-22T23:45:38.482 INFO:tasks.workunit.client.0.smithi031.stdout:6/264: fsync d3/f18 0 2024-04-22T23:45:38.484 INFO:tasks.workunit.client.0.smithi031.stdout:9/258: dwrite d2/f2e [0,4194304] 0 2024-04-22T23:45:38.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/346: dread db/d14/f3a [4194304,4194304] 0 2024-04-22T23:45:38.486 INFO:tasks.workunit.client.0.smithi031.stdout:4/347: chown db/d14/d2d/d49/f53 179032 1 2024-04-22T23:45:38.487 INFO:tasks.workunit.client.0.smithi031.stdout:5/215: link db/d16/d1d/f20 db/d16/d1d/d24/d32/d41/f46 0 2024-04-22T23:45:38.489 INFO:tasks.workunit.client.0.smithi031.stdout:1/145: write d1/f8 [689550,21722] 0 2024-04-22T23:45:38.490 INFO:tasks.workunit.client.0.smithi031.stdout:5/216: chown db/d16/c3d 53 1 2024-04-22T23:45:38.494 INFO:tasks.workunit.client.0.smithi031.stdout:6/265: dwrite d3/d2a/d3c/d40/f4a [0,4194304] 0 2024-04-22T23:45:38.497 INFO:tasks.workunit.client.0.smithi031.stdout:1/146: mknod d1/d35/c38 0 2024-04-22T23:45:38.498 INFO:tasks.workunit.client.0.smithi031.stdout:4/348: dwrite db/d14/f3a [8388608,4194304] 0 2024-04-22T23:45:38.502 INFO:tasks.workunit.client.0.smithi031.stdout:1/147: write d1/d21/d29/f1a [2579844,123495] 0 2024-04-22T23:45:38.505 INFO:tasks.workunit.client.0.smithi031.stdout:5/217: creat db/d16/d1d/f47 x:0 0 0 2024-04-22T23:45:38.508 INFO:tasks.workunit.client.0.smithi031.stdout:9/259: dwrite d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:38.511 INFO:tasks.workunit.client.0.smithi031.stdout:1/148: chown d1/db/l27 1014745 1 2024-04-22T23:45:38.511 INFO:tasks.workunit.client.0.smithi031.stdout:6/266: dwrite d3/f32 [0,4194304] 0 2024-04-22T23:45:38.512 INFO:tasks.workunit.client.0.smithi031.stdout:9/260: fdatasync d2/dc/f25 0 2024-04-22T23:45:38.515 INFO:tasks.workunit.client.0.smithi031.stdout:4/349: dwrite db/d14/f6e [0,4194304] 0 2024-04-22T23:45:38.515 INFO:tasks.workunit.client.0.smithi031.stdout:9/261: write d2/dc/f43 [666510,113468] 0 2024-04-22T23:45:38.515 INFO:tasks.workunit.client.0.smithi031.stdout:6/267: write d3/d2a/d3c/d40/f44 [4521371,24539] 0 2024-04-22T23:45:38.518 INFO:tasks.workunit.client.0.smithi031.stdout:1/149: mknod d1/d21/c39 0 2024-04-22T23:45:38.519 INFO:tasks.workunit.client.0.smithi031.stdout:1/150: chown d1/d21/c39 1985 1 2024-04-22T23:45:38.526 INFO:tasks.workunit.client.0.smithi031.stdout:6/268: creat d3/d2a/d3c/d40/d4d/f58 x:0 0 0 2024-04-22T23:45:38.533 INFO:tasks.workunit.client.0.smithi031.stdout:6/269: creat d3/d35/f59 x:0 0 0 2024-04-22T23:45:38.536 INFO:tasks.workunit.client.0.smithi031.stdout:6/270: truncate d3/fe 1951009 0 2024-04-22T23:45:38.537 INFO:tasks.workunit.client.0.smithi031.stdout:6/271: chown d3/l2c 518 1 2024-04-22T23:45:38.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/272: symlink d3/d2a/d3c/d40/d4d/l5a 0 2024-04-22T23:45:38.543 INFO:tasks.workunit.client.0.smithi031.stdout:6/273: dread d3/f15 [0,4194304] 0 2024-04-22T23:45:38.545 INFO:tasks.workunit.client.0.smithi031.stdout:6/274: truncate d3/d1f/d4c/f56 673083 0 2024-04-22T23:45:38.545 INFO:tasks.workunit.client.0.smithi031.stdout:4/350: truncate db/d10/f2f 1442518 0 2024-04-22T23:45:38.546 INFO:tasks.workunit.client.0.smithi031.stdout:6/275: chown d3/l2c 2514 1 2024-04-22T23:45:38.548 INFO:tasks.workunit.client.0.smithi031.stdout:6/276: mkdir d3/d2a/d3c/d40/d5b 0 2024-04-22T23:45:38.550 INFO:tasks.workunit.client.0.smithi031.stdout:4/351: dread db/d14/d1d/f25 [0,4194304] 0 2024-04-22T23:45:38.550 INFO:tasks.workunit.client.0.smithi031.stdout:6/277: creat d3/d1f/f5c x:0 0 0 2024-04-22T23:45:38.551 INFO:tasks.workunit.client.0.smithi031.stdout:6/278: fdatasync d3/d2a/d3c/f52 0 2024-04-22T23:45:38.552 INFO:tasks.workunit.client.0.smithi031.stdout:4/352: symlink db/d14/d1d/l75 0 2024-04-22T23:45:38.554 INFO:tasks.workunit.client.0.smithi031.stdout:6/279: creat d3/d35/f5d x:0 0 0 2024-04-22T23:45:38.555 INFO:tasks.workunit.client.0.smithi031.stdout:4/353: rename db/d10/d2a/f74 to db/d14/d1d/f76 0 2024-04-22T23:45:38.557 INFO:tasks.workunit.client.0.smithi031.stdout:6/280: mkdir d3/d2a/d3c/d5e 0 2024-04-22T23:45:38.560 INFO:tasks.workunit.client.0.smithi031.stdout:6/281: write d3/f15 [420458,20420] 0 2024-04-22T23:45:38.560 INFO:tasks.workunit.client.0.smithi031.stdout:1/151: dread d1/d21/d29/f1a [0,4194304] 0 2024-04-22T23:45:38.562 INFO:tasks.workunit.client.0.smithi031.stdout:4/354: dread db/d14/d1d/f20 [0,4194304] 0 2024-04-22T23:45:38.563 INFO:tasks.workunit.client.0.smithi031.stdout:1/152: mknod d1/de/c3a 0 2024-04-22T23:45:38.568 INFO:tasks.workunit.client.0.smithi031.stdout:6/282: dwrite d3/d1f/f2f [0,4194304] 0 2024-04-22T23:45:38.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/355: dread db/d14/f45 [0,4194304] 0 2024-04-22T23:45:38.570 INFO:tasks.workunit.client.0.smithi031.stdout:1/153: dread d1/d21/d29/f1b [0,4194304] 0 2024-04-22T23:45:38.571 INFO:tasks.workunit.client.0.smithi031.stdout:6/283: write d3/d2a/d3c/d40/f46 [1528283,126857] 0 2024-04-22T23:45:38.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/154: write d1/d21/d29/d28/f2a [817558,101753] 0 2024-04-22T23:45:38.578 INFO:tasks.workunit.client.0.smithi031.stdout:6/284: creat d3/d2a/d3c/f5f x:0 0 0 2024-04-22T23:45:38.579 INFO:tasks.workunit.client.0.smithi031.stdout:4/356: creat db/d31/d40/d62/f77 x:0 0 0 2024-04-22T23:45:38.580 INFO:tasks.workunit.client.0.smithi031.stdout:1/155: creat d1/db/d16/d2f/f3b x:0 0 0 2024-04-22T23:45:38.581 INFO:tasks.workunit.client.0.smithi031.stdout:1/156: chown d1/d2e 718458912 1 2024-04-22T23:45:38.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/285: readlink d3/l3b 0 2024-04-22T23:45:38.585 INFO:tasks.workunit.client.0.smithi031.stdout:6/286: mkdir d3/d1f/d60 0 2024-04-22T23:45:38.587 INFO:tasks.workunit.client.0.smithi031.stdout:4/357: creat db/d31/d4f/f78 x:0 0 0 2024-04-22T23:45:38.588 INFO:tasks.workunit.client.0.smithi031.stdout:6/287: creat d3/d2a/d3c/d40/d4d/f61 x:0 0 0 2024-04-22T23:45:38.590 INFO:tasks.workunit.client.0.smithi031.stdout:4/358: creat db/d10/d2a/f79 x:0 0 0 2024-04-22T23:45:38.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/359: creat db/d10/d2a/d47/f7a x:0 0 0 2024-04-22T23:45:38.593 INFO:tasks.workunit.client.0.smithi031.stdout:4/360: write db/d14/f6e [2116918,3554] 0 2024-04-22T23:45:38.594 INFO:tasks.workunit.client.0.smithi031.stdout:6/288: unlink d3/lf 0 2024-04-22T23:45:38.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/361: mkdir db/d14/d1d/d7b 0 2024-04-22T23:45:38.601 INFO:tasks.workunit.client.0.smithi031.stdout:6/289: creat d3/d1f/d60/f62 x:0 0 0 2024-04-22T23:45:38.602 INFO:tasks.workunit.client.0.smithi031.stdout:4/362: mknod db/d14/d2d/d49/c7c 0 2024-04-22T23:45:38.606 INFO:tasks.workunit.client.0.smithi031.stdout:4/363: creat db/d14/d2d/d49/d59/d6a/d41/f7d x:0 0 0 2024-04-22T23:45:38.616 INFO:tasks.workunit.client.0.smithi031.stdout:4/364: getdents db/d14/d2d/d49/d59/d5b 0 2024-04-22T23:45:38.619 INFO:tasks.workunit.client.0.smithi031.stdout:4/365: creat db/d14/d2d/d49/d59/d6a/d41/f7e x:0 0 0 2024-04-22T23:45:38.619 INFO:tasks.workunit.client.0.smithi031.stdout:4/366: chown db/d31/d40/d43 127033 1 2024-04-22T23:45:38.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/367: chown db/d31/d4f 176 1 2024-04-22T23:45:38.621 INFO:tasks.workunit.client.0.smithi031.stdout:4/368: fsync db/d14/f45 0 2024-04-22T23:45:38.628 INFO:tasks.workunit.client.0.smithi031.stdout:4/369: dread db/fc [0,4194304] 0 2024-04-22T23:45:38.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/290: dwrite d3/fe [0,4194304] 0 2024-04-22T23:45:38.630 INFO:tasks.workunit.client.0.smithi031.stdout:4/370: fdatasync db/d14/d2d/d49/d59/d6a/f64 0 2024-04-22T23:45:38.638 INFO:tasks.workunit.client.0.smithi031.stdout:6/291: rename d3/d2a/f30 to d3/d2a/d3c/d40/d5b/f63 0 2024-04-22T23:45:38.641 INFO:tasks.workunit.client.0.smithi031.stdout:4/371: dwrite db/fc [0,4194304] 0 2024-04-22T23:45:38.643 INFO:tasks.workunit.client.0.smithi031.stdout:6/292: creat d3/d1f/f64 x:0 0 0 2024-04-22T23:45:38.647 INFO:tasks.workunit.client.0.smithi031.stdout:6/293: mkdir d3/d2a/d3c/d65 0 2024-04-22T23:45:38.650 INFO:tasks.workunit.client.0.smithi031.stdout:4/372: dwrite db/d14/f3a [0,4194304] 0 2024-04-22T23:45:38.654 INFO:tasks.workunit.client.0.smithi031.stdout:6/294: dread d3/d2a/d3c/d40/f46 [0,4194304] 0 2024-04-22T23:45:38.657 INFO:tasks.workunit.client.0.smithi031.stdout:4/373: rename db/d10 to db/d14/d1d/d7b/d7f 0 2024-04-22T23:45:38.663 INFO:tasks.workunit.client.0.smithi031.stdout:4/374: mkdir db/d14/d1d/d7b/d7f/d2a/d47/d80 0 2024-04-22T23:45:38.663 INFO:tasks.workunit.client.0.smithi031.stdout:4/375: dread - db/d31/d40/d62/f77 zero size 2024-04-22T23:45:38.674 INFO:tasks.workunit.client.0.smithi031.stdout:6/295: write d3/f18 [1752212,77116] 0 2024-04-22T23:45:38.676 INFO:tasks.workunit.client.0.smithi031.stdout:6/296: write d3/f15 [2914363,58835] 0 2024-04-22T23:45:38.680 INFO:tasks.workunit.client.0.smithi031.stdout:4/376: dwrite db/d14/d1d/d7b/d7f/f2f [0,4194304] 0 2024-04-22T23:45:38.680 INFO:tasks.workunit.client.0.smithi031.stdout:6/297: mknod d3/c66 0 2024-04-22T23:45:38.682 INFO:tasks.workunit.client.0.smithi031.stdout:6/298: write d3/d2a/f53 [781401,20513] 0 2024-04-22T23:45:38.684 INFO:tasks.workunit.client.0.smithi031.stdout:6/299: chown d3/d1f/c2d 9332955 1 2024-04-22T23:45:38.687 INFO:tasks.workunit.client.0.smithi031.stdout:6/300: write d3/d2a/d3c/d40/f46 [2071440,109379] 0 2024-04-22T23:45:38.690 INFO:tasks.workunit.client.0.smithi031.stdout:6/301: mkdir d3/d1f/d4c/d67 0 2024-04-22T23:45:38.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/302: write d3/f17 [6917214,57355] 0 2024-04-22T23:45:38.695 INFO:tasks.workunit.client.0.smithi031.stdout:6/303: mknod d3/d2a/d3c/d40/d4d/c68 0 2024-04-22T23:45:38.697 INFO:tasks.workunit.client.0.smithi031.stdout:6/304: creat d3/d2a/d3c/d40/d4d/f69 x:0 0 0 2024-04-22T23:45:38.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/305: readlink d3/d1f/l27 0 2024-04-22T23:45:38.704 INFO:tasks.workunit.client.0.smithi031.stdout:6/306: dwrite d3/d1f/f2f [4194304,4194304] 0 2024-04-22T23:45:38.706 INFO:tasks.workunit.client.0.smithi031.stdout:2/191: sync 2024-04-22T23:45:38.707 INFO:tasks.workunit.client.0.smithi031.stdout:6/307: write d3/f39 [2313721,74990] 0 2024-04-22T23:45:38.709 INFO:tasks.workunit.client.0.smithi031.stdout:6/308: chown d3/d1f/f5c 24373893 1 2024-04-22T23:45:38.715 INFO:tasks.workunit.client.0.smithi031.stdout:2/192: dwrite d6/f11 [0,4194304] 0 2024-04-22T23:45:38.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/193: creat d6/d1e/d24/d32/f3a x:0 0 0 2024-04-22T23:45:38.724 INFO:tasks.workunit.client.0.smithi031.stdout:2/194: rename d6/d1e/c23 to d6/d1e/d24/d32/d35/c3b 0 2024-04-22T23:45:38.726 INFO:tasks.workunit.client.0.smithi031.stdout:2/195: creat d6/dc/d17/f3c x:0 0 0 2024-04-22T23:45:38.728 INFO:tasks.workunit.client.0.smithi031.stdout:2/196: mknod d6/d1e/d24/d32/c3d 0 2024-04-22T23:45:38.730 INFO:tasks.workunit.client.0.smithi031.stdout:2/197: readlink d6/d1e/d24/d32/l33 0 2024-04-22T23:45:38.734 INFO:tasks.workunit.client.0.smithi031.stdout:2/198: getdents d6/dc 0 2024-04-22T23:45:38.736 INFO:tasks.workunit.client.0.smithi031.stdout:2/199: mknod d6/dc/d17/c3e 0 2024-04-22T23:45:38.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/200: write f3 [5898133,55242] 0 2024-04-22T23:45:38.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/201: readlink d6/l7 0 2024-04-22T23:45:38.744 INFO:tasks.workunit.client.0.smithi031.stdout:2/202: dwrite d6/f1c [0,4194304] 0 2024-04-22T23:45:38.747 INFO:tasks.workunit.client.0.smithi031.stdout:2/203: symlink d6/dc/d17/l3f 0 2024-04-22T23:45:38.752 INFO:tasks.workunit.client.0.smithi031.stdout:2/204: rename d6/dc/f26 to d6/f40 0 2024-04-22T23:45:38.761 INFO:tasks.workunit.client.0.smithi031.stdout:2/205: dwrite d6/f16 [0,4194304] 0 2024-04-22T23:45:38.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/206: truncate d6/d1e/d28/f2a 821205 0 2024-04-22T23:45:38.767 INFO:tasks.workunit.client.0.smithi031.stdout:2/207: creat d6/d1e/d28/f41 x:0 0 0 2024-04-22T23:45:38.787 INFO:tasks.workunit.client.0.smithi031.stdout:2/208: rmdir d6/d1e/d28 39 2024-04-22T23:45:38.794 INFO:tasks.workunit.client.0.smithi031.stdout:2/209: dread d6/dc/f1a [0,4194304] 0 2024-04-22T23:45:38.802 INFO:tasks.workunit.client.0.smithi031.stdout:2/210: rename d6/d1e to d6/dc/d42 0 2024-04-22T23:45:38.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/211: creat d6/dc/d42/f43 x:0 0 0 2024-04-22T23:45:38.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/212: truncate d6/f19 4842049 0 2024-04-22T23:45:38.814 INFO:tasks.workunit.client.0.smithi031.stdout:2/213: dread f4 [4194304,4194304] 0 2024-04-22T23:45:38.818 INFO:tasks.workunit.client.0.smithi031.stdout:2/214: fsync d6/dc/d17/f25 0 2024-04-22T23:45:38.823 INFO:tasks.workunit.client.0.smithi031.stdout:2/215: mknod d6/dc/d42/d24/c44 0 2024-04-22T23:45:38.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/216: mkdir d6/d45 0 2024-04-22T23:45:38.829 INFO:tasks.workunit.client.0.smithi031.stdout:2/217: read d6/f11 [2591686,112206] 0 2024-04-22T23:45:38.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/218: chown d6/dc/d42/f2e 3 1 2024-04-22T23:45:38.843 INFO:tasks.workunit.client.0.smithi031.stdout:2/219: rename d6/dc/d42/d24/d32/c3d to d6/c46 0 2024-04-22T23:45:38.846 INFO:tasks.workunit.client.0.smithi031.stdout:2/220: symlink d6/dc/d42/l47 0 2024-04-22T23:45:38.850 INFO:tasks.workunit.client.0.smithi031.stdout:2/221: unlink d6/dc/d17/c1b 0 2024-04-22T23:45:38.852 INFO:tasks.workunit.client.0.smithi031.stdout:2/222: fsync d6/dc/d42/d28/f37 0 2024-04-22T23:45:38.852 INFO:tasks.workunit.client.0.smithi031.stdout:2/223: readlink d6/l7 0 2024-04-22T23:45:38.855 INFO:tasks.workunit.client.0.smithi031.stdout:5/218: sync 2024-04-22T23:45:38.855 INFO:tasks.workunit.client.0.smithi031.stdout:3/176: sync 2024-04-22T23:45:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:9/262: sync 2024-04-22T23:45:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:7/252: sync 2024-04-22T23:45:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:1/157: sync 2024-04-22T23:45:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:0/185: sync 2024-04-22T23:45:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:8/228: sync 2024-04-22T23:45:38.856 INFO:tasks.workunit.client.0.smithi031.stdout:2/224: mkdir d6/dc/d48 0 2024-04-22T23:45:38.857 INFO:tasks.workunit.client.0.smithi031.stdout:1/158: write d1/f6 [1927922,9972] 0 2024-04-22T23:45:38.858 INFO:tasks.workunit.client.0.smithi031.stdout:5/219: fsync db/d16/d1d/d24/d27/f2d 0 2024-04-22T23:45:38.859 INFO:tasks.workunit.client.0.smithi031.stdout:1/159: stat d1/la 0 2024-04-22T23:45:38.863 INFO:tasks.workunit.client.0.smithi031.stdout:2/225: chown d6/dc/d17/l2d 0 1 2024-04-22T23:45:38.865 INFO:tasks.workunit.client.0.smithi031.stdout:0/186: dread d0/d1/d2/d36/fd [0,4194304] 0 2024-04-22T23:45:38.866 INFO:tasks.workunit.client.0.smithi031.stdout:8/229: dwrite d5/da/f2c [0,4194304] 0 2024-04-22T23:45:38.869 INFO:tasks.workunit.client.0.smithi031.stdout:1/160: dwrite d1/f8 [0,4194304] 0 2024-04-22T23:45:38.879 INFO:tasks.workunit.client.0.smithi031.stdout:0/187: unlink d0/d1/d2/f9 0 2024-04-22T23:45:38.886 INFO:tasks.workunit.client.0.smithi031.stdout:5/220: symlink db/d16/l48 0 2024-04-22T23:45:38.887 INFO:tasks.workunit.client.0.smithi031.stdout:5/221: stat db/d16/d1d/f20 0 2024-04-22T23:45:38.891 INFO:tasks.workunit.client.0.smithi031.stdout:0/188: rmdir d0/d13/d18 39 2024-04-22T23:45:38.892 INFO:tasks.workunit.client.0.smithi031.stdout:0/189: chown d0/d1/d2/d36/d11/d1a/l29 0 1 2024-04-22T23:45:38.893 INFO:tasks.workunit.client.0.smithi031.stdout:0/190: dread - d0/d1/d2/d36/d11/d1a/d3c/f3e zero size 2024-04-22T23:45:38.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/191: stat d0/d1/d2/d36/l12 0 2024-04-22T23:45:38.894 INFO:tasks.workunit.client.0.smithi031.stdout:8/230: rename d5/da/df/d33/f39 to d5/da/d12/d37/d3b/f42 0 2024-04-22T23:45:38.895 INFO:tasks.workunit.client.0.smithi031.stdout:0/192: readlink d0/d1/d2/d36/l16 0 2024-04-22T23:45:38.895 INFO:tasks.workunit.client.0.smithi031.stdout:8/231: chown d5/fb 6616 1 2024-04-22T23:45:38.899 INFO:tasks.workunit.client.0.smithi031.stdout:5/222: mknod db/c49 0 2024-04-22T23:45:38.904 INFO:tasks.workunit.client.0.smithi031.stdout:1/161: symlink d1/db/d16/l3c 0 2024-04-22T23:45:38.905 INFO:tasks.workunit.client.0.smithi031.stdout:3/177: write d4/d10/f2d [3313767,21810] 0 2024-04-22T23:45:38.908 INFO:tasks.workunit.client.0.smithi031.stdout:0/193: dread - d0/d13/d18/d1e/f30 zero size 2024-04-22T23:45:38.908 INFO:tasks.workunit.client.0.smithi031.stdout:7/253: dwrite d9/fa [0,4194304] 0 2024-04-22T23:45:38.909 INFO:tasks.workunit.client.0.smithi031.stdout:1/162: rename d1/db/d16/d2f to d1/db/d16/d2f/d30/d3d 22 2024-04-22T23:45:38.911 INFO:tasks.workunit.client.0.smithi031.stdout:3/178: readlink d4/l1a 0 2024-04-22T23:45:38.916 INFO:tasks.workunit.client.0.smithi031.stdout:0/194: symlink d0/d1/d2/d36/l41 0 2024-04-22T23:45:38.917 INFO:tasks.workunit.client.0.smithi031.stdout:7/254: dwrite d9/f3b [0,4194304] 0 2024-04-22T23:45:38.919 INFO:tasks.workunit.client.0.smithi031.stdout:8/232: creat d5/da/df/f43 x:0 0 0 2024-04-22T23:45:38.920 INFO:tasks.workunit.client.0.smithi031.stdout:5/223: dread db/d16/d1d/d24/f2e [0,4194304] 0 2024-04-22T23:45:38.920 INFO:tasks.workunit.client.0.smithi031.stdout:0/195: fdatasync d0/d1/d2/d36/d11/d1a/d3c/f3e 0 2024-04-22T23:45:38.924 INFO:tasks.workunit.client.0.smithi031.stdout:3/179: dread d4/d6/d11/f1c [0,4194304] 0 2024-04-22T23:45:38.925 INFO:tasks.workunit.client.0.smithi031.stdout:0/196: write d0/d1/d2/d36/d11/d1a/d3c/f40 [465152,84164] 0 2024-04-22T23:45:38.926 INFO:tasks.workunit.client.0.smithi031.stdout:0/197: chown d0 3 1 2024-04-22T23:45:38.928 INFO:tasks.workunit.client.0.smithi031.stdout:2/226: truncate d6/dc/d42/d24/f2b 313424 0 2024-04-22T23:45:38.928 INFO:tasks.workunit.client.0.smithi031.stdout:1/163: dwrite d1/d21/d29/f1a [0,4194304] 0 2024-04-22T23:45:38.930 INFO:tasks.workunit.client.0.smithi031.stdout:1/164: write d1/f6 [2237890,93282] 0 2024-04-22T23:45:38.930 INFO:tasks.workunit.client.0.smithi031.stdout:1/165: readlink d1/d21/l2d 0 2024-04-22T23:45:38.933 INFO:tasks.workunit.client.0.smithi031.stdout:8/233: dread d5/da/d12/d37/d3b/f42 [0,4194304] 0 2024-04-22T23:45:38.937 INFO:tasks.workunit.client.0.smithi031.stdout:5/224: mkdir db/d4a 0 2024-04-22T23:45:38.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/225: dread db/d16/d1d/d24/f2e [0,4194304] 0 2024-04-22T23:45:38.942 INFO:tasks.workunit.client.0.smithi031.stdout:3/180: rename d4/fe to d4/d6/da/f35 0 2024-04-22T23:45:38.942 INFO:tasks.workunit.client.0.smithi031.stdout:3/181: fsync d4/d10/f1b 0 2024-04-22T23:45:38.943 INFO:tasks.workunit.client.0.smithi031.stdout:7/255: getdents d9/d14/d1f/d47 0 2024-04-22T23:45:38.944 INFO:tasks.workunit.client.0.smithi031.stdout:7/256: chown d9/d15/f21 856957422 1 2024-04-22T23:45:38.945 INFO:tasks.workunit.client.0.smithi031.stdout:7/257: write d9/d14/f23 [804106,59091] 0 2024-04-22T23:45:38.946 INFO:tasks.workunit.client.0.smithi031.stdout:0/198: mknod d0/d1/d2/c42 0 2024-04-22T23:45:38.947 INFO:tasks.workunit.client.0.smithi031.stdout:3/182: dread d4/d6/d11/f1c [0,4194304] 0 2024-04-22T23:45:38.952 INFO:tasks.workunit.client.0.smithi031.stdout:1/166: write d1/d21/d29/f1b [2391628,18168] 0 2024-04-22T23:45:38.959 INFO:tasks.workunit.client.0.smithi031.stdout:5/226: mknod db/d16/d1d/d24/c4b 0 2024-04-22T23:45:38.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/234: dwrite d5/da/d12/d37/f3c [0,4194304] 0 2024-04-22T23:45:38.962 INFO:tasks.workunit.client.0.smithi031.stdout:5/227: read db/f42 [1927024,81502] 0 2024-04-22T23:45:38.965 INFO:tasks.workunit.client.0.smithi031.stdout:4/377: sync 2024-04-22T23:45:38.969 INFO:tasks.workunit.client.0.smithi031.stdout:0/199: creat d0/d1/d2/d36/d11/f43 x:0 0 0 2024-04-22T23:45:38.970 INFO:tasks.workunit.client.0.smithi031.stdout:3/183: rmdir d4/d6/d22 39 2024-04-22T23:45:38.973 INFO:tasks.workunit.client.0.smithi031.stdout:2/227: mkdir d6/dc/d49 0 2024-04-22T23:45:38.975 INFO:tasks.workunit.client.0.smithi031.stdout:4/378: dwrite db/d14/d1d/d7b/d7f/f32 [0,4194304] 0 2024-04-22T23:45:38.978 INFO:tasks.workunit.client.0.smithi031.stdout:0/200: dread d0/d1/f10 [0,4194304] 0 2024-04-22T23:45:38.979 INFO:tasks.workunit.client.0.smithi031.stdout:8/235: creat d5/da/d12/f44 x:0 0 0 2024-04-22T23:45:38.985 INFO:tasks.workunit.client.0.smithi031.stdout:3/184: creat d4/d6/d11/d25/f36 x:0 0 0 2024-04-22T23:45:38.987 INFO:tasks.workunit.client.0.smithi031.stdout:1/167: creat d1/d2e/f3e x:0 0 0 2024-04-22T23:45:38.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/168: write d1/f8 [342293,77870] 0 2024-04-22T23:45:38.989 INFO:tasks.workunit.client.0.smithi031.stdout:3/185: read f3 [142860,5880] 0 2024-04-22T23:45:38.992 INFO:tasks.workunit.client.0.smithi031.stdout:5/228: mknod db/d4a/c4c 0 2024-04-22T23:45:38.993 INFO:tasks.workunit.client.0.smithi031.stdout:5/229: chown db/fd 13275997 1 2024-04-22T23:45:38.996 INFO:tasks.workunit.client.0.smithi031.stdout:8/236: fsync d5/da/d12/f44 0 2024-04-22T23:45:38.997 INFO:tasks.workunit.client.0.smithi031.stdout:7/258: dread f2 [0,4194304] 0 2024-04-22T23:45:38.999 INFO:tasks.workunit.client.0.smithi031.stdout:4/379: dwrite db/d14/d1d/d7b/d7f/d2a/d47/f54 [0,4194304] 0 2024-04-22T23:45:38.999 INFO:tasks.workunit.client.0.smithi031.stdout:0/201: rename d0/d1/d2/ce to d0/d1/d2/d36/d11/d1c/c44 0 2024-04-22T23:45:38.999 INFO:tasks.workunit.client.0.smithi031.stdout:7/259: write f6 [3223462,52936] 0 2024-04-22T23:45:39.001 INFO:tasks.workunit.client.0.smithi031.stdout:0/202: write d0/d13/d18/f39 [371795,64361] 0 2024-04-22T23:45:39.003 INFO:tasks.workunit.client.0.smithi031.stdout:4/380: stat db/l23 0 2024-04-22T23:45:39.005 INFO:tasks.workunit.client.0.smithi031.stdout:2/228: symlink d6/dc/d48/l4a 0 2024-04-22T23:45:39.007 INFO:tasks.workunit.client.0.smithi031.stdout:5/230: dread db/d16/d1d/d24/d32/f39 [0,4194304] 0 2024-04-22T23:45:39.011 INFO:tasks.workunit.client.0.smithi031.stdout:3/186: creat d4/d6/d11/f37 x:0 0 0 2024-04-22T23:45:39.012 INFO:tasks.workunit.client.0.smithi031.stdout:6/309: sync 2024-04-22T23:45:39.012 INFO:tasks.workunit.client.0.smithi031.stdout:3/187: read d4/f1e [4420365,38441] 0 2024-04-22T23:45:39.014 INFO:tasks.workunit.client.0.smithi031.stdout:8/237: chown d5/da/d12/d2e 28 1 2024-04-22T23:45:39.020 INFO:tasks.workunit.client.0.smithi031.stdout:2/229: mkdir d6/dc/d48/d4b 0 2024-04-22T23:45:39.022 INFO:tasks.workunit.client.0.smithi031.stdout:4/381: creat db/d14/d1d/d7b/d7f/d2a/d30/f81 x:0 0 0 2024-04-22T23:45:39.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/382: truncate db/d14/d1d/d7b/d7f/d2a/d47/f54 4728579 0 2024-04-22T23:45:39.025 INFO:tasks.workunit.client.0.smithi031.stdout:1/169: mkdir d1/d3f 0 2024-04-22T23:45:39.028 INFO:tasks.workunit.client.0.smithi031.stdout:3/188: mknod d4/d10/d2a/c38 0 2024-04-22T23:45:39.030 INFO:tasks.workunit.client.0.smithi031.stdout:3/189: truncate d4/d10/f2b 131643 0 2024-04-22T23:45:39.032 INFO:tasks.workunit.client.0.smithi031.stdout:5/231: write f7 [257284,44762] 0 2024-04-22T23:45:39.033 INFO:tasks.workunit.client.0.smithi031.stdout:7/260: creat d9/d14/d22/d5b/f5d x:0 0 0 2024-04-22T23:45:39.034 INFO:tasks.workunit.client.0.smithi031.stdout:5/232: readlink db/d16/d1d/d24/d27/l3b 0 2024-04-22T23:45:39.037 INFO:tasks.workunit.client.0.smithi031.stdout:4/383: mknod db/d14/d1d/d7b/d7f/c82 0 2024-04-22T23:45:39.037 INFO:tasks.workunit.client.0.smithi031.stdout:6/310: dwrite d3/d1f/f28 [4194304,4194304] 0 2024-04-22T23:45:39.037 INFO:tasks.workunit.client.0.smithi031.stdout:4/384: fsync f4 0 2024-04-22T23:45:39.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/385: write db/d14/d1d/d7b/d7f/d2a/d30/f60 [384718,47911] 0 2024-04-22T23:45:39.042 INFO:tasks.workunit.client.0.smithi031.stdout:4/386: write db/d31/f33 [1996456,17449] 0 2024-04-22T23:45:39.042 INFO:tasks.workunit.client.0.smithi031.stdout:0/203: creat d0/d1/f45 x:0 0 0 2024-04-22T23:45:39.043 INFO:tasks.workunit.client.0.smithi031.stdout:4/387: stat db/d14/d1d/d7b/d7f/l22 0 2024-04-22T23:45:39.045 INFO:tasks.workunit.client.0.smithi031.stdout:7/261: creat d9/d14/f5e x:0 0 0 2024-04-22T23:45:39.046 INFO:tasks.workunit.client.0.smithi031.stdout:6/311: dread d3/f15 [0,4194304] 0 2024-04-22T23:45:39.046 INFO:tasks.workunit.client.0.smithi031.stdout:0/204: write d0/d1/d2/d36/d11/d1a/d3c/f40 [775790,124516] 0 2024-04-22T23:45:39.047 INFO:tasks.workunit.client.0.smithi031.stdout:2/230: write d6/dc/d42/f2e [1039819,48154] 0 2024-04-22T23:45:39.048 INFO:tasks.workunit.client.0.smithi031.stdout:0/205: chown d0/f3d 77606 1 2024-04-22T23:45:39.049 INFO:tasks.workunit.client.0.smithi031.stdout:5/233: write db/d16/d1d/f1f [248896,117634] 0 2024-04-22T23:45:39.051 INFO:tasks.workunit.client.0.smithi031.stdout:2/231: fdatasync d6/dc/d42/d24/d32/f3a 0 2024-04-22T23:45:39.051 INFO:tasks.workunit.client.0.smithi031.stdout:3/190: creat d4/d10/d2e/f39 x:0 0 0 2024-04-22T23:45:39.054 INFO:tasks.workunit.client.0.smithi031.stdout:8/238: rmdir d5/da/df/d41 0 2024-04-22T23:45:39.056 INFO:tasks.workunit.client.0.smithi031.stdout:4/388: mkdir db/d14/d2d/d49/d59/d83 0 2024-04-22T23:45:39.058 INFO:tasks.workunit.client.0.smithi031.stdout:6/312: creat d3/d1f/d60/f6a x:0 0 0 2024-04-22T23:45:39.060 INFO:tasks.workunit.client.0.smithi031.stdout:5/234: readlink db/l1b 0 2024-04-22T23:45:39.061 INFO:tasks.workunit.client.0.smithi031.stdout:5/235: write db/d16/d2a/d36/f38 [701365,61461] 0 2024-04-22T23:45:39.062 INFO:tasks.workunit.client.0.smithi031.stdout:0/206: symlink d0/d1/d2/d36/d11/d1c/l46 0 2024-04-22T23:45:39.065 INFO:tasks.workunit.client.0.smithi031.stdout:8/239: write d5/da/df/f13 [2494009,82439] 0 2024-04-22T23:45:39.066 INFO:tasks.workunit.client.0.smithi031.stdout:8/240: read d5/da/f1e [85799,87186] 0 2024-04-22T23:45:39.067 INFO:tasks.workunit.client.0.smithi031.stdout:4/389: rename db/d14/d2d/d3c/c67 to db/d14/d2d/d49/d59/d6a/d63/c84 0 2024-04-22T23:45:39.069 INFO:tasks.workunit.client.0.smithi031.stdout:6/313: rmdir d3/d1f/d4c 39 2024-04-22T23:45:39.073 INFO:tasks.workunit.client.0.smithi031.stdout:1/170: getdents d1/db/d16 0 2024-04-22T23:45:39.074 INFO:tasks.workunit.client.0.smithi031.stdout:1/171: chown d1/l7 34347 1 2024-04-22T23:45:39.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/207: chown d0/d1/d2/d36/d11/d1a/c33 1114 1 2024-04-22T23:45:39.078 INFO:tasks.workunit.client.0.smithi031.stdout:0/208: rename d0/d1/d2/d36/d11/d1a to d0/d1/d2/d36/d11/d1a/d47 22 2024-04-22T23:45:39.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/241: rmdir d5/da/d12/d2f 39 2024-04-22T23:45:39.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/242: chown c3 1617 1 2024-04-22T23:45:39.087 INFO:tasks.workunit.client.0.smithi031.stdout:4/390: mknod db/d14/d2d/d49/d59/d6a/c85 0 2024-04-22T23:45:39.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/209: creat d0/d13/d18/f48 x:0 0 0 2024-04-22T23:45:39.098 INFO:tasks.workunit.client.0.smithi031.stdout:8/243: unlink d5/le 0 2024-04-22T23:45:39.100 INFO:tasks.workunit.client.0.smithi031.stdout:7/262: getdents d9/d14/d22 0 2024-04-22T23:45:39.105 INFO:tasks.workunit.client.0.smithi031.stdout:5/236: link db/l3a db/d16/d1d/d24/l4d 0 2024-04-22T23:45:39.110 INFO:tasks.workunit.client.0.smithi031.stdout:0/210: symlink d0/d1/d28/l49 0 2024-04-22T23:45:39.112 INFO:tasks.workunit.client.0.smithi031.stdout:8/244: symlink d5/da/d12/d37/l45 0 2024-04-22T23:45:39.120 INFO:tasks.workunit.client.0.smithi031.stdout:0/211: symlink d0/d1/d2/d36/d11/d1a/l4a 0 2024-04-22T23:45:39.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/245: unlink d5/da/d12/f2a 0 2024-04-22T23:45:39.123 INFO:tasks.workunit.client.0.smithi031.stdout:7/263: mkdir d9/d14/d22/d5c/d5f 0 2024-04-22T23:45:39.124 INFO:tasks.workunit.client.0.smithi031.stdout:5/237: dread db/d16/d1d/f22 [0,4194304] 0 2024-04-22T23:45:39.124 INFO:tasks.workunit.client.0.smithi031.stdout:7/264: fdatasync d9/d14/d22/d28/f4a 0 2024-04-22T23:45:39.124 INFO:tasks.workunit.client.0.smithi031.stdout:6/314: link d3/d1f/c4b d3/c6b 0 2024-04-22T23:45:39.126 INFO:tasks.workunit.client.0.smithi031.stdout:3/191: write d4/f5 [484622,53479] 0 2024-04-22T23:45:39.126 INFO:tasks.workunit.client.0.smithi031.stdout:5/238: write db/d16/d2a/f2c [912547,16194] 0 2024-04-22T23:45:39.129 INFO:tasks.workunit.client.0.smithi031.stdout:3/192: write d4/d6/da/f13 [340375,51972] 0 2024-04-22T23:45:39.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/212: rename d0/d1/d2/d36/l41 to d0/d1/d28/d2b/l4b 0 2024-04-22T23:45:39.131 INFO:tasks.workunit.client.0.smithi031.stdout:8/246: creat d5/da/d12/d37/d34/f46 x:0 0 0 2024-04-22T23:45:39.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/213: chown d0/d1/f45 125 1 2024-04-22T23:45:39.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/265: creat d9/d15/f60 x:0 0 0 2024-04-22T23:45:39.135 INFO:tasks.workunit.client.0.smithi031.stdout:6/315: dwrite d3/d1f/d60/f62 [0,4194304] 0 2024-04-22T23:45:39.136 INFO:tasks.workunit.client.0.smithi031.stdout:0/214: write d0/d1/d2/f23 [461956,69384] 0 2024-04-22T23:45:39.138 INFO:tasks.workunit.client.0.smithi031.stdout:5/239: symlink db/d16/d1d/d24/d27/l4e 0 2024-04-22T23:45:39.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/193: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:45:39.147 INFO:tasks.workunit.client.0.smithi031.stdout:8/247: dread d5/da/df/f1b [0,4194304] 0 2024-04-22T23:45:39.148 INFO:tasks.workunit.client.0.smithi031.stdout:8/248: truncate d5/da/f20 1870970 0 2024-04-22T23:45:39.175 INFO:tasks.workunit.client.0.smithi031.stdout:9/263: sync 2024-04-22T23:45:39.177 INFO:tasks.workunit.client.0.smithi031.stdout:3/194: unlink d4/f32 0 2024-04-22T23:45:39.178 INFO:tasks.workunit.client.0.smithi031.stdout:7/266: link f6 d9/d14/f61 0 2024-04-22T23:45:39.179 INFO:tasks.workunit.client.0.smithi031.stdout:8/249: dwrite d5/f9 [0,4194304] 0 2024-04-22T23:45:39.179 INFO:tasks.workunit.client.0.smithi031.stdout:1/172: truncate d1/f6 2176691 0 2024-04-22T23:45:39.180 INFO:tasks.workunit.client.0.smithi031.stdout:9/264: write d2/dc/f48 [4429711,123153] 0 2024-04-22T23:45:39.183 INFO:tasks.workunit.client.0.smithi031.stdout:1/173: chown f0 172744808 1 2024-04-22T23:45:39.187 INFO:tasks.workunit.client.0.smithi031.stdout:3/195: symlink d4/d10/d2a/d2c/l3a 0 2024-04-22T23:45:39.188 INFO:tasks.workunit.client.0.smithi031.stdout:4/391: dwrite db/d14/d1d/f20 [0,4194304] 0 2024-04-22T23:45:39.190 INFO:tasks.workunit.client.0.smithi031.stdout:0/215: dread d0/ff [0,4194304] 0 2024-04-22T23:45:39.191 INFO:tasks.workunit.client.0.smithi031.stdout:9/265: symlink d2/l4d 0 2024-04-22T23:45:39.191 INFO:tasks.workunit.client.0.smithi031.stdout:3/196: chown d4/d6/da/f13 98 1 2024-04-22T23:45:39.195 INFO:tasks.workunit.client.0.smithi031.stdout:5/240: dwrite db/f1e [0,4194304] 0 2024-04-22T23:45:39.196 INFO:tasks.workunit.client.0.smithi031.stdout:1/174: creat d1/db/d16/f40 x:0 0 0 2024-04-22T23:45:39.199 INFO:tasks.workunit.client.0.smithi031.stdout:5/241: readlink db/l40 0 2024-04-22T23:45:39.200 INFO:tasks.workunit.client.0.smithi031.stdout:1/175: write d1/d2e/f3e [724025,95788] 0 2024-04-22T23:45:39.201 INFO:tasks.workunit.client.0.smithi031.stdout:1/176: read - d1/f1d zero size 2024-04-22T23:45:39.207 INFO:tasks.workunit.client.0.smithi031.stdout:0/216: mkdir d0/d13/d18/d1e/d25/d4c 0 2024-04-22T23:45:39.207 INFO:tasks.workunit.client.0.smithi031.stdout:4/392: dwrite db/d14/d1d/d7b/d7f/f32 [0,4194304] 0 2024-04-22T23:45:39.207 INFO:tasks.workunit.client.0.smithi031.stdout:9/266: dwrite d2/dc/ff [0,4194304] 0 2024-04-22T23:45:39.210 INFO:tasks.workunit.client.0.smithi031.stdout:3/197: mkdir d4/d6/d11/d3b 0 2024-04-22T23:45:39.212 INFO:tasks.workunit.client.0.smithi031.stdout:7/267: getdents d9/d15/d3a 0 2024-04-22T23:45:39.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/242: rename db/d16/d1d/d24/d32/d41/f46 to db/d4a/f4f 0 2024-04-22T23:45:39.217 INFO:tasks.workunit.client.0.smithi031.stdout:0/217: truncate d0/d1/d2/d36/d11/f43 377525 0 2024-04-22T23:45:39.219 INFO:tasks.workunit.client.0.smithi031.stdout:7/268: write d9/d14/d22/f52 [485975,90872] 0 2024-04-22T23:45:39.222 INFO:tasks.workunit.client.0.smithi031.stdout:8/250: dwrite d5/da/df/f1b [0,4194304] 0 2024-04-22T23:45:39.222 INFO:tasks.workunit.client.0.smithi031.stdout:9/267: unlink d2/d15/c16 0 2024-04-22T23:45:39.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/269: chown d9/d14/d22/c34 54037304 1 2024-04-22T23:45:39.224 INFO:tasks.workunit.client.0.smithi031.stdout:4/393: mkdir db/d14/d2d/d49/d59/d6a/d63/d86 0 2024-04-22T23:45:39.226 INFO:tasks.workunit.client.0.smithi031.stdout:4/394: fsync db/d14/d1d/d7b/d7f/d2a/d30/f42 0 2024-04-22T23:45:39.227 INFO:tasks.workunit.client.0.smithi031.stdout:4/395: chown db/d14/d2d/d49/f53 11858849 1 2024-04-22T23:45:39.228 INFO:tasks.workunit.client.0.smithi031.stdout:3/198: dread d4/d6/f7 [0,4194304] 0 2024-04-22T23:45:39.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/243: rename c9 to db/d16/d1d/d24/d32/d41/c50 0 2024-04-22T23:45:39.234 INFO:tasks.workunit.client.0.smithi031.stdout:5/244: read f7 [182413,55705] 0 2024-04-22T23:45:39.235 INFO:tasks.workunit.client.0.smithi031.stdout:9/268: mknod d2/d15/c4e 0 2024-04-22T23:45:39.237 INFO:tasks.workunit.client.0.smithi031.stdout:8/251: creat d5/da/d12/d1d/d38/f47 x:0 0 0 2024-04-22T23:45:39.239 INFO:tasks.workunit.client.0.smithi031.stdout:3/199: write f3 [2817700,102484] 0 2024-04-22T23:45:39.242 INFO:tasks.workunit.client.0.smithi031.stdout:0/218: rename d0/d1/f10 to d0/d1/d2/d36/d11/d1a/d3c/f4d 0 2024-04-22T23:45:39.245 INFO:tasks.workunit.client.0.smithi031.stdout:8/252: dwrite d5/da/d12/d1d/d38/f3f [0,4194304] 0 2024-04-22T23:45:39.245 INFO:tasks.workunit.client.0.smithi031.stdout:5/245: creat db/d4a/f51 x:0 0 0 2024-04-22T23:45:39.246 INFO:tasks.workunit.client.0.smithi031.stdout:9/269: rmdir d2/dd 39 2024-04-22T23:45:39.252 INFO:tasks.workunit.client.0.smithi031.stdout:3/200: creat d4/d6/d11/f3c x:0 0 0 2024-04-22T23:45:39.257 INFO:tasks.workunit.client.0.smithi031.stdout:1/177: rename d1/db/d16/d2f/d30 to d1/de/d1e/d34/d41 0 2024-04-22T23:45:39.258 INFO:tasks.workunit.client.0.smithi031.stdout:0/219: fsync d0/d1/d2/d36/f1d 0 2024-04-22T23:45:39.259 INFO:tasks.workunit.client.0.smithi031.stdout:8/253: mkdir d5/d48 0 2024-04-22T23:45:39.260 INFO:tasks.workunit.client.0.smithi031.stdout:9/270: rmdir d2/dc/d3e 39 2024-04-22T23:45:39.260 INFO:tasks.workunit.client.0.smithi031.stdout:7/270: dwrite f3 [0,4194304] 0 2024-04-22T23:45:39.266 INFO:tasks.workunit.client.0.smithi031.stdout:1/178: unlink d1/f8 0 2024-04-22T23:45:39.268 INFO:tasks.workunit.client.0.smithi031.stdout:5/246: rename db/d16/d1d/d24/d32/d41/c50 to db/d16/d1d/d24/d32/d41/c52 0 2024-04-22T23:45:39.269 INFO:tasks.workunit.client.0.smithi031.stdout:9/271: write d2/f20 [3322755,80457] 0 2024-04-22T23:45:39.271 INFO:tasks.workunit.client.0.smithi031.stdout:3/201: link d4/d6/f16 d4/d6/da/f3d 0 2024-04-22T23:45:39.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/271: dwrite d9/d14/f20 [0,4194304] 0 2024-04-22T23:45:39.275 INFO:tasks.workunit.client.0.smithi031.stdout:1/179: creat d1/de/d1e/d34/d41/f42 x:0 0 0 2024-04-22T23:45:39.275 INFO:tasks.workunit.client.0.smithi031.stdout:7/272: chown d9/d15/f21 60553 1 2024-04-22T23:45:39.278 INFO:tasks.workunit.client.0.smithi031.stdout:3/202: mkdir d4/d6/d11/d3e 0 2024-04-22T23:45:39.279 INFO:tasks.workunit.client.0.smithi031.stdout:7/273: chown d9/d14/f23 89187 1 2024-04-22T23:45:39.280 INFO:tasks.workunit.client.0.smithi031.stdout:7/274: stat d9/d14/d22 0 2024-04-22T23:45:39.280 INFO:tasks.workunit.client.0.smithi031.stdout:4/396: rename db/d14/d2d/d49/d59/d5b/c6c to db/d31/c87 0 2024-04-22T23:45:39.282 INFO:tasks.workunit.client.0.smithi031.stdout:5/247: dwrite db/f1e [0,4194304] 0 2024-04-22T23:45:39.284 INFO:tasks.workunit.client.0.smithi031.stdout:8/254: getdents d5/da/df 0 2024-04-22T23:45:39.284 INFO:tasks.workunit.client.0.smithi031.stdout:1/180: write d1/db/d16/d1f/f2c [637354,25249] 0 2024-04-22T23:45:39.285 INFO:tasks.workunit.client.0.smithi031.stdout:4/397: read f9 [2095983,13511] 0 2024-04-22T23:45:39.287 INFO:tasks.workunit.client.0.smithi031.stdout:0/220: dread d0/d1/d2/d36/f1d [0,4194304] 0 2024-04-22T23:45:39.289 INFO:tasks.workunit.client.0.smithi031.stdout:0/221: stat d0/d1/d28/l49 0 2024-04-22T23:45:39.289 INFO:tasks.workunit.client.0.smithi031.stdout:4/398: write db/d31/d4f/f78 [357435,100853] 0 2024-04-22T23:45:39.291 INFO:tasks.workunit.client.0.smithi031.stdout:7/275: rename d9/d14/l2b to d9/d14/d1f/d45/l62 0 2024-04-22T23:45:39.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/181: mknod d1/db/d16/d2f/c43 0 2024-04-22T23:45:39.295 INFO:tasks.workunit.client.0.smithi031.stdout:8/255: dread d5/da/f20 [0,4194304] 0 2024-04-22T23:45:39.297 INFO:tasks.workunit.client.0.smithi031.stdout:4/399: creat db/d14/d2d/d3c/f88 x:0 0 0 2024-04-22T23:45:39.298 INFO:tasks.workunit.client.0.smithi031.stdout:7/276: symlink d9/d14/d1f/d45/l63 0 2024-04-22T23:45:39.299 INFO:tasks.workunit.client.0.smithi031.stdout:7/277: chown d9/l4e 341912 1 2024-04-22T23:45:39.299 INFO:tasks.workunit.client.0.smithi031.stdout:7/278: fsync d9/d15/f56 0 2024-04-22T23:45:39.301 INFO:tasks.workunit.client.0.smithi031.stdout:0/222: creat d0/d1/d2/d36/d11/d1a/d3c/f4e x:0 0 0 2024-04-22T23:45:39.302 INFO:tasks.workunit.client.0.smithi031.stdout:0/223: dread - d0/f3d zero size 2024-04-22T23:45:39.303 INFO:tasks.workunit.client.0.smithi031.stdout:1/182: creat d1/de/d1e/d34/d41/f44 x:0 0 0 2024-04-22T23:45:39.305 INFO:tasks.workunit.client.0.smithi031.stdout:1/183: truncate d1/db/d16/f40 920871 0 2024-04-22T23:45:39.305 INFO:tasks.workunit.client.0.smithi031.stdout:8/256: mkdir d5/da/d12/d37/d49 0 2024-04-22T23:45:39.306 INFO:tasks.workunit.client.0.smithi031.stdout:1/184: chown d1/d21/d29/d28/c2b 0 1 2024-04-22T23:45:39.309 INFO:tasks.workunit.client.0.smithi031.stdout:7/279: rename f8 to d9/d14/d22/d28/f64 0 2024-04-22T23:45:39.309 INFO:tasks.workunit.client.0.smithi031.stdout:7/280: readlink d9/d14/l1a 0 2024-04-22T23:45:39.310 INFO:tasks.workunit.client.0.smithi031.stdout:3/203: dread d4/d6/da/f3d [0,4194304] 0 2024-04-22T23:45:39.312 INFO:tasks.workunit.client.0.smithi031.stdout:1/185: mknod d1/de/d1e/d34/d41/c45 0 2024-04-22T23:45:39.312 INFO:tasks.workunit.client.0.smithi031.stdout:3/204: dread d4/d10/f2b [0,4194304] 0 2024-04-22T23:45:39.312 INFO:tasks.workunit.client.0.smithi031.stdout:8/257: symlink d5/d48/l4a 0 2024-04-22T23:45:39.315 INFO:tasks.workunit.client.0.smithi031.stdout:5/248: dread db/d16/d1d/f20 [0,4194304] 0 2024-04-22T23:45:39.317 INFO:tasks.workunit.client.0.smithi031.stdout:3/205: write d4/d6/d11/f37 [199422,16119] 0 2024-04-22T23:45:39.317 INFO:tasks.workunit.client.0.smithi031.stdout:1/186: dread d1/db/d16/f40 [0,4194304] 0 2024-04-22T23:45:39.318 INFO:tasks.workunit.client.0.smithi031.stdout:9/272: dwrite d2/f5 [4194304,4194304] 0 2024-04-22T23:45:39.319 INFO:tasks.workunit.client.0.smithi031.stdout:7/281: chown d9/d14/d1f/l4d 67901695 1 2024-04-22T23:45:39.320 INFO:tasks.workunit.client.0.smithi031.stdout:8/258: truncate d5/da/f1e 518545 0 2024-04-22T23:45:39.321 INFO:tasks.workunit.client.0.smithi031.stdout:9/273: write d2/f5 [2325801,114289] 0 2024-04-22T23:45:39.322 INFO:tasks.workunit.client.0.smithi031.stdout:9/274: read - d2/dc/d21/d3d/d40/f39 zero size 2024-04-22T23:45:39.325 INFO:tasks.workunit.client.0.smithi031.stdout:7/282: mkdir d9/d4f/d65 0 2024-04-22T23:45:39.326 INFO:tasks.workunit.client.0.smithi031.stdout:3/206: mknod d4/c3f 0 2024-04-22T23:45:39.333 INFO:tasks.workunit.client.0.smithi031.stdout:1/187: link d1/f1d d1/f46 0 2024-04-22T23:45:39.335 INFO:tasks.workunit.client.0.smithi031.stdout:9/275: rename d2/f5 to d2/dc/d3e/f4f 0 2024-04-22T23:45:39.335 INFO:tasks.workunit.client.0.smithi031.stdout:1/188: chown d1/db/d16/d1f/f26 11677177 1 2024-04-22T23:45:39.335 INFO:tasks.workunit.client.0.smithi031.stdout:5/249: link db/d16/l48 db/d16/d1d/l53 0 2024-04-22T23:45:39.336 INFO:tasks.workunit.client.0.smithi031.stdout:3/207: dwrite d4/d10/f2b [0,4194304] 0 2024-04-22T23:45:39.338 INFO:tasks.workunit.client.0.smithi031.stdout:1/189: write d1/d21/d29/f1a [2015414,106310] 0 2024-04-22T23:45:39.343 INFO:tasks.workunit.client.0.smithi031.stdout:3/208: rmdir d4/d6 39 2024-04-22T23:45:39.350 INFO:tasks.workunit.client.0.smithi031.stdout:1/190: mkdir d1/d47 0 2024-04-22T23:45:39.352 INFO:tasks.workunit.client.0.smithi031.stdout:1/191: read d1/d21/d29/f23 [1614136,124756] 0 2024-04-22T23:45:39.353 INFO:tasks.workunit.client.0.smithi031.stdout:1/192: write d1/d21/d29/f1a [1803816,65164] 0 2024-04-22T23:45:39.353 INFO:tasks.workunit.client.0.smithi031.stdout:5/250: creat db/d16/f54 x:0 0 0 2024-04-22T23:45:39.358 INFO:tasks.workunit.client.0.smithi031.stdout:9/276: getdents d2/dc/d21 0 2024-04-22T23:45:39.360 INFO:tasks.workunit.client.0.smithi031.stdout:5/251: rmdir db 39 2024-04-22T23:45:39.364 INFO:tasks.workunit.client.0.smithi031.stdout:1/193: rename d1/db/l27 to d1/d35/l48 0 2024-04-22T23:45:39.366 INFO:tasks.workunit.client.0.smithi031.stdout:3/209: readlink d4/d6/d22/l24 0 2024-04-22T23:45:39.366 INFO:tasks.workunit.client.0.smithi031.stdout:3/210: stat d4/c18 0 2024-04-22T23:45:39.368 INFO:tasks.workunit.client.0.smithi031.stdout:8/259: truncate d5/f26 1773303 0 2024-04-22T23:45:39.368 INFO:tasks.workunit.client.0.smithi031.stdout:1/194: dread d1/d21/d29/d28/f2a [0,4194304] 0 2024-04-22T23:45:39.374 INFO:tasks.workunit.client.0.smithi031.stdout:3/211: mknod d4/d10/d2a/c40 0 2024-04-22T23:45:39.374 INFO:tasks.workunit.client.0.smithi031.stdout:1/195: dwrite f0 [0,4194304] 0 2024-04-22T23:45:39.377 INFO:tasks.workunit.client.0.smithi031.stdout:3/212: chown d4/d6/d11/f19 436 1 2024-04-22T23:45:39.378 INFO:tasks.workunit.client.0.smithi031.stdout:8/260: creat d5/da/d12/d37/d3b/f4b x:0 0 0 2024-04-22T23:45:39.383 INFO:tasks.workunit.client.0.smithi031.stdout:3/213: dwrite d4/d6/d11/f37 [0,4194304] 0 2024-04-22T23:45:39.383 INFO:tasks.workunit.client.0.smithi031.stdout:5/252: rmdir db/d16 39 2024-04-22T23:45:39.384 INFO:tasks.workunit.client.0.smithi031.stdout:3/214: fdatasync d4/d6/da/f1d 0 2024-04-22T23:45:39.385 INFO:tasks.workunit.client.0.smithi031.stdout:9/277: creat d2/dc/f50 x:0 0 0 2024-04-22T23:45:39.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/196: mknod d1/d21/d29/d28/c49 0 2024-04-22T23:45:39.389 INFO:tasks.workunit.client.0.smithi031.stdout:5/253: write db/d16/d1d/f1f [97716,58325] 0 2024-04-22T23:45:39.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/278: creat d2/dc/d21/d3d/f51 x:0 0 0 2024-04-22T23:45:39.394 INFO:tasks.workunit.client.0.smithi031.stdout:1/197: creat d1/d35/f4a x:0 0 0 2024-04-22T23:45:39.394 INFO:tasks.workunit.client.0.smithi031.stdout:3/215: dwrite d4/d6/d11/f37 [0,4194304] 0 2024-04-22T23:45:39.395 INFO:tasks.workunit.client.0.smithi031.stdout:1/198: chown d1/d35 934028733 1 2024-04-22T23:45:39.401 INFO:tasks.workunit.client.0.smithi031.stdout:3/216: write d4/d6/d11/f1c [3528670,107572] 0 2024-04-22T23:45:39.404 INFO:tasks.workunit.client.0.smithi031.stdout:5/254: symlink db/d16/d1d/l55 0 2024-04-22T23:45:39.406 INFO:tasks.workunit.client.0.smithi031.stdout:5/255: stat db/d16/d1d/d24/l43 0 2024-04-22T23:45:39.409 INFO:tasks.workunit.client.0.smithi031.stdout:9/279: dwrite d2/dd/f2b [0,4194304] 0 2024-04-22T23:45:39.410 INFO:tasks.workunit.client.0.smithi031.stdout:9/280: fdatasync d2/dc/f48 0 2024-04-22T23:45:39.411 INFO:tasks.workunit.client.0.smithi031.stdout:3/217: mknod d4/d6/d11/d3e/c41 0 2024-04-22T23:45:39.412 INFO:tasks.workunit.client.0.smithi031.stdout:1/199: dread d1/db/d16/f25 [0,4194304] 0 2024-04-22T23:45:39.416 INFO:tasks.workunit.client.0.smithi031.stdout:9/281: symlink d2/d15/l52 0 2024-04-22T23:45:39.416 INFO:tasks.workunit.client.0.smithi031.stdout:1/200: truncate d1/db/d16/d1f/f2c 1583665 0 2024-04-22T23:45:39.416 INFO:tasks.workunit.client.0.smithi031.stdout:8/261: read d5/f9 [232016,35113] 0 2024-04-22T23:45:39.417 INFO:tasks.workunit.client.0.smithi031.stdout:8/262: chown d5/da/d12/f14 209144347 1 2024-04-22T23:45:39.418 INFO:tasks.workunit.client.0.smithi031.stdout:8/263: chown d5/da/df/f43 34079 1 2024-04-22T23:45:39.419 INFO:tasks.workunit.client.0.smithi031.stdout:8/264: write d5/f8 [2739930,91759] 0 2024-04-22T23:45:39.420 INFO:tasks.workunit.client.0.smithi031.stdout:3/218: mkdir d4/d10/d2a/d2c/d42 0 2024-04-22T23:45:39.421 INFO:tasks.workunit.client.0.smithi031.stdout:9/282: symlink d2/d15/l53 0 2024-04-22T23:45:39.422 INFO:tasks.workunit.client.0.smithi031.stdout:5/256: creat db/d16/d1d/f56 x:0 0 0 2024-04-22T23:45:39.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/283: read d2/dc/d21/d3d/d40/f44 [3308811,104750] 0 2024-04-22T23:45:39.424 INFO:tasks.workunit.client.0.smithi031.stdout:5/257: truncate db/d16/d2a/f30 907103 0 2024-04-22T23:45:39.425 INFO:tasks.workunit.client.0.smithi031.stdout:1/201: mkdir d1/d21/d4b 0 2024-04-22T23:45:39.426 INFO:tasks.workunit.client.0.smithi031.stdout:8/265: creat d5/da/d12/d37/d34/f4c x:0 0 0 2024-04-22T23:45:39.427 INFO:tasks.workunit.client.0.smithi031.stdout:1/202: write d1/d35/f4a [28886,68933] 0 2024-04-22T23:45:39.428 INFO:tasks.workunit.client.0.smithi031.stdout:3/219: rename d4/d6/da/f1d to d4/d6/d11/d3e/f43 0 2024-04-22T23:45:39.429 INFO:tasks.workunit.client.0.smithi031.stdout:1/203: fsync d1/de/f11 0 2024-04-22T23:45:39.429 INFO:tasks.workunit.client.0.smithi031.stdout:8/266: write d5/da/d12/d1d/d38/f3f [1378839,13447] 0 2024-04-22T23:45:39.430 INFO:tasks.workunit.client.0.smithi031.stdout:1/204: stat d1/d21/d4b 0 2024-04-22T23:45:39.431 INFO:tasks.workunit.client.0.smithi031.stdout:8/267: chown d5/da/d12/d1d/d38/c3e 0 1 2024-04-22T23:45:39.431 INFO:tasks.workunit.client.0.smithi031.stdout:9/284: dread d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:39.432 INFO:tasks.workunit.client.0.smithi031.stdout:1/205: fsync d1/d21/d29/f1b 0 2024-04-22T23:45:39.435 INFO:tasks.workunit.client.0.smithi031.stdout:3/220: mkdir d4/d6/d11/d25/d44 0 2024-04-22T23:45:39.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/221: dread d4/f5 [0,4194304] 0 2024-04-22T23:45:39.439 INFO:tasks.workunit.client.0.smithi031.stdout:3/222: chown d4/d6/d22/l28 30349 1 2024-04-22T23:45:39.440 INFO:tasks.workunit.client.0.smithi031.stdout:5/258: link db/d16/d1d/d24/f2e db/d16/d1d/d24/d27/f57 0 2024-04-22T23:45:39.440 INFO:tasks.workunit.client.0.smithi031.stdout:3/223: write d4/d6/da/ff [836892,41522] 0 2024-04-22T23:45:39.447 INFO:tasks.workunit.client.0.smithi031.stdout:5/259: dwrite db/d16/d2a/f2c [0,4194304] 0 2024-04-22T23:45:39.447 INFO:tasks.workunit.client.0.smithi031.stdout:2/232: sync 2024-04-22T23:45:39.453 INFO:tasks.workunit.client.0.smithi031.stdout:3/224: mkdir d4/d10/d2e/d45 0 2024-04-22T23:45:39.453 INFO:tasks.workunit.client.0.smithi031.stdout:5/260: dwrite f5 [0,4194304] 0 2024-04-22T23:45:39.454 INFO:tasks.workunit.client.0.smithi031.stdout:8/268: mkdir d5/da/d12/d2e/d4d 0 2024-04-22T23:45:39.455 INFO:tasks.workunit.client.0.smithi031.stdout:3/225: chown d4/d10/d2a/d2c/l3a 32 1 2024-04-22T23:45:39.462 INFO:tasks.workunit.client.0.smithi031.stdout:2/233: symlink d6/dc/d42/d24/d32/d35/l4c 0 2024-04-22T23:45:39.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/261: rename db/d4a to db/d16/d1d/d24/d58 0 2024-04-22T23:45:39.466 INFO:tasks.workunit.client.0.smithi031.stdout:3/226: rmdir d4/d6 39 2024-04-22T23:45:39.468 INFO:tasks.workunit.client.0.smithi031.stdout:8/269: mkdir d5/da/d12/d37/d4e 0 2024-04-22T23:45:39.469 INFO:tasks.workunit.client.0.smithi031.stdout:8/270: fsync d5/da/d12/d37/d34/f46 0 2024-04-22T23:45:39.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/234: fdatasync d6/f11 0 2024-04-22T23:45:39.472 INFO:tasks.workunit.client.0.smithi031.stdout:2/235: fsync d6/dc/d42/d24/f34 0 2024-04-22T23:45:39.474 INFO:tasks.workunit.client.0.smithi031.stdout:5/262: mkdir db/d16/d2a/d59 0 2024-04-22T23:45:39.476 INFO:tasks.workunit.client.0.smithi031.stdout:3/227: write d4/d6/da/f13 [1046247,99399] 0 2024-04-22T23:45:39.476 INFO:tasks.workunit.client.0.smithi031.stdout:1/206: getdents d1/d21/d29 0 2024-04-22T23:45:39.478 INFO:tasks.workunit.client.0.smithi031.stdout:9/285: getdents d2/dc/d21 0 2024-04-22T23:45:39.479 INFO:tasks.workunit.client.0.smithi031.stdout:2/236: dwrite d6/ff [0,4194304] 0 2024-04-22T23:45:39.479 INFO:tasks.workunit.client.0.smithi031.stdout:9/286: chown d2/dc/d21/d3d 1761911 1 2024-04-22T23:45:39.480 INFO:tasks.workunit.client.0.smithi031.stdout:9/287: chown d2/dc/f4c 2 1 2024-04-22T23:45:39.481 INFO:tasks.workunit.client.0.smithi031.stdout:9/288: stat d2/dc 0 2024-04-22T23:45:39.481 INFO:tasks.workunit.client.0.smithi031.stdout:2/237: truncate d6/dc/d42/d24/d32/f3a 634855 0 2024-04-22T23:45:39.482 INFO:tasks.workunit.client.0.smithi031.stdout:5/263: stat db/d16/l48 0 2024-04-22T23:45:39.485 INFO:tasks.workunit.client.0.smithi031.stdout:3/228: mknod d4/d10/d2a/c46 0 2024-04-22T23:45:39.486 INFO:tasks.workunit.client.0.smithi031.stdout:1/207: fdatasync d1/db/d16/f25 0 2024-04-22T23:45:39.488 INFO:tasks.workunit.client.0.smithi031.stdout:8/271: creat d5/da/d12/d37/d49/f4f x:0 0 0 2024-04-22T23:45:39.488 INFO:tasks.workunit.client.0.smithi031.stdout:5/264: dread db/d16/d1d/d24/d27/f2d [0,4194304] 0 2024-04-22T23:45:39.491 INFO:tasks.workunit.client.0.smithi031.stdout:2/238: creat d6/dc/d42/d24/d32/d35/f4d x:0 0 0 2024-04-22T23:45:39.492 INFO:tasks.workunit.client.0.smithi031.stdout:5/265: dread db/f3f [0,4194304] 0 2024-04-22T23:45:39.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/229: dwrite d4/d10/f2b [0,4194304] 0 2024-04-22T23:45:39.494 INFO:tasks.workunit.client.0.smithi031.stdout:3/230: fdatasync d4/d6/d11/f3c 0 2024-04-22T23:45:39.495 INFO:tasks.workunit.client.0.smithi031.stdout:1/208: creat d1/de/d1e/d34/d41/f4c x:0 0 0 2024-04-22T23:45:39.497 INFO:tasks.workunit.client.0.smithi031.stdout:9/289: link d2/d15/f2c d2/d15/f54 0 2024-04-22T23:45:39.503 INFO:tasks.workunit.client.0.smithi031.stdout:3/231: mknod d4/d6/d11/c47 0 2024-04-22T23:45:39.505 INFO:tasks.workunit.client.0.smithi031.stdout:9/290: write d2/dc/d3e/f4f [8285525,118068] 0 2024-04-22T23:45:39.508 INFO:tasks.workunit.client.0.smithi031.stdout:6/316: sync 2024-04-22T23:45:39.508 INFO:tasks.workunit.client.0.smithi031.stdout:0/224: sync 2024-04-22T23:45:39.508 INFO:tasks.workunit.client.0.smithi031.stdout:4/400: sync 2024-04-22T23:45:39.508 INFO:tasks.workunit.client.0.smithi031.stdout:7/283: sync 2024-04-22T23:45:39.510 INFO:tasks.workunit.client.0.smithi031.stdout:4/401: dread - db/d14/d2d/d49/f53 zero size 2024-04-22T23:45:39.511 INFO:tasks.workunit.client.0.smithi031.stdout:0/225: dread - d0/d13/d18/f48 zero size 2024-04-22T23:45:39.512 INFO:tasks.workunit.client.0.smithi031.stdout:8/272: rename d5/da/df/d33/c2b to d5/da/d12/c50 0 2024-04-22T23:45:39.515 INFO:tasks.workunit.client.0.smithi031.stdout:4/402: chown db/d14/d2d/d49/d59/d6a 84 1 2024-04-22T23:45:39.516 INFO:tasks.workunit.client.0.smithi031.stdout:2/239: dread f3 [0,4194304] 0 2024-04-22T23:45:39.516 INFO:tasks.workunit.client.0.smithi031.stdout:8/273: fdatasync d5/da/d12/d37/d3b/f4b 0 2024-04-22T23:45:39.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/226: dread d0/ff [0,4194304] 0 2024-04-22T23:45:39.519 INFO:tasks.workunit.client.0.smithi031.stdout:3/232: rmdir d4/d10/d2a/d2c/d42 0 2024-04-22T23:45:39.520 INFO:tasks.workunit.client.0.smithi031.stdout:4/403: chown db/d14/d1d/d7b/d7f/d2a/d30/f81 244 1 2024-04-22T23:45:39.521 INFO:tasks.workunit.client.0.smithi031.stdout:6/317: dwrite d3/d1f/d60/f62 [4194304,4194304] 0 2024-04-22T23:45:39.521 INFO:tasks.workunit.client.0.smithi031.stdout:8/274: write d5/da/f2c [4112454,113924] 0 2024-04-22T23:45:39.523 INFO:tasks.workunit.client.0.smithi031.stdout:5/266: truncate db/d16/d2a/f2c 2827894 0 2024-04-22T23:45:39.525 INFO:tasks.workunit.client.0.smithi031.stdout:1/209: write d1/f6 [1903199,90809] 0 2024-04-22T23:45:39.525 INFO:tasks.workunit.client.0.smithi031.stdout:0/227: creat d0/f4f x:0 0 0 2024-04-22T23:45:39.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/291: getdents d2/d15 0 2024-04-22T23:45:39.532 INFO:tasks.workunit.client.0.smithi031.stdout:4/404: mkdir db/d14/d2d/d49/d59/d5b/d89 0 2024-04-22T23:45:39.532 INFO:tasks.workunit.client.0.smithi031.stdout:9/292: truncate d2/f35 2640608 0 2024-04-22T23:45:39.534 INFO:tasks.workunit.client.0.smithi031.stdout:2/240: dwrite f4 [4194304,4194304] 0 2024-04-22T23:45:39.536 INFO:tasks.workunit.client.0.smithi031.stdout:5/267: fsync db/f3f 0 2024-04-22T23:45:39.537 INFO:tasks.workunit.client.0.smithi031.stdout:1/210: mkdir d1/de/d1e/d34/d4d 0 2024-04-22T23:45:39.537 INFO:tasks.workunit.client.0.smithi031.stdout:5/268: chown db/d16 26497246 1 2024-04-22T23:45:39.541 INFO:tasks.workunit.client.0.smithi031.stdout:8/275: creat d5/da/d12/d2f/f51 x:0 0 0 2024-04-22T23:45:39.544 INFO:tasks.workunit.client.0.smithi031.stdout:3/233: dread d4/d10/f1b [0,4194304] 0 2024-04-22T23:45:39.544 INFO:tasks.workunit.client.0.smithi031.stdout:9/293: dwrite d2/dc/d3e/f4f [4194304,4194304] 0 2024-04-22T23:45:39.547 INFO:tasks.workunit.client.0.smithi031.stdout:9/294: read d2/f3c [612026,70257] 0 2024-04-22T23:45:39.548 INFO:tasks.workunit.client.0.smithi031.stdout:0/228: link d0/d1/d2/d36/f1d d0/d1/d2/f50 0 2024-04-22T23:45:39.550 INFO:tasks.workunit.client.0.smithi031.stdout:3/234: dread d4/d6/d11/f37 [0,4194304] 0 2024-04-22T23:45:39.550 INFO:tasks.workunit.client.0.smithi031.stdout:1/211: mkdir d1/d21/d4e 0 2024-04-22T23:45:39.551 INFO:tasks.workunit.client.0.smithi031.stdout:9/295: dread d2/dc/d3e/f41 [0,4194304] 0 2024-04-22T23:45:39.557 INFO:tasks.workunit.client.0.smithi031.stdout:6/318: rmdir d3/d1f/d4c/d67 0 2024-04-22T23:45:39.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/241: creat d6/dc/f4e x:0 0 0 2024-04-22T23:45:39.564 INFO:tasks.workunit.client.0.smithi031.stdout:1/212: rename d1/de/d1e/d34/d41/c45 to d1/db/d16/d2f/c4f 0 2024-04-22T23:45:39.565 INFO:tasks.workunit.client.0.smithi031.stdout:3/235: mknod d4/c48 0 2024-04-22T23:45:39.570 INFO:tasks.workunit.client.0.smithi031.stdout:8/276: creat d5/da/d12/d2e/d4d/f52 x:0 0 0 2024-04-22T23:45:39.572 INFO:tasks.workunit.client.0.smithi031.stdout:3/236: dread d4/d10/f2d [0,4194304] 0 2024-04-22T23:45:39.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/213: unlink d1/db/d16/d1f/f2c 0 2024-04-22T23:45:39.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/319: mkdir d3/d2a/d3c/d65/d6c 0 2024-04-22T23:45:39.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/277: stat d5/da/d12/d37/d3b/f4b 0 2024-04-22T23:45:39.575 INFO:tasks.workunit.client.0.smithi031.stdout:4/405: dread db/d14/d1d/d7b/d7f/d2a/d30/f60 [0,4194304] 0 2024-04-22T23:45:39.575 INFO:tasks.workunit.client.0.smithi031.stdout:9/296: dwrite d2/fb [4194304,4194304] 0 2024-04-22T23:45:39.577 INFO:tasks.workunit.client.0.smithi031.stdout:0/229: getdents d0/d1/d2/d36/d11/d1a 0 2024-04-22T23:45:39.581 INFO:tasks.workunit.client.0.smithi031.stdout:0/230: chown d0/d1/d2/d36/f2f 29129295 1 2024-04-22T23:45:39.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/320: unlink d3/d2a/l3d 0 2024-04-22T23:45:39.582 INFO:tasks.workunit.client.0.smithi031.stdout:1/214: truncate d1/de/d1e/d34/d41/f4c 670093 0 2024-04-22T23:45:39.585 INFO:tasks.workunit.client.0.smithi031.stdout:5/269: read db/d16/d2a/f2c [2491115,36924] 0 2024-04-22T23:45:39.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/215: dread - d1/db/d16/d2f/f3b zero size 2024-04-22T23:45:39.586 INFO:tasks.workunit.client.0.smithi031.stdout:4/406: rename db/d14/d2d/d49/d59/d5b to db/d14/d2d/d49/d59/d8a 0 2024-04-22T23:45:39.588 INFO:tasks.workunit.client.0.smithi031.stdout:6/321: symlink d3/d2a/d3c/l6d 0 2024-04-22T23:45:39.589 INFO:tasks.workunit.client.0.smithi031.stdout:2/242: dread d6/f9 [0,4194304] 0 2024-04-22T23:45:39.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/278: dwrite d5/da/d12/d37/d3b/f4b [0,4194304] 0 2024-04-22T23:45:39.592 INFO:tasks.workunit.client.0.smithi031.stdout:0/231: creat d0/d1/d28/d2b/f51 x:0 0 0 2024-04-22T23:45:39.592 INFO:tasks.workunit.client.0.smithi031.stdout:1/216: write d1/db/d16/f25 [1424209,119676] 0 2024-04-22T23:45:39.594 INFO:tasks.workunit.client.0.smithi031.stdout:6/322: fdatasync d3/d2a/d3c/d40/f46 0 2024-04-22T23:45:39.595 INFO:tasks.workunit.client.0.smithi031.stdout:9/297: rename d2/fb to d2/dc/d21/d3d/d40/d2a/f55 0 2024-04-22T23:45:39.597 INFO:tasks.workunit.client.0.smithi031.stdout:4/407: mkdir db/d14/d2d/d49/d59/d6a/d63/d8b 0 2024-04-22T23:45:39.599 INFO:tasks.workunit.client.0.smithi031.stdout:0/232: dread d0/d1/d2/d36/f1d [0,4194304] 0 2024-04-22T23:45:39.606 INFO:tasks.workunit.client.0.smithi031.stdout:4/408: dwrite db/d14/d1d/d7b/d7f/d2a/f79 [0,4194304] 0 2024-04-22T23:45:39.606 INFO:tasks.workunit.client.0.smithi031.stdout:1/217: creat d1/db/d16/d2f/f50 x:0 0 0 2024-04-22T23:45:39.608 INFO:tasks.workunit.client.0.smithi031.stdout:6/323: rmdir d3/d1f/d60 39 2024-04-22T23:45:39.609 INFO:tasks.workunit.client.0.smithi031.stdout:9/298: creat d2/dc/d3e/f56 x:0 0 0 2024-04-22T23:45:39.613 INFO:tasks.workunit.client.0.smithi031.stdout:5/270: link db/d16/d1d/f20 db/d16/d1d/d24/f5a 0 2024-04-22T23:45:39.614 INFO:tasks.workunit.client.0.smithi031.stdout:1/218: dwrite d1/de/d1e/d34/d41/f42 [0,4194304] 0 2024-04-22T23:45:39.616 INFO:tasks.workunit.client.0.smithi031.stdout:5/271: write db/d16/f54 [640650,123671] 0 2024-04-22T23:45:39.616 INFO:tasks.workunit.client.0.smithi031.stdout:1/219: dread - d1/db/d16/d2f/f50 zero size 2024-04-22T23:45:39.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/243: mknod d6/dc/d42/d28/c4f 0 2024-04-22T23:45:39.620 INFO:tasks.workunit.client.0.smithi031.stdout:0/233: creat d0/d1/d2/f52 x:0 0 0 2024-04-22T23:45:39.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/324: mknod d3/d2a/d3c/d40/c6e 0 2024-04-22T23:45:39.624 INFO:tasks.workunit.client.0.smithi031.stdout:9/299: readlink d2/dc/d21/d3d/d40/d2a/l3f 0 2024-04-22T23:45:39.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/272: creat db/d16/d1d/d24/d27/f5b x:0 0 0 2024-04-22T23:45:39.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/273: dread - db/d16/f3e zero size 2024-04-22T23:45:39.630 INFO:tasks.workunit.client.0.smithi031.stdout:8/279: symlink d5/da/d12/d2e/d30/l53 0 2024-04-22T23:45:39.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/234: mknod d0/d13/d18/d1e/d25/c53 0 2024-04-22T23:45:39.632 INFO:tasks.workunit.client.0.smithi031.stdout:6/325: symlink d3/d2a/l6f 0 2024-04-22T23:45:39.634 INFO:tasks.workunit.client.0.smithi031.stdout:9/300: creat d2/dc/d21/d38/f57 x:0 0 0 2024-04-22T23:45:39.634 INFO:tasks.workunit.client.0.smithi031.stdout:8/280: truncate d5/da/d12/d37/d34/f46 310027 0 2024-04-22T23:45:39.638 INFO:tasks.workunit.client.0.smithi031.stdout:0/235: mkdir d0/d1/d2/d36/d11/d1c/d54 0 2024-04-22T23:45:39.638 INFO:tasks.workunit.client.0.smithi031.stdout:5/274: dwrite db/d16/f54 [0,4194304] 0 2024-04-22T23:45:39.639 INFO:tasks.workunit.client.0.smithi031.stdout:6/326: creat d3/d2a/f70 x:0 0 0 2024-04-22T23:45:39.641 INFO:tasks.workunit.client.0.smithi031.stdout:9/301: creat d2/dc/d21/f58 x:0 0 0 2024-04-22T23:45:39.641 INFO:tasks.workunit.client.0.smithi031.stdout:5/275: dread db/d16/d1d/d24/d58/f4f [4194304,4194304] 0 2024-04-22T23:45:39.644 INFO:tasks.workunit.client.0.smithi031.stdout:2/244: getdents d6/dc/d42/d24/d32/d35 0 2024-04-22T23:45:39.647 INFO:tasks.workunit.client.0.smithi031.stdout:0/236: creat d0/d1/d28/f55 x:0 0 0 2024-04-22T23:45:39.649 INFO:tasks.workunit.client.0.smithi031.stdout:6/327: chown d3/c13 1010136 1 2024-04-22T23:45:39.650 INFO:tasks.workunit.client.0.smithi031.stdout:1/220: dwrite d1/f1d [0,4194304] 0 2024-04-22T23:45:39.650 INFO:tasks.workunit.client.0.smithi031.stdout:1/221: stat d1/de/d1e/d34 0 2024-04-22T23:45:39.652 INFO:tasks.workunit.client.0.smithi031.stdout:9/302: fsync d2/d15/f2c 0 2024-04-22T23:45:39.653 INFO:tasks.workunit.client.0.smithi031.stdout:1/222: read d1/f6 [559065,37773] 0 2024-04-22T23:45:39.654 INFO:tasks.workunit.client.0.smithi031.stdout:5/276: fdatasync db/d16/d1d/f22 0 2024-04-22T23:45:39.656 INFO:tasks.workunit.client.0.smithi031.stdout:1/223: dread - d1/db/d16/d2f/f50 zero size 2024-04-22T23:45:39.657 INFO:tasks.workunit.client.0.smithi031.stdout:8/281: link d5/da/d12/d1d/d38/f3f d5/da/d12/d37/d49/f54 0 2024-04-22T23:45:39.660 INFO:tasks.workunit.client.0.smithi031.stdout:0/237: mkdir d0/d1/d2/d36/d11/d1a/d56 0 2024-04-22T23:45:39.661 INFO:tasks.workunit.client.0.smithi031.stdout:8/282: truncate d5/da/d12/d2f/f51 765496 0 2024-04-22T23:45:39.662 INFO:tasks.workunit.client.0.smithi031.stdout:6/328: creat d3/d2a/d3c/f71 x:0 0 0 2024-04-22T23:45:39.664 INFO:tasks.workunit.client.0.smithi031.stdout:9/303: dwrite d2/dc/d3e/f41 [0,4194304] 0 2024-04-22T23:45:39.664 INFO:tasks.workunit.client.0.smithi031.stdout:0/238: write d0/d13/d18/f39 [76129,58413] 0 2024-04-22T23:45:39.666 INFO:tasks.workunit.client.0.smithi031.stdout:0/239: dread - d0/f4f zero size 2024-04-22T23:45:39.666 INFO:tasks.workunit.client.0.smithi031.stdout:6/329: fsync d3/d2a/d3c/d40/d4d/f58 0 2024-04-22T23:45:39.668 INFO:tasks.workunit.client.0.smithi031.stdout:2/245: mkdir d6/d45/d50 0 2024-04-22T23:45:39.669 INFO:tasks.workunit.client.0.smithi031.stdout:2/246: readlink d6/l7 0 2024-04-22T23:45:39.672 INFO:tasks.workunit.client.0.smithi031.stdout:2/247: write d6/dc/d42/d28/f2a [1100822,127402] 0 2024-04-22T23:45:39.673 INFO:tasks.workunit.client.0.smithi031.stdout:9/304: dwrite d2/dc/f48 [0,4194304] 0 2024-04-22T23:45:39.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/240: mknod d0/c57 0 2024-04-22T23:45:39.679 INFO:tasks.workunit.client.0.smithi031.stdout:8/283: dread d5/da/df/f13 [0,4194304] 0 2024-04-22T23:45:39.679 INFO:tasks.workunit.client.0.smithi031.stdout:6/330: rename d3/d2a/l6f to d3/d1f/d4c/l72 0 2024-04-22T23:45:39.680 INFO:tasks.workunit.client.0.smithi031.stdout:5/277: write db/d16/d1d/d24/d58/f4f [345176,118660] 0 2024-04-22T23:45:39.682 INFO:tasks.workunit.client.0.smithi031.stdout:9/305: mkdir d2/dc/d21/d38/d59 0 2024-04-22T23:45:39.684 INFO:tasks.workunit.client.0.smithi031.stdout:9/306: fdatasync d2/dc/d3e/f4f 0 2024-04-22T23:45:39.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/241: rename d0/d1/d2/f37 to d0/d1/d2/d17/f58 0 2024-04-22T23:45:39.689 INFO:tasks.workunit.client.0.smithi031.stdout:6/331: dwrite d3/d1f/f5c [0,4194304] 0 2024-04-22T23:45:39.690 INFO:tasks.workunit.client.0.smithi031.stdout:2/248: getdents d6/dc/d49 0 2024-04-22T23:45:39.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/242: creat d0/d1/d2/d36/d11/d1a/d3c/f59 x:0 0 0 2024-04-22T23:45:39.700 INFO:tasks.workunit.client.0.smithi031.stdout:9/307: dwrite d2/d47/d4a/f49 [0,4194304] 0 2024-04-22T23:45:39.700 INFO:tasks.workunit.client.0.smithi031.stdout:0/243: fdatasync d0/d1/f45 0 2024-04-22T23:45:39.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/284: link d5/da/d12/c50 d5/da/d12/d1d/c55 0 2024-04-22T23:45:39.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/278: dwrite db/d16/f3e [0,4194304] 0 2024-04-22T23:45:39.707 INFO:tasks.workunit.client.0.smithi031.stdout:6/332: dwrite d3/d1f/f2f [4194304,4194304] 0 2024-04-22T23:45:39.708 INFO:tasks.workunit.client.0.smithi031.stdout:9/308: dread - d2/dc/d21/f58 zero size 2024-04-22T23:45:39.711 INFO:tasks.workunit.client.0.smithi031.stdout:9/309: readlink d2/d15/l53 0 2024-04-22T23:45:39.711 INFO:tasks.workunit.client.0.smithi031.stdout:2/249: dwrite d6/dc/d42/d24/f39 [0,4194304] 0 2024-04-22T23:45:39.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/333: chown d3/c3f 153 1 2024-04-22T23:45:39.715 INFO:tasks.workunit.client.0.smithi031.stdout:5/279: write db/f42 [816974,30139] 0 2024-04-22T23:45:39.717 INFO:tasks.workunit.client.0.smithi031.stdout:2/250: write d6/f16 [3223321,51930] 0 2024-04-22T23:45:39.721 INFO:tasks.workunit.client.0.smithi031.stdout:2/251: rmdir d6/dc/d42/d24 39 2024-04-22T23:45:39.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/285: dwrite d5/da/d12/d37/d3b/f4b [0,4194304] 0 2024-04-22T23:45:39.725 INFO:tasks.workunit.client.0.smithi031.stdout:6/334: creat d3/d2a/d3c/f73 x:0 0 0 2024-04-22T23:45:39.727 INFO:tasks.workunit.client.0.smithi031.stdout:6/335: chown d3/d2a/d3c/d40/d5b 569556660 1 2024-04-22T23:45:39.729 INFO:tasks.workunit.client.0.smithi031.stdout:9/310: dread d2/dc/f43 [0,4194304] 0 2024-04-22T23:45:39.730 INFO:tasks.workunit.client.0.smithi031.stdout:8/286: mkdir d5/da/d12/d2e/d4d/d56 0 2024-04-22T23:45:39.730 INFO:tasks.workunit.client.0.smithi031.stdout:9/311: chown d2/dc/f10 292208112 1 2024-04-22T23:45:39.732 INFO:tasks.workunit.client.0.smithi031.stdout:9/312: stat d2/dc/f25 0 2024-04-22T23:45:39.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/252: mkdir d6/dc/d51 0 2024-04-22T23:45:39.736 INFO:tasks.workunit.client.0.smithi031.stdout:6/336: write d3/f25 [678156,87278] 0 2024-04-22T23:45:39.736 INFO:tasks.workunit.client.0.smithi031.stdout:5/280: link db/d16/d1d/d24/c4b db/d16/c5c 0 2024-04-22T23:45:39.738 INFO:tasks.workunit.client.0.smithi031.stdout:6/337: write d3/d2a/d3c/f5f [129706,123118] 0 2024-04-22T23:45:39.741 INFO:tasks.workunit.client.0.smithi031.stdout:5/281: mkdir db/d16/d1d/d24/d58/d5d 0 2024-04-22T23:45:39.741 INFO:tasks.workunit.client.0.smithi031.stdout:0/244: dwrite d0/d1/d2/d36/fd [0,4194304] 0 2024-04-22T23:45:39.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/253: mkdir d6/dc/d42/d24/d32/d35/d52 0 2024-04-22T23:45:39.743 INFO:tasks.workunit.client.0.smithi031.stdout:0/245: fdatasync d0/d1/d2/f52 0 2024-04-22T23:45:39.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/254: chown d6 4430 1 2024-04-22T23:45:39.744 INFO:tasks.workunit.client.0.smithi031.stdout:5/282: stat db/d16/d1d/d24/d58/c4c 0 2024-04-22T23:45:39.745 INFO:tasks.workunit.client.0.smithi031.stdout:6/338: creat d3/f74 x:0 0 0 2024-04-22T23:45:39.748 INFO:tasks.workunit.client.0.smithi031.stdout:6/339: stat d3/d1f/l3e 0 2024-04-22T23:45:39.751 INFO:tasks.workunit.client.0.smithi031.stdout:0/246: rename d0/d1/d28/l49 to d0/d1/l5a 0 2024-04-22T23:45:39.752 INFO:tasks.workunit.client.0.smithi031.stdout:2/255: dwrite d6/dc/d42/d24/d32/f3a [0,4194304] 0 2024-04-22T23:45:39.753 INFO:tasks.workunit.client.0.smithi031.stdout:0/247: write d0/f4f [953478,126595] 0 2024-04-22T23:45:39.760 INFO:tasks.workunit.client.0.smithi031.stdout:6/340: mkdir d3/d35/d75 0 2024-04-22T23:45:39.763 INFO:tasks.workunit.client.0.smithi031.stdout:0/248: creat d0/d13/f5b x:0 0 0 2024-04-22T23:45:39.766 INFO:tasks.workunit.client.0.smithi031.stdout:6/341: mknod d3/d2a/d3c/d65/c76 0 2024-04-22T23:45:39.770 INFO:tasks.workunit.client.0.smithi031.stdout:8/287: dread d5/da/d12/d37/f3c [0,4194304] 0 2024-04-22T23:45:39.770 INFO:tasks.workunit.client.0.smithi031.stdout:5/283: truncate db/d16/d1d/f1f 98566 0 2024-04-22T23:45:39.772 INFO:tasks.workunit.client.0.smithi031.stdout:6/342: mkdir d3/d77 0 2024-04-22T23:45:39.778 INFO:tasks.workunit.client.0.smithi031.stdout:5/284: symlink db/l5e 0 2024-04-22T23:45:39.779 INFO:tasks.workunit.client.0.smithi031.stdout:6/343: dwrite d3/d2a/f53 [0,4194304] 0 2024-04-22T23:45:39.781 INFO:tasks.workunit.client.0.smithi031.stdout:8/288: creat d5/da/d12/d2e/f57 x:0 0 0 2024-04-22T23:45:39.782 INFO:tasks.workunit.client.0.smithi031.stdout:2/256: dread d6/f11 [0,4194304] 0 2024-04-22T23:45:39.786 INFO:tasks.workunit.client.0.smithi031.stdout:0/249: dwrite d0/d1/d2/f50 [4194304,4194304] 0 2024-04-22T23:45:39.786 INFO:tasks.workunit.client.0.smithi031.stdout:5/285: creat db/d16/d2a/d59/f5f x:0 0 0 2024-04-22T23:45:39.792 INFO:tasks.workunit.client.0.smithi031.stdout:6/344: creat d3/f78 x:0 0 0 2024-04-22T23:45:39.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/286: fsync db/d16/d1d/d24/f2e 0 2024-04-22T23:45:39.796 INFO:tasks.workunit.client.0.smithi031.stdout:2/257: dwrite d6/dc/d42/d28/f41 [0,4194304] 0 2024-04-22T23:45:39.797 INFO:tasks.workunit.client.0.smithi031.stdout:8/289: dwrite d5/da/d12/d37/d3b/f4b [0,4194304] 0 2024-04-22T23:45:39.798 INFO:tasks.workunit.client.0.smithi031.stdout:0/250: rename d0/d13/d18/l32 to d0/d1/d2/d36/d11/d1c/l5c 0 2024-04-22T23:45:39.799 INFO:tasks.workunit.client.0.smithi031.stdout:7/284: sync 2024-04-22T23:45:39.801 INFO:tasks.workunit.client.0.smithi031.stdout:8/290: dread - d5/da/d12/d37/d3b/f40 zero size 2024-04-22T23:45:39.802 INFO:tasks.workunit.client.0.smithi031.stdout:0/251: write d0/f3d [493695,121947] 0 2024-04-22T23:45:39.804 INFO:tasks.workunit.client.0.smithi031.stdout:2/258: dread d6/f22 [4194304,4194304] 0 2024-04-22T23:45:39.806 INFO:tasks.workunit.client.0.smithi031.stdout:6/345: rename d3/d1f/c48 to d3/d2a/d3c/d65/c79 0 2024-04-22T23:45:39.808 INFO:tasks.workunit.client.0.smithi031.stdout:6/346: fsync d3/d2a/f70 0 2024-04-22T23:45:39.809 INFO:tasks.workunit.client.0.smithi031.stdout:6/347: fsync d3/f25 0 2024-04-22T23:45:39.809 INFO:tasks.workunit.client.0.smithi031.stdout:8/291: dwrite d5/da/df/f43 [0,4194304] 0 2024-04-22T23:45:39.810 INFO:tasks.workunit.client.0.smithi031.stdout:7/285: symlink d9/d4f/l66 0 2024-04-22T23:45:39.811 INFO:tasks.workunit.client.0.smithi031.stdout:8/292: chown d5/da/d12/d37/d4e 2502 1 2024-04-22T23:45:39.813 INFO:tasks.workunit.client.0.smithi031.stdout:8/293: fsync d5/da/d12/d37/d49/f4f 0 2024-04-22T23:45:39.814 INFO:tasks.workunit.client.0.smithi031.stdout:5/287: mknod db/d16/c60 0 2024-04-22T23:45:39.814 INFO:tasks.workunit.client.0.smithi031.stdout:8/294: chown d5/da/d12/d37/d3b/f4b 221995 1 2024-04-22T23:45:39.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/295: read - d5/da/d12/d37/d3b/f40 zero size 2024-04-22T23:45:39.818 INFO:tasks.workunit.client.0.smithi031.stdout:2/259: write d6/f19 [1390611,74179] 0 2024-04-22T23:45:39.823 INFO:tasks.workunit.client.0.smithi031.stdout:0/252: rename d0/d1/d2/d36/d11/d1c/l35 to d0/d13/d18/d1e/d25/d4c/l5d 0 2024-04-22T23:45:39.824 INFO:tasks.workunit.client.0.smithi031.stdout:0/253: write d0/d13/f5b [934740,57362] 0 2024-04-22T23:45:39.830 INFO:tasks.workunit.client.0.smithi031.stdout:5/288: creat db/d16/d2a/d59/f61 x:0 0 0 2024-04-22T23:45:39.832 INFO:tasks.workunit.client.0.smithi031.stdout:8/296: creat d5/da/d12/d2e/d4d/f58 x:0 0 0 2024-04-22T23:45:39.834 INFO:tasks.workunit.client.0.smithi031.stdout:7/286: dread d9/fa [0,4194304] 0 2024-04-22T23:45:39.837 INFO:tasks.workunit.client.0.smithi031.stdout:5/289: dread db/d16/d1d/d24/d58/f4f [0,4194304] 0 2024-04-22T23:45:39.842 INFO:tasks.workunit.client.0.smithi031.stdout:2/260: dwrite d6/dc/d42/f2c [0,4194304] 0 2024-04-22T23:45:39.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/261: chown d6/dc/d42/d24/f34 462160 1 2024-04-22T23:45:39.844 INFO:tasks.workunit.client.0.smithi031.stdout:0/254: unlink d0/d1/d2/d36/d11/d1c/c44 0 2024-04-22T23:45:39.846 INFO:tasks.workunit.client.0.smithi031.stdout:8/297: dread d5/da/f1e [0,4194304] 0 2024-04-22T23:45:39.847 INFO:tasks.workunit.client.0.smithi031.stdout:0/255: dread - d0/d1/d2/d36/d11/d1a/d3c/f4e zero size 2024-04-22T23:45:39.850 INFO:tasks.workunit.client.0.smithi031.stdout:7/287: rename d9/l1e to d9/d4f/d65/l67 0 2024-04-22T23:45:39.857 INFO:tasks.workunit.client.0.smithi031.stdout:5/290: mkdir db/d16/d1d/d24/d32/d62 0 2024-04-22T23:45:39.858 INFO:tasks.workunit.client.0.smithi031.stdout:5/291: read - db/d16/d1d/d24/d58/f51 zero size 2024-04-22T23:45:39.860 INFO:tasks.workunit.client.0.smithi031.stdout:2/262: unlink f4 0 2024-04-22T23:45:39.862 INFO:tasks.workunit.client.0.smithi031.stdout:2/263: write d6/dc/d42/d24/f39 [4785684,26548] 0 2024-04-22T23:45:39.864 INFO:tasks.workunit.client.0.smithi031.stdout:0/256: truncate d0/ff 2847901 0 2024-04-22T23:45:39.865 INFO:tasks.workunit.client.0.smithi031.stdout:5/292: dwrite db/d16/d2a/d59/f5f [0,4194304] 0 2024-04-22T23:45:39.867 INFO:tasks.workunit.client.0.smithi031.stdout:0/257: readlink d0/d1/d2/d36/d11/d1c/l46 0 2024-04-22T23:45:39.871 INFO:tasks.workunit.client.0.smithi031.stdout:2/264: dread d6/ff [0,4194304] 0 2024-04-22T23:45:39.871 INFO:tasks.workunit.client.0.smithi031.stdout:6/348: truncate d3/fe 1317315 0 2024-04-22T23:45:39.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/349: write d3/f25 [353871,108255] 0 2024-04-22T23:45:39.874 INFO:tasks.workunit.client.0.smithi031.stdout:7/288: dwrite d9/d14/d22/d28/f36 [0,4194304] 0 2024-04-22T23:45:39.878 INFO:tasks.workunit.client.0.smithi031.stdout:0/258: mknod d0/d1/d28/d2b/c5e 0 2024-04-22T23:45:39.883 INFO:tasks.workunit.client.0.smithi031.stdout:8/298: creat d5/da/f59 x:0 0 0 2024-04-22T23:45:39.883 INFO:tasks.workunit.client.0.smithi031.stdout:7/289: dwrite d9/d15/d3a/d41/f5a [0,4194304] 0 2024-04-22T23:45:39.887 INFO:tasks.workunit.client.0.smithi031.stdout:0/259: dwrite d0/d13/d18/d1e/f30 [0,4194304] 0 2024-04-22T23:45:39.888 INFO:tasks.workunit.client.0.smithi031.stdout:6/350: mkdir d3/d2a/d3c/d40/d7a 0 2024-04-22T23:45:39.889 INFO:tasks.workunit.client.0.smithi031.stdout:0/260: write d0/d1/d2/f23 [440852,37375] 0 2024-04-22T23:45:39.889 INFO:tasks.workunit.client.0.smithi031.stdout:6/351: chown d3/d2a/f70 268300065 1 2024-04-22T23:45:39.890 INFO:tasks.workunit.client.0.smithi031.stdout:0/261: fdatasync d0/d1/d2/f52 0 2024-04-22T23:45:39.894 INFO:tasks.workunit.client.0.smithi031.stdout:4/409: sync 2024-04-22T23:45:39.894 INFO:tasks.workunit.client.0.smithi031.stdout:3/237: sync 2024-04-22T23:45:39.928 INFO:tasks.workunit.client.0.smithi031.stdout:5/293: write db/d16/d2a/f2c [1190411,108364] 0 2024-04-22T23:45:39.933 INFO:tasks.workunit.client.0.smithi031.stdout:6/352: creat d3/d2a/d3c/d5e/f7b x:0 0 0 2024-04-22T23:45:39.935 INFO:tasks.workunit.client.0.smithi031.stdout:0/262: dwrite d0/d13/d18/d1e/f30 [0,4194304] 0 2024-04-22T23:45:39.938 INFO:tasks.workunit.client.0.smithi031.stdout:0/263: write d0/d13/f5b [551117,99760] 0 2024-04-22T23:45:39.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/238: dwrite f3 [0,4194304] 0 2024-04-22T23:45:39.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/294: creat db/d16/d1d/d24/f63 x:0 0 0 2024-04-22T23:45:39.943 INFO:tasks.workunit.client.0.smithi031.stdout:5/295: readlink db/l5e 0 2024-04-22T23:45:39.944 INFO:tasks.workunit.client.0.smithi031.stdout:2/265: getdents d6 0 2024-04-22T23:45:39.944 INFO:tasks.workunit.client.0.smithi031.stdout:3/239: chown d4/d6/d11/c47 867 1 2024-04-22T23:45:39.944 INFO:tasks.workunit.client.0.smithi031.stdout:8/299: dwrite d5/da/d12/d37/d49/f54 [0,4194304] 0 2024-04-22T23:45:39.945 INFO:tasks.workunit.client.0.smithi031.stdout:5/296: write db/d16/f54 [1610105,77082] 0 2024-04-22T23:45:39.947 INFO:tasks.workunit.client.0.smithi031.stdout:5/297: chown db/d16/d2a/f2c 7 1 2024-04-22T23:45:39.951 INFO:tasks.workunit.client.0.smithi031.stdout:4/410: creat db/d14/d2d/d49/d59/d6a/d63/d86/f8c x:0 0 0 2024-04-22T23:45:39.955 INFO:tasks.workunit.client.0.smithi031.stdout:2/266: chown d6/dc/d17 3595 1 2024-04-22T23:45:39.956 INFO:tasks.workunit.client.0.smithi031.stdout:2/267: fsync d6/dc/d17/f3c 0 2024-04-22T23:45:39.957 INFO:tasks.workunit.client.0.smithi031.stdout:5/298: dwrite db/d16/f3e [0,4194304] 0 2024-04-22T23:45:39.962 INFO:tasks.workunit.client.0.smithi031.stdout:6/353: unlink d3/c19 0 2024-04-22T23:45:39.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/268: dread d6/dc/d42/f2c [0,4194304] 0 2024-04-22T23:45:39.964 INFO:tasks.workunit.client.0.smithi031.stdout:2/269: chown d6/dc/d42/d24/d32/d35 232611 1 2024-04-22T23:45:39.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/300: rename d5/da/f20 to d5/da/d12/d2e/d4d/d56/f5a 0 2024-04-22T23:45:39.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/411: write db/d14/d1d/d7b/d7f/d2a/d30/f60 [3527589,110254] 0 2024-04-22T23:45:39.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/412: stat db/fc 0 2024-04-22T23:45:39.971 INFO:tasks.workunit.client.0.smithi031.stdout:3/240: mkdir d4/d49 0 2024-04-22T23:45:39.972 INFO:tasks.workunit.client.0.smithi031.stdout:3/241: write f3 [1857494,128238] 0 2024-04-22T23:45:39.974 INFO:tasks.workunit.client.0.smithi031.stdout:5/299: symlink db/d16/d1d/d24/d27/l64 0 2024-04-22T23:45:39.978 INFO:tasks.workunit.client.0.smithi031.stdout:6/354: symlink d3/d2a/d3c/d40/d5b/l7c 0 2024-04-22T23:45:39.983 INFO:tasks.workunit.client.0.smithi031.stdout:4/413: rmdir db/d31/d4f 39 2024-04-22T23:45:39.994 INFO:tasks.workunit.client.0.smithi031.stdout:2/270: dread d6/f19 [0,4194304] 0 2024-04-22T23:45:39.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/414: symlink db/d14/d2d/d49/d59/l8d 0 2024-04-22T23:45:39.999 INFO:tasks.workunit.client.0.smithi031.stdout:4/415: write db/d31/f33 [4425006,40725] 0 2024-04-22T23:45:40.000 INFO:tasks.workunit.client.0.smithi031.stdout:4/416: read - db/d14/d2d/d49/f53 zero size 2024-04-22T23:45:40.000 INFO:tasks.workunit.client.0.smithi031.stdout:3/242: mkdir d4/d6/d11/d3b/d4a 0 2024-04-22T23:45:40.002 INFO:tasks.workunit.client.0.smithi031.stdout:4/417: rename db/d14/d1d/d7b/d7f to db/d14/d1d/d7b/d7f/d2a/d8e 22 2024-04-22T23:45:40.012 INFO:tasks.workunit.client.0.smithi031.stdout:0/264: truncate d0/d1/d2/f50 5553193 0 2024-04-22T23:45:40.015 INFO:tasks.workunit.client.0.smithi031.stdout:5/300: creat db/d16/f65 x:0 0 0 2024-04-22T23:45:40.017 INFO:tasks.workunit.client.0.smithi031.stdout:4/418: rename db/d14/d1d/d7b/d7f/d2a/d47/f54 to db/d14/d2d/d49/d59/d8a/d89/f8f 0 2024-04-22T23:45:40.018 INFO:tasks.workunit.client.0.smithi031.stdout:0/265: dwrite d0/d13/d18/f39 [0,4194304] 0 2024-04-22T23:45:40.019 INFO:tasks.workunit.client.0.smithi031.stdout:4/419: write db/d31/d40/d62/f77 [444234,101374] 0 2024-04-22T23:45:40.020 INFO:tasks.workunit.client.0.smithi031.stdout:4/420: chown db/d14/d2d/d49/d59/d8a 18773 1 2024-04-22T23:45:40.024 INFO:tasks.workunit.client.0.smithi031.stdout:1/224: sync 2024-04-22T23:45:40.024 INFO:tasks.workunit.client.0.smithi031.stdout:9/313: sync 2024-04-22T23:45:40.025 INFO:tasks.workunit.client.0.smithi031.stdout:9/314: dread - d2/d47/d4a/f27 zero size 2024-04-22T23:45:40.026 INFO:tasks.workunit.client.0.smithi031.stdout:1/225: dread d1/db/d16/d1f/f26 [0,4194304] 0 2024-04-22T23:45:40.027 INFO:tasks.workunit.client.0.smithi031.stdout:1/226: chown d1/d35 419682071 1 2024-04-22T23:45:40.028 INFO:tasks.workunit.client.0.smithi031.stdout:1/227: chown d1/de/f11 7245 1 2024-04-22T23:45:40.030 INFO:tasks.workunit.client.0.smithi031.stdout:8/301: dwrite d5/f26 [0,4194304] 0 2024-04-22T23:45:40.031 INFO:tasks.workunit.client.0.smithi031.stdout:8/302: readlink d5/da/d12/d37/l45 0 2024-04-22T23:45:40.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/271: rmdir d6/dc/d51 0 2024-04-22T23:45:40.035 INFO:tasks.workunit.client.0.smithi031.stdout:6/355: write d3/f45 [4901399,37793] 0 2024-04-22T23:45:40.037 INFO:tasks.workunit.client.0.smithi031.stdout:4/421: symlink db/d31/d40/d43/l90 0 2024-04-22T23:45:40.039 INFO:tasks.workunit.client.0.smithi031.stdout:3/243: link d4/d6/d11/d3e/c41 d4/d10/d2e/c4b 0 2024-04-22T23:45:40.039 INFO:tasks.workunit.client.0.smithi031.stdout:3/244: truncate f3 4315064 0 2024-04-22T23:45:40.044 INFO:tasks.workunit.client.0.smithi031.stdout:9/315: mkdir d2/d15/d5a 0 2024-04-22T23:45:40.047 INFO:tasks.workunit.client.0.smithi031.stdout:5/301: mknod db/d16/d1d/d24/d58/d5d/c66 0 2024-04-22T23:45:40.051 INFO:tasks.workunit.client.0.smithi031.stdout:0/266: unlink d0/d1/d2/d36/d11/d1c/l5c 0 2024-04-22T23:45:40.053 INFO:tasks.workunit.client.0.smithi031.stdout:8/303: mkdir d5/da/d12/d37/d3b/d5b 0 2024-04-22T23:45:40.055 INFO:tasks.workunit.client.0.smithi031.stdout:6/356: chown d3/d1f/c4b 23889 1 2024-04-22T23:45:40.056 INFO:tasks.workunit.client.0.smithi031.stdout:6/357: chown d3/d2a/d3c/d40/d4d/f69 8 1 2024-04-22T23:45:40.057 INFO:tasks.workunit.client.0.smithi031.stdout:2/272: symlink d6/dc/d42/d24/l53 0 2024-04-22T23:45:40.059 INFO:tasks.workunit.client.0.smithi031.stdout:6/358: truncate d3/d2a/d3c/d40/d4d/f61 1042752 0 2024-04-22T23:45:40.060 INFO:tasks.workunit.client.0.smithi031.stdout:3/245: truncate d4/d6/da/f29 979592 0 2024-04-22T23:45:40.061 INFO:tasks.workunit.client.0.smithi031.stdout:3/246: dread d4/d10/f1b [0,4194304] 0 2024-04-22T23:45:40.062 INFO:tasks.workunit.client.0.smithi031.stdout:1/228: symlink d1/l51 0 2024-04-22T23:45:40.065 INFO:tasks.workunit.client.0.smithi031.stdout:0/267: mkdir d0/d1/d2/d36/d11/d1c/d5f 0 2024-04-22T23:45:40.073 INFO:tasks.workunit.client.0.smithi031.stdout:7/290: sync 2024-04-22T23:45:40.098 INFO:tasks.workunit.client.0.smithi031.stdout:4/422: symlink db/d31/l91 0 2024-04-22T23:45:40.099 INFO:tasks.workunit.client.0.smithi031.stdout:4/423: dread db/d14/d1d/d7b/d7f/d2a/d30/f42 [0,4194304] 0 2024-04-22T23:45:40.103 INFO:tasks.workunit.client.0.smithi031.stdout:9/316: symlink d2/d15/d5a/l5b 0 2024-04-22T23:45:40.104 INFO:tasks.workunit.client.0.smithi031.stdout:1/229: mkdir d1/db/d16/d1f/d52 0 2024-04-22T23:45:40.106 INFO:tasks.workunit.client.0.smithi031.stdout:4/424: dwrite db/d14/d2d/f35 [0,4194304] 0 2024-04-22T23:45:40.107 INFO:tasks.workunit.client.0.smithi031.stdout:4/425: write db/d31/d40/d62/f77 [1356955,80006] 0 2024-04-22T23:45:40.109 INFO:tasks.workunit.client.0.smithi031.stdout:0/268: mknod d0/d1/d2/d36/d11/d1a/d3c/c60 0 2024-04-22T23:45:40.110 INFO:tasks.workunit.client.0.smithi031.stdout:8/304: link d5/da/d12/d2e/d4d/f58 d5/da/df/d33/f5c 0 2024-04-22T23:45:40.111 INFO:tasks.workunit.client.0.smithi031.stdout:8/305: chown d5/da/d12/d37 8015 1 2024-04-22T23:45:40.122 INFO:tasks.workunit.client.0.smithi031.stdout:1/230: fsync d1/db/d16/d1f/f26 0 2024-04-22T23:45:40.124 INFO:tasks.workunit.client.0.smithi031.stdout:5/302: write db/d16/f2b [4090972,112162] 0 2024-04-22T23:45:40.126 INFO:tasks.workunit.client.0.smithi031.stdout:4/426: readlink db/d14/d2d/d3c/l70 0 2024-04-22T23:45:40.127 INFO:tasks.workunit.client.0.smithi031.stdout:1/231: dread d1/f46 [0,4194304] 0 2024-04-22T23:45:40.132 INFO:tasks.workunit.client.0.smithi031.stdout:8/306: mknod d5/d48/c5d 0 2024-04-22T23:45:40.132 INFO:tasks.workunit.client.0.smithi031.stdout:8/307: write d5/f9 [4601249,30045] 0 2024-04-22T23:45:40.138 INFO:tasks.workunit.client.0.smithi031.stdout:8/308: dwrite d5/da/df/f1b [4194304,4194304] 0 2024-04-22T23:45:40.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/247: rename d4/d6/d11/d3b to d4/d6/d4c 0 2024-04-22T23:45:40.149 INFO:tasks.workunit.client.0.smithi031.stdout:1/232: mknod d1/de/d1e/d34/c53 0 2024-04-22T23:45:40.156 INFO:tasks.workunit.client.0.smithi031.stdout:9/317: creat d2/dc/d21/d3d/d40/f5c x:0 0 0 2024-04-22T23:45:40.159 INFO:tasks.workunit.client.0.smithi031.stdout:8/309: creat d5/da/d12/d37/d34/f5e x:0 0 0 2024-04-22T23:45:40.163 INFO:tasks.workunit.client.0.smithi031.stdout:3/248: mkdir d4/d6/d4c/d4d 0 2024-04-22T23:45:40.165 INFO:tasks.workunit.client.0.smithi031.stdout:3/249: chown d4/d6/d11/d3e/f43 214183062 1 2024-04-22T23:45:40.167 INFO:tasks.workunit.client.0.smithi031.stdout:8/310: dwrite d5/da/f2c [4194304,4194304] 0 2024-04-22T23:45:40.168 INFO:tasks.workunit.client.0.smithi031.stdout:8/311: read d5/f26 [1326171,54982] 0 2024-04-22T23:45:40.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/312: chown d5/da/d12/d37/d34/f4c 138 1 2024-04-22T23:45:40.176 INFO:tasks.workunit.client.0.smithi031.stdout:7/291: rename d9/d15/c1d to d9/d14/c68 0 2024-04-22T23:45:40.180 INFO:tasks.workunit.client.0.smithi031.stdout:3/250: mknod d4/d6/d4c/c4e 0 2024-04-22T23:45:40.181 INFO:tasks.workunit.client.0.smithi031.stdout:3/251: chown d4/d10/d2a/c40 0 1 2024-04-22T23:45:40.183 INFO:tasks.workunit.client.0.smithi031.stdout:8/313: creat d5/d48/f5f x:0 0 0 2024-04-22T23:45:40.184 INFO:tasks.workunit.client.0.smithi031.stdout:8/314: read d5/da/df/f43 [3779807,5137] 0 2024-04-22T23:45:40.185 INFO:tasks.workunit.client.0.smithi031.stdout:1/233: mkdir d1/d3f/d54 0 2024-04-22T23:45:40.188 INFO:tasks.workunit.client.0.smithi031.stdout:9/318: rename d2/dc/d21/d38 to d2/dc/d21/d3d/d40/d2a/d5d 0 2024-04-22T23:45:40.197 INFO:tasks.workunit.client.0.smithi031.stdout:2/273: truncate d6/dc/d42/d24/d32/f3a 935082 0 2024-04-22T23:45:40.202 INFO:tasks.workunit.client.0.smithi031.stdout:9/319: creat d2/d47/f5e x:0 0 0 2024-04-22T23:45:40.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/320: write d2/d47/d4a/f34 [2939399,26615] 0 2024-04-22T23:45:40.203 INFO:tasks.workunit.client.0.smithi031.stdout:7/292: creat d9/d14/d1f/d47/f69 x:0 0 0 2024-04-22T23:45:40.213 INFO:tasks.workunit.client.0.smithi031.stdout:9/321: dwrite d2/dd/f13 [0,4194304] 0 2024-04-22T23:45:40.213 INFO:tasks.workunit.client.0.smithi031.stdout:2/274: rename d6/dc/d42/f2e to d6/dc/d42/d28/f54 0 2024-04-22T23:45:40.213 INFO:tasks.workunit.client.0.smithi031.stdout:7/293: dwrite d9/d14/d22/f2d [0,4194304] 0 2024-04-22T23:45:40.218 INFO:tasks.workunit.client.0.smithi031.stdout:3/252: fsync d4/d6/da/f29 0 2024-04-22T23:45:40.219 INFO:tasks.workunit.client.0.smithi031.stdout:6/359: truncate d3/f38 1619110 0 2024-04-22T23:45:40.220 INFO:tasks.workunit.client.0.smithi031.stdout:3/253: write d4/d6/d11/f3c [716685,10469] 0 2024-04-22T23:45:40.221 INFO:tasks.workunit.client.0.smithi031.stdout:6/360: write d3/d2a/d3c/d40/f57 [145511,18807] 0 2024-04-22T23:45:40.222 INFO:tasks.workunit.client.0.smithi031.stdout:2/275: creat d6/dc/d48/f55 x:0 0 0 2024-04-22T23:45:40.227 INFO:tasks.workunit.client.0.smithi031.stdout:6/361: chown d3/c5 630141 1 2024-04-22T23:45:40.229 INFO:tasks.workunit.client.0.smithi031.stdout:2/276: dread - d6/dc/d42/f43 zero size 2024-04-22T23:45:40.231 INFO:tasks.workunit.client.0.smithi031.stdout:4/427: write db/d14/d2d/d49/d59/d6a/d41/f4b [4374127,38379] 0 2024-04-22T23:45:40.232 INFO:tasks.workunit.client.0.smithi031.stdout:6/362: mkdir d3/d35/d75/d7d 0 2024-04-22T23:45:40.233 INFO:tasks.workunit.client.0.smithi031.stdout:3/254: dread d4/f1e [4194304,4194304] 0 2024-04-22T23:45:40.234 INFO:tasks.workunit.client.0.smithi031.stdout:2/277: read d6/dc/d42/f2c [1713441,79576] 0 2024-04-22T23:45:40.234 INFO:tasks.workunit.client.0.smithi031.stdout:6/363: write d3/d1f/f5c [2157550,24030] 0 2024-04-22T23:45:40.235 INFO:tasks.workunit.client.0.smithi031.stdout:0/269: creat d0/f61 x:0 0 0 2024-04-22T23:45:40.236 INFO:tasks.workunit.client.0.smithi031.stdout:0/270: stat d0/c57 0 2024-04-22T23:45:40.237 INFO:tasks.workunit.client.0.smithi031.stdout:0/271: chown d0/d13/d18/d1e/d25 1057686855 1 2024-04-22T23:45:40.238 INFO:tasks.workunit.client.0.smithi031.stdout:4/428: dread db/d14/d1d/f25 [0,4194304] 0 2024-04-22T23:45:40.239 INFO:tasks.workunit.client.0.smithi031.stdout:3/255: creat d4/d10/d2e/f4f x:0 0 0 2024-04-22T23:45:40.240 INFO:tasks.workunit.client.0.smithi031.stdout:2/278: stat d6/f22 0 2024-04-22T23:45:40.240 INFO:tasks.workunit.client.0.smithi031.stdout:4/429: fdatasync db/d14/f45 0 2024-04-22T23:45:40.240 INFO:tasks.workunit.client.0.smithi031.stdout:6/364: readlink d3/l1c 0 2024-04-22T23:45:40.242 INFO:tasks.workunit.client.0.smithi031.stdout:3/256: chown d4/d10/d2e/l2f 191 1 2024-04-22T23:45:40.244 INFO:tasks.workunit.client.0.smithi031.stdout:0/272: truncate d0/d1/d2/d36/d11/d1a/d3c/f3e 9453 0 2024-04-22T23:45:40.244 INFO:tasks.workunit.client.0.smithi031.stdout:2/279: unlink d6/f11 0 2024-04-22T23:45:40.245 INFO:tasks.workunit.client.0.smithi031.stdout:6/365: mknod d3/d2a/d3c/d40/d5b/c7e 0 2024-04-22T23:45:40.245 INFO:tasks.workunit.client.0.smithi031.stdout:3/257: write d4/d6/d11/f1c [2494098,46409] 0 2024-04-22T23:45:40.247 INFO:tasks.workunit.client.0.smithi031.stdout:6/366: fdatasync d3/d2a/d3c/d40/f4e 0 2024-04-22T23:45:40.247 INFO:tasks.workunit.client.0.smithi031.stdout:9/322: dread d2/dc/d3e/f4f [0,4194304] 0 2024-04-22T23:45:40.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/258: fdatasync d4/d6/da/f13 0 2024-04-22T23:45:40.248 INFO:tasks.workunit.client.0.smithi031.stdout:4/430: rename f4 to db/d14/d2d/d49/d59/d6a/f92 0 2024-04-22T23:45:40.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/273: symlink d0/l62 0 2024-04-22T23:45:40.255 INFO:tasks.workunit.client.0.smithi031.stdout:5/303: truncate db/d16/f15 470213 0 2024-04-22T23:45:40.256 INFO:tasks.workunit.client.0.smithi031.stdout:3/259: dwrite d4/d6/da/ff [0,4194304] 0 2024-04-22T23:45:40.259 INFO:tasks.workunit.client.0.smithi031.stdout:4/431: mknod db/d14/d2d/d49/d59/d6a/d63/c93 0 2024-04-22T23:45:40.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/280: rename d6/c2f to d6/dc/d42/d28/c56 0 2024-04-22T23:45:40.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/281: stat d6/f22 0 2024-04-22T23:45:40.267 INFO:tasks.workunit.client.0.smithi031.stdout:4/432: dwrite db/d31/f33 [4194304,4194304] 0 2024-04-22T23:45:40.270 INFO:tasks.workunit.client.0.smithi031.stdout:5/304: creat db/d16/d1d/d24/d32/d62/f67 x:0 0 0 2024-04-22T23:45:40.270 INFO:tasks.workunit.client.0.smithi031.stdout:4/433: write db/d14/d2d/d49/d59/d6a/d41/f7d [424191,71954] 0 2024-04-22T23:45:40.271 INFO:tasks.workunit.client.0.smithi031.stdout:5/305: stat db/le 0 2024-04-22T23:45:40.273 INFO:tasks.workunit.client.0.smithi031.stdout:6/367: getdents d3/d2a 0 2024-04-22T23:45:40.276 INFO:tasks.workunit.client.0.smithi031.stdout:4/434: mknod db/d14/d1d/d7b/d7f/d2a/d30/c94 0 2024-04-22T23:45:40.277 INFO:tasks.workunit.client.0.smithi031.stdout:9/323: rmdir d2/dc/d21/d3d/d40/d2a/d5d 39 2024-04-22T23:45:40.282 INFO:tasks.workunit.client.0.smithi031.stdout:4/435: dwrite db/d14/d1d/d7b/d7f/d2a/d30/f5e [0,4194304] 0 2024-04-22T23:45:40.284 INFO:tasks.workunit.client.0.smithi031.stdout:6/368: symlink d3/d35/d75/d7d/l7f 0 2024-04-22T23:45:40.288 INFO:tasks.workunit.client.0.smithi031.stdout:0/274: rename d0/d1/c3a to d0/d1/d2/c63 0 2024-04-22T23:45:40.289 INFO:tasks.workunit.client.0.smithi031.stdout:0/275: chown d0/d1/d3f 7179 1 2024-04-22T23:45:40.292 INFO:tasks.workunit.client.0.smithi031.stdout:4/436: truncate db/d14/f45 554092 0 2024-04-22T23:45:40.292 INFO:tasks.workunit.client.0.smithi031.stdout:1/234: dwrite d1/db/d16/f40 [0,4194304] 0 2024-04-22T23:45:40.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/369: mknod d3/d2a/d3c/d65/c80 0 2024-04-22T23:45:40.294 INFO:tasks.workunit.client.0.smithi031.stdout:6/370: fdatasync d3/f18 0 2024-04-22T23:45:40.298 INFO:tasks.workunit.client.0.smithi031.stdout:3/260: rename d4/d6/da/f35 to d4/d6/d11/d25/d44/f50 0 2024-04-22T23:45:40.301 INFO:tasks.workunit.client.0.smithi031.stdout:8/315: truncate d5/da/d12/f14 504252 0 2024-04-22T23:45:40.303 INFO:tasks.workunit.client.0.smithi031.stdout:8/316: write d5/da/d12/d37/d34/f46 [1207818,42065] 0 2024-04-22T23:45:40.303 INFO:tasks.workunit.client.0.smithi031.stdout:5/306: mknod db/c68 0 2024-04-22T23:45:40.305 INFO:tasks.workunit.client.0.smithi031.stdout:5/307: write db/d16/f3e [5173511,56093] 0 2024-04-22T23:45:40.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/317: truncate d5/da/d12/d37/d34/f4c 187971 0 2024-04-22T23:45:40.307 INFO:tasks.workunit.client.0.smithi031.stdout:8/318: dread - d5/da/d12/d2e/d4d/f58 zero size 2024-04-22T23:45:40.307 INFO:tasks.workunit.client.0.smithi031.stdout:1/235: dread d1/f1d [0,4194304] 0 2024-04-22T23:45:40.308 INFO:tasks.workunit.client.0.smithi031.stdout:8/319: readlink d5/da/d12/d1d/l22 0 2024-04-22T23:45:40.309 INFO:tasks.workunit.client.0.smithi031.stdout:8/320: write d5/da/d12/d1d/d38/f3f [4063914,43792] 0 2024-04-22T23:45:40.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/294: write d9/d15/d3a/f51 [699846,92647] 0 2024-04-22T23:45:40.316 INFO:tasks.workunit.client.0.smithi031.stdout:4/437: creat db/d14/d1d/d7b/d7f/d2a/d47/d80/f95 x:0 0 0 2024-04-22T23:45:40.317 INFO:tasks.workunit.client.0.smithi031.stdout:5/308: symlink db/d16/d2a/d36/l69 0 2024-04-22T23:45:40.317 INFO:tasks.workunit.client.0.smithi031.stdout:4/438: dread - db/d14/d1d/d7b/d7f/d2a/d48/f5d zero size 2024-04-22T23:45:40.322 INFO:tasks.workunit.client.0.smithi031.stdout:1/236: rmdir d1/d2e 39 2024-04-22T23:45:40.325 INFO:tasks.workunit.client.0.smithi031.stdout:6/371: fdatasync d3/f8 0 2024-04-22T23:45:40.326 INFO:tasks.workunit.client.0.smithi031.stdout:6/372: dread - d3/d35/f59 zero size 2024-04-22T23:45:40.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/373: dread d3/f25 [0,4194304] 0 2024-04-22T23:45:40.333 INFO:tasks.workunit.client.0.smithi031.stdout:4/439: mknod db/d14/d1d/d7b/d7f/c96 0 2024-04-22T23:45:40.334 INFO:tasks.workunit.client.0.smithi031.stdout:5/309: truncate db/d16/d1d/d24/f2e 3519954 0 2024-04-22T23:45:40.341 INFO:tasks.workunit.client.0.smithi031.stdout:2/282: rename d6/dc/d42/d24/d32/d35 to d6/dc/d57 0 2024-04-22T23:45:40.348 INFO:tasks.workunit.client.0.smithi031.stdout:1/237: getdents d1/db/d16/d1f/d52 0 2024-04-22T23:45:40.350 INFO:tasks.workunit.client.0.smithi031.stdout:1/238: read d1/d21/d29/f1a [3665226,116316] 0 2024-04-22T23:45:40.355 INFO:tasks.workunit.client.0.smithi031.stdout:9/324: truncate d2/dc/d21/d3d/d40/d2a/f55 4621315 0 2024-04-22T23:45:40.362 INFO:tasks.workunit.client.0.smithi031.stdout:0/276: rename d0/d1/d2/f50 to d0/d1/d28/d2b/f64 0 2024-04-22T23:45:40.363 INFO:tasks.workunit.client.0.smithi031.stdout:0/277: dread d0/d1/d2/d36/d11/f43 [0,4194304] 0 2024-04-22T23:45:40.369 INFO:tasks.workunit.client.0.smithi031.stdout:1/239: getdents d1/d35/d36 0 2024-04-22T23:45:40.370 INFO:tasks.workunit.client.0.smithi031.stdout:1/240: write d1/de/f11 [799075,7203] 0 2024-04-22T23:45:40.373 INFO:tasks.workunit.client.0.smithi031.stdout:2/283: creat d6/d45/d50/f58 x:0 0 0 2024-04-22T23:45:40.375 INFO:tasks.workunit.client.0.smithi031.stdout:5/310: link db/f3f db/d16/d1d/f6a 0 2024-04-22T23:45:40.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/278: dread d0/d1/d2/d36/d11/f14 [0,4194304] 0 2024-04-22T23:45:40.383 INFO:tasks.workunit.client.0.smithi031.stdout:2/284: write d6/f19 [3284265,33013] 0 2024-04-22T23:45:40.384 INFO:tasks.workunit.client.0.smithi031.stdout:9/325: creat d2/dc/d21/d3d/d40/d2a/f5f x:0 0 0 2024-04-22T23:45:40.386 INFO:tasks.workunit.client.0.smithi031.stdout:3/261: rename d4/d6/l30 to d4/d10/l51 0 2024-04-22T23:45:40.386 INFO:tasks.workunit.client.0.smithi031.stdout:9/326: read d2/dd/f2b [2186385,84096] 0 2024-04-22T23:45:40.387 INFO:tasks.workunit.client.0.smithi031.stdout:1/241: dread d1/d21/d29/f1b [0,4194304] 0 2024-04-22T23:45:40.387 INFO:tasks.workunit.client.0.smithi031.stdout:7/295: write f2 [1647651,54674] 0 2024-04-22T23:45:40.390 INFO:tasks.workunit.client.0.smithi031.stdout:9/327: truncate d2/dc/d3e/f41 5125293 0 2024-04-22T23:45:40.394 INFO:tasks.workunit.client.0.smithi031.stdout:3/262: mknod d4/d6/d22/c52 0 2024-04-22T23:45:40.398 INFO:tasks.workunit.client.0.smithi031.stdout:2/285: mkdir d6/d59 0 2024-04-22T23:45:40.398 INFO:tasks.workunit.client.0.smithi031.stdout:2/286: stat d6/dc 0 2024-04-22T23:45:40.399 INFO:tasks.workunit.client.0.smithi031.stdout:8/321: truncate d5/da/d12/d37/d34/f46 939884 0 2024-04-22T23:45:40.401 INFO:tasks.workunit.client.0.smithi031.stdout:6/374: write d3/f15 [4046316,82219] 0 2024-04-22T23:45:40.403 INFO:tasks.workunit.client.0.smithi031.stdout:9/328: unlink d2/d47/d4a/c42 0 2024-04-22T23:45:40.408 INFO:tasks.workunit.client.0.smithi031.stdout:3/263: mknod d4/d6/da/c53 0 2024-04-22T23:45:40.409 INFO:tasks.workunit.client.0.smithi031.stdout:0/279: link d0/l62 d0/d1/d2/l65 0 2024-04-22T23:45:40.410 INFO:tasks.workunit.client.0.smithi031.stdout:2/287: rmdir d6/dc/d57 39 2024-04-22T23:45:40.410 INFO:tasks.workunit.client.0.smithi031.stdout:0/280: stat d0/d1/d2/f23 0 2024-04-22T23:45:40.411 INFO:tasks.workunit.client.0.smithi031.stdout:0/281: stat d0/d1/d28 0 2024-04-22T23:45:40.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/375: readlink d3/l10 0 2024-04-22T23:45:40.416 INFO:tasks.workunit.client.0.smithi031.stdout:4/440: rename l8 to db/l97 0 2024-04-22T23:45:40.422 INFO:tasks.workunit.client.0.smithi031.stdout:0/282: mknod d0/d1/d2/d36/c66 0 2024-04-22T23:45:40.423 INFO:tasks.workunit.client.0.smithi031.stdout:0/283: dread d0/d1/d2/d36/f2f [0,4194304] 0 2024-04-22T23:45:40.428 INFO:tasks.workunit.client.0.smithi031.stdout:7/296: rename d9/d14/d22/d28/c44 to d9/d14/d22/c6a 0 2024-04-22T23:45:40.429 INFO:tasks.workunit.client.0.smithi031.stdout:7/297: write d9/d15/d3a/d41/f54 [850245,68333] 0 2024-04-22T23:45:40.429 INFO:tasks.workunit.client.0.smithi031.stdout:4/441: mkdir db/d14/d2d/d49/d59/d98 0 2024-04-22T23:45:40.440 INFO:tasks.workunit.client.0.smithi031.stdout:4/442: creat db/d14/d2d/d49/f99 x:0 0 0 2024-04-22T23:45:40.442 INFO:tasks.workunit.client.0.smithi031.stdout:2/288: mknod d6/dc/d57/d52/c5a 0 2024-04-22T23:45:40.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:39 smithi031 ceph-mon[26721]: pgmap v26: 65 pgs: 65 active+clean; 293 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-22T23:45:40.445 INFO:tasks.workunit.client.0.smithi031.stdout:1/242: dwrite d1/db/d16/d1f/f26 [0,4194304] 0 2024-04-22T23:45:40.447 INFO:tasks.workunit.client.0.smithi031.stdout:1/243: truncate d1/de/d1e/d34/d41/f4c 1248953 0 2024-04-22T23:45:40.448 INFO:tasks.workunit.client.0.smithi031.stdout:6/376: rmdir d3/d2a/d3c/d65/d6c 0 2024-04-22T23:45:40.449 INFO:tasks.workunit.client.0.smithi031.stdout:1/244: fsync d1/db/d16/f40 0 2024-04-22T23:45:40.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/377: write d3/d2a/d3c/d40/d4d/f61 [1335148,114981] 0 2024-04-22T23:45:40.454 INFO:tasks.workunit.client.0.smithi031.stdout:2/289: creat d6/d45/d50/f5b x:0 0 0 2024-04-22T23:45:40.456 INFO:tasks.workunit.client.0.smithi031.stdout:0/284: creat d0/d13/f67 x:0 0 0 2024-04-22T23:45:40.458 INFO:tasks.workunit.client.0.smithi031.stdout:3/264: rename d4/d10/d2a to d4/d54 0 2024-04-22T23:45:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:39 smithi044 ceph-mon[34463]: pgmap v26: 65 pgs: 65 active+clean; 293 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 18 MiB/s wr, 225 op/s 2024-04-22T23:45:40.460 INFO:tasks.workunit.client.0.smithi031.stdout:8/322: dwrite d5/da/f1e [0,4194304] 0 2024-04-22T23:45:40.462 INFO:tasks.workunit.client.0.smithi031.stdout:9/329: write d2/dc/d21/d3d/d40/f44 [708873,65453] 0 2024-04-22T23:45:40.470 INFO:tasks.workunit.client.0.smithi031.stdout:0/285: creat d0/d13/d18/d1e/f68 x:0 0 0 2024-04-22T23:45:40.475 INFO:tasks.workunit.client.0.smithi031.stdout:4/443: rename db/d14/l28 to db/d14/d2d/d49/d59/d6a/d63/d86/l9a 0 2024-04-22T23:45:40.475 INFO:tasks.workunit.client.0.smithi031.stdout:8/323: rmdir d5/da/d12/d1d 39 2024-04-22T23:45:40.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/245: symlink d1/l55 0 2024-04-22T23:45:40.480 INFO:tasks.workunit.client.0.smithi031.stdout:7/298: dwrite d9/d15/f21 [0,4194304] 0 2024-04-22T23:45:40.482 INFO:tasks.workunit.client.0.smithi031.stdout:3/265: mkdir d4/d6/d55 0 2024-04-22T23:45:40.485 INFO:tasks.workunit.client.0.smithi031.stdout:8/324: symlink d5/da/df/d33/l60 0 2024-04-22T23:45:40.486 INFO:tasks.workunit.client.0.smithi031.stdout:3/266: chown d4/d6/da 367303465 1 2024-04-22T23:45:40.490 INFO:tasks.workunit.client.0.smithi031.stdout:1/246: dwrite d1/db/d16/d2f/f3b [0,4194304] 0 2024-04-22T23:45:40.491 INFO:tasks.workunit.client.0.smithi031.stdout:2/290: link d6/dc/d42/d28/f2a d6/dc/d42/f5c 0 2024-04-22T23:45:40.493 INFO:tasks.workunit.client.0.smithi031.stdout:2/291: readlink d6/dc/d17/l3f 0 2024-04-22T23:45:40.493 INFO:tasks.workunit.client.0.smithi031.stdout:0/286: creat d0/d1/d2/d36/d11/d1c/d54/f69 x:0 0 0 2024-04-22T23:45:40.494 INFO:tasks.workunit.client.0.smithi031.stdout:7/299: creat d9/d4f/d65/f6b x:0 0 0 2024-04-22T23:45:40.495 INFO:tasks.workunit.client.0.smithi031.stdout:9/330: creat d2/dc/f60 x:0 0 0 2024-04-22T23:45:40.496 INFO:tasks.workunit.client.0.smithi031.stdout:4/444: dwrite db/d14/d1d/f76 [0,4194304] 0 2024-04-22T23:45:40.499 INFO:tasks.workunit.client.0.smithi031.stdout:6/378: getdents d3/d35/d75 0 2024-04-22T23:45:40.499 INFO:tasks.workunit.client.0.smithi031.stdout:3/267: dwrite d4/d6/da/f3d [0,4194304] 0 2024-04-22T23:45:40.501 INFO:tasks.workunit.client.0.smithi031.stdout:4/445: chown db/d14/f3a 342 1 2024-04-22T23:45:40.502 INFO:tasks.workunit.client.0.smithi031.stdout:1/247: mknod d1/de/c56 0 2024-04-22T23:45:40.504 INFO:tasks.workunit.client.0.smithi031.stdout:2/292: dread d6/dc/d42/d24/f2b [0,4194304] 0 2024-04-22T23:45:40.504 INFO:tasks.workunit.client.0.smithi031.stdout:0/287: creat d0/d13/d18/d1e/f6a x:0 0 0 2024-04-22T23:45:40.504 INFO:tasks.workunit.client.0.smithi031.stdout:2/293: stat d6/dc/d42/d28 0 2024-04-22T23:45:40.505 INFO:tasks.workunit.client.0.smithi031.stdout:9/331: chown d2/dc/d21/d3d/d40/l3b 30 1 2024-04-22T23:45:40.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/332: write d2/f35 [534271,125615] 0 2024-04-22T23:45:40.507 INFO:tasks.workunit.client.0.smithi031.stdout:6/379: mknod d3/d35/c81 0 2024-04-22T23:45:40.508 INFO:tasks.workunit.client.0.smithi031.stdout:3/268: mkdir d4/d54/d56 0 2024-04-22T23:45:40.512 INFO:tasks.workunit.client.0.smithi031.stdout:3/269: dread d4/d6/da/f3d [0,4194304] 0 2024-04-22T23:45:40.516 INFO:tasks.workunit.client.0.smithi031.stdout:4/446: rename db/d31/d40/d73 to db/d31/d40/d9b 0 2024-04-22T23:45:40.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/288: fsync d0/d1/d2/d36/d11/f14 0 2024-04-22T23:45:40.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/289: dread - d0/d1/d2/d36/d11/d1a/d3c/f59 zero size 2024-04-22T23:45:40.520 INFO:tasks.workunit.client.0.smithi031.stdout:3/270: dwrite d4/d10/f2b [0,4194304] 0 2024-04-22T23:45:40.522 INFO:tasks.workunit.client.0.smithi031.stdout:3/271: write d4/d6/da/f3d [2729490,7916] 0 2024-04-22T23:45:40.524 INFO:tasks.workunit.client.0.smithi031.stdout:6/380: creat d3/d2a/d3c/d65/f82 x:0 0 0 2024-04-22T23:45:40.530 INFO:tasks.workunit.client.0.smithi031.stdout:1/248: mknod d1/c57 0 2024-04-22T23:45:40.530 INFO:tasks.workunit.client.0.smithi031.stdout:9/333: dread d2/dc/ff [0,4194304] 0 2024-04-22T23:45:40.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/294: truncate d6/dc/d42/f5c 214347 0 2024-04-22T23:45:40.532 INFO:tasks.workunit.client.0.smithi031.stdout:9/334: write d2/dc/f50 [911809,67880] 0 2024-04-22T23:45:40.535 INFO:tasks.workunit.client.0.smithi031.stdout:4/447: rename db/d14/d2d/d3c to db/d14/d2d/d49/d59/d83/d9c 0 2024-04-22T23:45:40.536 INFO:tasks.workunit.client.0.smithi031.stdout:8/325: dwrite d5/da/d12/d37/d3b/f42 [0,4194304] 0 2024-04-22T23:45:40.538 INFO:tasks.workunit.client.0.smithi031.stdout:1/249: dread d1/f46 [0,4194304] 0 2024-04-22T23:45:40.540 INFO:tasks.workunit.client.0.smithi031.stdout:6/381: chown d3/d2a/d3c/d65/c79 1030770 1 2024-04-22T23:45:40.540 INFO:tasks.workunit.client.0.smithi031.stdout:1/250: write d1/db/d16/d2f/f3b [2902805,7404] 0 2024-04-22T23:45:40.542 INFO:tasks.workunit.client.0.smithi031.stdout:1/251: fsync d1/d21/d29/f1b 0 2024-04-22T23:45:40.542 INFO:tasks.workunit.client.0.smithi031.stdout:2/295: dwrite d6/d45/d50/f58 [0,4194304] 0 2024-04-22T23:45:40.543 INFO:tasks.workunit.client.0.smithi031.stdout:1/252: read d1/db/d16/f40 [1640599,42268] 0 2024-04-22T23:45:40.548 INFO:tasks.workunit.client.0.smithi031.stdout:9/335: creat d2/d47/d4a/f61 x:0 0 0 2024-04-22T23:45:40.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/336: write d2/dc/d21/d3d/f51 [412810,64179] 0 2024-04-22T23:45:40.550 INFO:tasks.workunit.client.0.smithi031.stdout:3/272: rename d4/d6/da/c53 to d4/d54/c57 0 2024-04-22T23:45:40.559 INFO:tasks.workunit.client.0.smithi031.stdout:3/273: write d4/d6/f16 [1833166,2615] 0 2024-04-22T23:45:40.566 INFO:tasks.workunit.client.0.smithi031.stdout:2/296: rename d6/dc/d42/d24/c38 to d6/dc/d48/c5d 0 2024-04-22T23:45:40.570 INFO:tasks.workunit.client.0.smithi031.stdout:1/253: link d1/db/f19 d1/de/d1e/d34/d41/f58 0 2024-04-22T23:45:40.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/254: chown d1/de/d1e/d34/d41/f42 581 1 2024-04-22T23:45:40.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/255: chown d1/de/d1e/d34/d41/f58 3884 1 2024-04-22T23:45:40.573 INFO:tasks.workunit.client.0.smithi031.stdout:9/337: dread d2/f2e [0,4194304] 0 2024-04-22T23:45:40.575 INFO:tasks.workunit.client.0.smithi031.stdout:2/297: write d6/f1c [145758,59884] 0 2024-04-22T23:45:40.575 INFO:tasks.workunit.client.0.smithi031.stdout:2/298: chown d6/dc/d17/c18 7 1 2024-04-22T23:45:40.576 INFO:tasks.workunit.client.0.smithi031.stdout:6/382: creat d3/f83 x:0 0 0 2024-04-22T23:45:40.576 INFO:tasks.workunit.client.0.smithi031.stdout:8/326: write d5/da/d12/d37/d34/f46 [1536941,130766] 0 2024-04-22T23:45:40.577 INFO:tasks.workunit.client.0.smithi031.stdout:8/327: fsync d5/da/f1e 0 2024-04-22T23:45:40.581 INFO:tasks.workunit.client.0.smithi031.stdout:3/274: rmdir d4/d6/d55 0 2024-04-22T23:45:40.582 INFO:tasks.workunit.client.0.smithi031.stdout:3/275: readlink d4/l14 0 2024-04-22T23:45:40.584 INFO:tasks.workunit.client.0.smithi031.stdout:2/299: mknod d6/dc/d17/c5e 0 2024-04-22T23:45:40.586 INFO:tasks.workunit.client.0.smithi031.stdout:2/300: dread d6/dc/d17/f25 [0,4194304] 0 2024-04-22T23:45:40.587 INFO:tasks.workunit.client.0.smithi031.stdout:1/256: mknod d1/d35/d36/c59 0 2024-04-22T23:45:40.589 INFO:tasks.workunit.client.0.smithi031.stdout:6/383: creat d3/d1f/d60/f84 x:0 0 0 2024-04-22T23:45:40.591 INFO:tasks.workunit.client.0.smithi031.stdout:3/276: mknod d4/d6/d4c/c58 0 2024-04-22T23:45:40.593 INFO:tasks.workunit.client.0.smithi031.stdout:2/301: dwrite d6/dc/d42/d24/f39 [4194304,4194304] 0 2024-04-22T23:45:40.594 INFO:tasks.workunit.client.0.smithi031.stdout:3/277: chown d4/d6/d11/d25/f36 5092 1 2024-04-22T23:45:40.595 INFO:tasks.workunit.client.0.smithi031.stdout:2/302: read - d6/dc/d48/f55 zero size 2024-04-22T23:45:40.596 INFO:tasks.workunit.client.0.smithi031.stdout:1/257: dwrite d1/d21/d29/f1b [4194304,4194304] 0 2024-04-22T23:45:40.598 INFO:tasks.workunit.client.0.smithi031.stdout:9/338: getdents d2/dc 0 2024-04-22T23:45:40.601 INFO:tasks.workunit.client.0.smithi031.stdout:3/278: mkdir d4/d54/d59 0 2024-04-22T23:45:40.604 INFO:tasks.workunit.client.0.smithi031.stdout:2/303: chown d6/dc/d42/d28/f2a 1713 1 2024-04-22T23:45:40.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/384: rmdir d3/d1f/d60 39 2024-04-22T23:45:40.604 INFO:tasks.workunit.client.0.smithi031.stdout:9/339: read d2/dc/f10 [512415,47417] 0 2024-04-22T23:45:40.605 INFO:tasks.workunit.client.0.smithi031.stdout:6/385: write d3/d2a/d3c/d40/d4d/f58 [954604,123630] 0 2024-04-22T23:45:40.605 INFO:tasks.workunit.client.0.smithi031.stdout:1/258: mkdir d1/d35/d36/d5a 0 2024-04-22T23:45:40.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/259: dwrite f0 [4194304,4194304] 0 2024-04-22T23:45:40.614 INFO:tasks.workunit.client.0.smithi031.stdout:6/386: mkdir d3/d2a/d85 0 2024-04-22T23:45:40.615 INFO:tasks.workunit.client.0.smithi031.stdout:1/260: read d1/d21/d29/f23 [1971910,82212] 0 2024-04-22T23:45:40.616 INFO:tasks.workunit.client.0.smithi031.stdout:3/279: truncate d4/d6/d11/f1c 3570996 0 2024-04-22T23:45:40.619 INFO:tasks.workunit.client.0.smithi031.stdout:6/387: mkdir d3/d35/d86 0 2024-04-22T23:45:40.621 INFO:tasks.workunit.client.0.smithi031.stdout:1/261: symlink d1/d21/d29/l5b 0 2024-04-22T23:45:40.621 INFO:tasks.workunit.client.0.smithi031.stdout:2/304: dwrite d6/dc/d42/d24/f2b [0,4194304] 0 2024-04-22T23:45:40.622 INFO:tasks.workunit.client.0.smithi031.stdout:2/305: stat d6/dc/d42/d28/f54 0 2024-04-22T23:45:40.624 INFO:tasks.workunit.client.0.smithi031.stdout:2/306: chown d6/dc/d57/l4c 0 1 2024-04-22T23:45:40.625 INFO:tasks.workunit.client.0.smithi031.stdout:9/340: rename d2/dc/d21/d3d/d40/l45 to d2/l62 0 2024-04-22T23:45:40.625 INFO:tasks.workunit.client.0.smithi031.stdout:3/280: dread d4/d6/f16 [0,4194304] 0 2024-04-22T23:45:40.627 INFO:tasks.workunit.client.0.smithi031.stdout:6/388: write d3/d1f/d60/f62 [3328735,62522] 0 2024-04-22T23:45:40.633 INFO:tasks.workunit.client.0.smithi031.stdout:9/341: creat d2/dc/d21/f63 x:0 0 0 2024-04-22T23:45:40.634 INFO:tasks.workunit.client.0.smithi031.stdout:3/281: fdatasync d4/d6/d11/d25/d44/f50 0 2024-04-22T23:45:40.635 INFO:tasks.workunit.client.0.smithi031.stdout:3/282: fdatasync f3 0 2024-04-22T23:45:40.636 INFO:tasks.workunit.client.0.smithi031.stdout:9/342: dread d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:40.638 INFO:tasks.workunit.client.0.smithi031.stdout:1/262: creat d1/db/d16/d1f/d52/f5c x:0 0 0 2024-04-22T23:45:40.642 INFO:tasks.workunit.client.0.smithi031.stdout:3/283: creat d4/d6/d11/d3e/f5a x:0 0 0 2024-04-22T23:45:40.646 INFO:tasks.workunit.client.0.smithi031.stdout:2/307: creat d6/dc/f5f x:0 0 0 2024-04-22T23:45:40.649 INFO:tasks.workunit.client.0.smithi031.stdout:9/343: dread d2/d47/d4a/f32 [0,4194304] 0 2024-04-22T23:45:40.653 INFO:tasks.workunit.client.0.smithi031.stdout:1/263: link d1/db/d16/d1f/d52/f5c d1/d21/d4b/f5d 0 2024-04-22T23:45:40.654 INFO:tasks.workunit.client.0.smithi031.stdout:3/284: mkdir d4/d6/d4c/d4d/d5b 0 2024-04-22T23:45:40.655 INFO:tasks.workunit.client.0.smithi031.stdout:6/389: getdents d3/d2a/d3c/d5e 0 2024-04-22T23:45:40.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/285: read d4/d10/f2b [593804,68172] 0 2024-04-22T23:45:40.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/286: chown d4/d6/d11/d25/d44 3 1 2024-04-22T23:45:40.657 INFO:tasks.workunit.client.0.smithi031.stdout:9/344: truncate d2/d15/f54 1811309 0 2024-04-22T23:45:40.658 INFO:tasks.workunit.client.0.smithi031.stdout:9/345: chown d2/dc/d3e/f56 383360459 1 2024-04-22T23:45:40.662 INFO:tasks.workunit.client.0.smithi031.stdout:3/287: mknod d4/d6/d4c/d4d/c5c 0 2024-04-22T23:45:40.664 INFO:tasks.workunit.client.0.smithi031.stdout:3/288: read d4/d6/d11/d25/d44/f50 [330056,124598] 0 2024-04-22T23:45:40.664 INFO:tasks.workunit.client.0.smithi031.stdout:3/289: stat d4/d54/d56 0 2024-04-22T23:45:40.665 INFO:tasks.workunit.client.0.smithi031.stdout:2/308: creat d6/f60 x:0 0 0 2024-04-22T23:45:40.666 INFO:tasks.workunit.client.0.smithi031.stdout:9/346: fsync d2/d47/d4a/f32 0 2024-04-22T23:45:40.666 INFO:tasks.workunit.client.0.smithi031.stdout:9/347: chown d2/dc/f43 3 1 2024-04-22T23:45:40.667 INFO:tasks.workunit.client.0.smithi031.stdout:9/348: read - d2/dc/d3e/f56 zero size 2024-04-22T23:45:40.672 INFO:tasks.workunit.client.0.smithi031.stdout:6/390: write d3/f8 [201756,61552] 0 2024-04-22T23:45:40.674 INFO:tasks.workunit.client.0.smithi031.stdout:3/290: write d4/d6/d11/d3e/f43 [103140,86830] 0 2024-04-22T23:45:40.678 INFO:tasks.workunit.client.0.smithi031.stdout:9/349: rename d2/dc/d21/d3d/d40/d2a to d2/d47/d64 0 2024-04-22T23:45:40.678 INFO:tasks.workunit.client.0.smithi031.stdout:1/264: link d1/de/d1e/d34/d41/c32 d1/d3f/c5e 0 2024-04-22T23:45:40.681 INFO:tasks.workunit.client.0.smithi031.stdout:6/391: creat d3/d35/d75/d7d/f87 x:0 0 0 2024-04-22T23:45:40.681 INFO:tasks.workunit.client.0.smithi031.stdout:6/392: chown d3/d2a/d3c/c47 12586 1 2024-04-22T23:45:40.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/265: symlink d1/de/d1e/d34/l5f 0 2024-04-22T23:45:40.685 INFO:tasks.workunit.client.0.smithi031.stdout:2/309: creat d6/dc/d42/f61 x:0 0 0 2024-04-22T23:45:40.685 INFO:tasks.workunit.client.0.smithi031.stdout:6/393: mknod d3/d2a/c88 0 2024-04-22T23:45:40.687 INFO:tasks.workunit.client.0.smithi031.stdout:1/266: symlink d1/d3f/d54/l60 0 2024-04-22T23:45:40.687 INFO:tasks.workunit.client.0.smithi031.stdout:2/310: read d6/f16 [1363089,3573] 0 2024-04-22T23:45:40.688 INFO:tasks.workunit.client.0.smithi031.stdout:6/394: stat d3/c6 0 2024-04-22T23:45:40.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/311: creat d6/dc/f62 x:0 0 0 2024-04-22T23:45:40.694 INFO:tasks.workunit.client.0.smithi031.stdout:6/395: dwrite d3/d35/d75/d7d/f87 [0,4194304] 0 2024-04-22T23:45:40.696 INFO:tasks.workunit.client.0.smithi031.stdout:2/312: chown d6/f9 283358391 1 2024-04-22T23:45:40.696 INFO:tasks.workunit.client.0.smithi031.stdout:2/313: chown d6/dc/d17/c20 1382474 1 2024-04-22T23:45:40.697 INFO:tasks.workunit.client.0.smithi031.stdout:6/396: creat d3/d35/d75/f89 x:0 0 0 2024-04-22T23:45:40.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/397: chown d3/d1f/c2d 800 1 2024-04-22T23:45:40.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/398: chown d3/c13 7747460 1 2024-04-22T23:45:40.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/399: fsync d3/d2a/d3c/d5e/f7b 0 2024-04-22T23:45:40.701 INFO:tasks.workunit.client.0.smithi031.stdout:2/314: symlink d6/dc/d42/l63 0 2024-04-22T23:45:40.706 INFO:tasks.workunit.client.0.smithi031.stdout:6/400: getdents d3/d2a/d3c/d65 0 2024-04-22T23:45:40.707 INFO:tasks.workunit.client.0.smithi031.stdout:1/267: truncate d1/d2e/f3e 549589 0 2024-04-22T23:45:40.708 INFO:tasks.workunit.client.0.smithi031.stdout:2/315: dwrite d6/dc/d57/f4d [0,4194304] 0 2024-04-22T23:45:40.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/401: mknod d3/d2a/d3c/d40/d7a/c8a 0 2024-04-22T23:45:40.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/316: rmdir d6/d59 0 2024-04-22T23:45:40.720 INFO:tasks.workunit.client.0.smithi031.stdout:6/402: mknod d3/d35/d86/c8b 0 2024-04-22T23:45:40.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/403: fsync d3/d2a/d3c/d40/f46 0 2024-04-22T23:45:40.722 INFO:tasks.workunit.client.0.smithi031.stdout:1/268: write d1/de/d1e/d34/d41/f58 [216575,44275] 0 2024-04-22T23:45:40.723 INFO:tasks.workunit.client.0.smithi031.stdout:6/404: mknod d3/d35/d86/c8c 0 2024-04-22T23:45:40.725 INFO:tasks.workunit.client.0.smithi031.stdout:2/317: creat d6/dc/d48/d4b/f64 x:0 0 0 2024-04-22T23:45:40.726 INFO:tasks.workunit.client.0.smithi031.stdout:1/269: mknod d1/de/d1e/c61 0 2024-04-22T23:45:40.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/405: mkdir d3/d77/d8d 0 2024-04-22T23:45:40.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/318: dwrite d6/dc/d42/d28/f37 [0,4194304] 0 2024-04-22T23:45:40.789 INFO:tasks.workunit.client.0.smithi031.stdout:6/406: dread - d3/d1f/f64 zero size 2024-04-22T23:45:40.793 INFO:tasks.workunit.client.0.smithi031.stdout:2/319: fsync d6/f16 0 2024-04-22T23:45:40.795 INFO:tasks.workunit.client.0.smithi031.stdout:6/407: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:45:40.800 INFO:tasks.workunit.client.0.smithi031.stdout:2/320: dwrite d6/dc/d42/d24/f34 [0,4194304] 0 2024-04-22T23:45:40.805 INFO:tasks.workunit.client.0.smithi031.stdout:1/270: getdents d1/d35/d36 0 2024-04-22T23:45:40.812 INFO:tasks.workunit.client.0.smithi031.stdout:1/271: dwrite d1/db/d16/d1f/f26 [0,4194304] 0 2024-04-22T23:45:40.815 INFO:tasks.workunit.client.0.smithi031.stdout:6/408: dread d3/f38 [0,4194304] 0 2024-04-22T23:45:40.817 INFO:tasks.workunit.client.0.smithi031.stdout:2/321: symlink d6/dc/d42/l65 0 2024-04-22T23:45:40.820 INFO:tasks.workunit.client.0.smithi031.stdout:1/272: creat d1/de/d1e/d34/d41/f62 x:0 0 0 2024-04-22T23:45:40.823 INFO:tasks.workunit.client.0.smithi031.stdout:1/273: dread d1/de/d1e/d34/d41/f58 [0,4194304] 0 2024-04-22T23:45:40.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/409: mkdir d3/d1f/d4c/d8e 0 2024-04-22T23:45:40.826 INFO:tasks.workunit.client.0.smithi031.stdout:2/322: symlink d6/dc/d42/d24/l66 0 2024-04-22T23:45:40.827 INFO:tasks.workunit.client.0.smithi031.stdout:5/311: sync 2024-04-22T23:45:40.828 INFO:tasks.workunit.client.0.smithi031.stdout:5/312: write db/d16/f54 [3847750,127854] 0 2024-04-22T23:45:40.830 INFO:tasks.workunit.client.0.smithi031.stdout:5/313: write db/d16/d1d/d24/f63 [658115,68701] 0 2024-04-22T23:45:40.858 INFO:tasks.workunit.client.0.smithi031.stdout:2/323: truncate d6/dc/d42/d28/f37 2550091 0 2024-04-22T23:45:40.860 INFO:tasks.workunit.client.0.smithi031.stdout:2/324: read d6/ff [2707351,125389] 0 2024-04-22T23:45:40.861 INFO:tasks.workunit.client.0.smithi031.stdout:5/314: truncate db/d16/d2a/f2c 1224438 0 2024-04-22T23:45:40.865 INFO:tasks.workunit.client.0.smithi031.stdout:2/325: creat d6/dc/d42/f67 x:0 0 0 2024-04-22T23:45:40.866 INFO:tasks.workunit.client.0.smithi031.stdout:5/315: dwrite db/d16/f65 [0,4194304] 0 2024-04-22T23:45:40.868 INFO:tasks.workunit.client.0.smithi031.stdout:2/326: mkdir d6/d45/d68 0 2024-04-22T23:45:40.875 INFO:tasks.workunit.client.0.smithi031.stdout:5/316: dwrite db/d16/d1d/d24/d58/f51 [0,4194304] 0 2024-04-22T23:45:40.887 INFO:tasks.workunit.client.0.smithi031.stdout:2/327: dread f3 [0,4194304] 0 2024-04-22T23:45:40.887 INFO:tasks.workunit.client.0.smithi031.stdout:2/328: dread - d6/dc/d42/f43 zero size 2024-04-22T23:45:40.888 INFO:tasks.workunit.client.0.smithi031.stdout:2/329: write d6/dc/d17/f3c [637073,22741] 0 2024-04-22T23:45:40.891 INFO:tasks.workunit.client.0.smithi031.stdout:2/330: fdatasync d6/f22 0 2024-04-22T23:45:40.895 INFO:tasks.workunit.client.0.smithi031.stdout:2/331: dread d6/dc/d42/d24/f39 [4194304,4194304] 0 2024-04-22T23:45:40.895 INFO:tasks.workunit.client.0.smithi031.stdout:2/332: readlink d6/dc/d17/l3f 0 2024-04-22T23:45:40.901 INFO:tasks.workunit.client.0.smithi031.stdout:2/333: mkdir d6/dc/d17/d69 0 2024-04-22T23:45:40.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/334: write d6/dc/f1a [159729,52858] 0 2024-04-22T23:45:40.905 INFO:tasks.workunit.client.0.smithi031.stdout:2/335: fdatasync d6/dc/d17/f3c 0 2024-04-22T23:45:40.910 INFO:tasks.workunit.client.0.smithi031.stdout:2/336: creat d6/dc/d57/d52/f6a x:0 0 0 2024-04-22T23:45:40.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/337: write d6/dc/d42/d24/f39 [6064257,37391] 0 2024-04-22T23:45:40.923 INFO:tasks.workunit.client.0.smithi031.stdout:5/317: write db/d16/d1d/d24/f2e [1575313,44265] 0 2024-04-22T23:45:40.925 INFO:tasks.workunit.client.0.smithi031.stdout:2/338: getdents d6/d45/d50 0 2024-04-22T23:45:40.930 INFO:tasks.workunit.client.0.smithi031.stdout:2/339: mkdir d6/dc/d48/d4b/d6b 0 2024-04-22T23:45:40.931 INFO:tasks.workunit.client.0.smithi031.stdout:2/340: write d6/dc/f1a [5072620,81259] 0 2024-04-22T23:45:40.936 INFO:tasks.workunit.client.0.smithi031.stdout:2/341: dwrite d6/dc/d42/f67 [0,4194304] 0 2024-04-22T23:45:40.938 INFO:tasks.workunit.client.0.smithi031.stdout:5/318: link db/d16/d2a/f30 db/d16/d1d/f6b 0 2024-04-22T23:45:40.949 INFO:tasks.workunit.client.0.smithi031.stdout:5/319: mknod db/d16/c6c 0 2024-04-22T23:45:40.954 INFO:tasks.workunit.client.0.smithi031.stdout:2/342: getdents d6/dc/d48 0 2024-04-22T23:45:40.955 INFO:tasks.workunit.client.0.smithi031.stdout:5/320: dwrite db/d16/d1d/d24/d32/f39 [0,4194304] 0 2024-04-22T23:45:40.957 INFO:tasks.workunit.client.0.smithi031.stdout:2/343: truncate d6/ff 3527407 0 2024-04-22T23:45:40.960 INFO:tasks.workunit.client.0.smithi031.stdout:5/321: symlink db/d16/d1d/d24/d27/l6d 0 2024-04-22T23:45:40.961 INFO:tasks.workunit.client.0.smithi031.stdout:0/290: sync 2024-04-22T23:45:40.961 INFO:tasks.workunit.client.0.smithi031.stdout:7/300: sync 2024-04-22T23:45:40.961 INFO:tasks.workunit.client.0.smithi031.stdout:4/448: sync 2024-04-22T23:45:40.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/291: fdatasync d0/d13/d18/d1e/f68 0 2024-04-22T23:45:40.966 INFO:tasks.workunit.client.0.smithi031.stdout:5/322: fsync db/d16/d1d/d24/f5a 0 2024-04-22T23:45:40.969 INFO:tasks.workunit.client.0.smithi031.stdout:7/301: mknod d9/d14/d1f/c6c 0 2024-04-22T23:45:40.969 INFO:tasks.workunit.client.0.smithi031.stdout:7/302: write d9/fa [2000744,65992] 0 2024-04-22T23:45:40.971 INFO:tasks.workunit.client.0.smithi031.stdout:7/303: dread d9/d15/f56 [0,4194304] 0 2024-04-22T23:45:40.972 INFO:tasks.workunit.client.0.smithi031.stdout:4/449: rename db/d14/d1d/d7b/d7f/l22 to db/d14/d1d/d7b/d7f/l9d 0 2024-04-22T23:45:40.984 INFO:tasks.workunit.client.0.smithi031.stdout:7/304: rename d9/d4f to d9/d15/d6d 0 2024-04-22T23:45:40.986 INFO:tasks.workunit.client.0.smithi031.stdout:4/450: mknod db/d14/d1d/d7b/c9e 0 2024-04-22T23:45:40.987 INFO:tasks.workunit.client.0.smithi031.stdout:7/305: dread d9/d15/f21 [0,4194304] 0 2024-04-22T23:45:40.989 INFO:tasks.workunit.client.0.smithi031.stdout:0/292: creat d0/d1/f6b x:0 0 0 2024-04-22T23:45:40.990 INFO:tasks.workunit.client.0.smithi031.stdout:0/293: chown d0/d13/f5b 2031 1 2024-04-22T23:45:40.992 INFO:tasks.workunit.client.0.smithi031.stdout:4/451: symlink db/d14/d2d/l9f 0 2024-04-22T23:45:40.993 INFO:tasks.workunit.client.0.smithi031.stdout:2/344: rmdir d6 39 2024-04-22T23:45:40.994 INFO:tasks.workunit.client.0.smithi031.stdout:9/350: sync 2024-04-22T23:45:40.994 INFO:tasks.workunit.client.0.smithi031.stdout:1/274: sync 2024-04-22T23:45:40.994 INFO:tasks.workunit.client.0.smithi031.stdout:8/328: sync 2024-04-22T23:45:40.994 INFO:tasks.workunit.client.0.smithi031.stdout:6/410: sync 2024-04-22T23:45:40.994 INFO:tasks.workunit.client.0.smithi031.stdout:3/291: sync 2024-04-22T23:45:40.995 INFO:tasks.workunit.client.0.smithi031.stdout:3/292: dread - d4/d6/f23 zero size 2024-04-22T23:45:40.995 INFO:tasks.workunit.client.0.smithi031.stdout:3/293: readlink d4/l14 0 2024-04-22T23:45:40.995 INFO:tasks.workunit.client.0.smithi031.stdout:5/323: rmdir db/d16/d1d/d24 39 2024-04-22T23:45:41.002 INFO:tasks.workunit.client.0.smithi031.stdout:8/329: chown d5/da/d12/d1d/d38/f3f 1812734 1 2024-04-22T23:45:41.002 INFO:tasks.workunit.client.0.smithi031.stdout:8/330: dread - d5/da/d12/d2e/d4d/f52 zero size 2024-04-22T23:45:41.004 INFO:tasks.workunit.client.0.smithi031.stdout:7/306: link d9/d14/d22/d5b/f5d d9/d14/d22/d5b/f6e 0 2024-04-22T23:45:41.005 INFO:tasks.workunit.client.0.smithi031.stdout:7/307: write d9/fa [4581623,98267] 0 2024-04-22T23:45:41.006 INFO:tasks.workunit.client.0.smithi031.stdout:7/308: write d9/d15/f56 [1570464,8394] 0 2024-04-22T23:45:41.006 INFO:tasks.workunit.client.0.smithi031.stdout:5/324: unlink db/d16/d2a/d59/f61 0 2024-04-22T23:45:41.010 INFO:tasks.workunit.client.0.smithi031.stdout:4/452: getdents db/d14/d2d/d49/d59/d83/d9c/d72 0 2024-04-22T23:45:41.010 INFO:tasks.workunit.client.0.smithi031.stdout:9/351: mkdir d2/dd/d65 0 2024-04-22T23:45:41.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/275: mkdir d1/d47/d63 0 2024-04-22T23:45:41.015 INFO:tasks.workunit.client.0.smithi031.stdout:8/331: mkdir d5/d48/d61 0 2024-04-22T23:45:41.016 INFO:tasks.workunit.client.0.smithi031.stdout:8/332: readlink d5/da/d12/d1d/l22 0 2024-04-22T23:45:41.018 INFO:tasks.workunit.client.0.smithi031.stdout:1/276: dwrite d1/de/d1e/d34/d41/f42 [0,4194304] 0 2024-04-22T23:45:41.022 INFO:tasks.workunit.client.0.smithi031.stdout:2/345: mkdir d6/d45/d68/d6c 0 2024-04-22T23:45:41.022 INFO:tasks.workunit.client.0.smithi031.stdout:7/309: dread d9/d15/d3a/d41/f5a [0,4194304] 0 2024-04-22T23:45:41.024 INFO:tasks.workunit.client.0.smithi031.stdout:4/453: creat db/d14/d2d/d49/d59/d83/d9c/fa0 x:0 0 0 2024-04-22T23:45:41.025 INFO:tasks.workunit.client.0.smithi031.stdout:1/277: dwrite f0 [0,4194304] 0 2024-04-22T23:45:41.031 INFO:tasks.workunit.client.0.smithi031.stdout:8/333: unlink d5/da/d12/d2e/d4d/f58 0 2024-04-22T23:45:41.035 INFO:tasks.workunit.client.0.smithi031.stdout:1/278: dwrite d1/de/f11 [0,4194304] 0 2024-04-22T23:45:41.035 INFO:tasks.workunit.client.0.smithi031.stdout:4/454: dwrite db/d14/d2d/d49/d59/d83/d9c/fa0 [0,4194304] 0 2024-04-22T23:45:41.036 INFO:tasks.workunit.client.0.smithi031.stdout:5/325: symlink db/d16/d1d/d24/d27/l6e 0 2024-04-22T23:45:41.036 INFO:tasks.workunit.client.0.smithi031.stdout:2/346: rmdir d6/dc/d42 39 2024-04-22T23:45:41.038 INFO:tasks.workunit.client.0.smithi031.stdout:7/310: symlink d9/d14/d1f/d47/l6f 0 2024-04-22T23:45:41.038 INFO:tasks.workunit.client.0.smithi031.stdout:5/326: read - db/d16/d1d/d24/d27/f5b zero size 2024-04-22T23:45:41.044 INFO:tasks.workunit.client.0.smithi031.stdout:6/411: getdents d3/d35/d86 0 2024-04-22T23:45:41.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/334: creat d5/da/d12/d2e/d4d/f62 x:0 0 0 2024-04-22T23:45:41.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/335: chown d5/f26 0 1 2024-04-22T23:45:41.053 INFO:tasks.workunit.client.0.smithi031.stdout:4/455: rmdir db/d14/d2d/d49/d59/d6a 39 2024-04-22T23:45:41.056 INFO:tasks.workunit.client.0.smithi031.stdout:1/279: rename d1/d35/f4a to d1/de/d1e/f64 0 2024-04-22T23:45:41.059 INFO:tasks.workunit.client.0.smithi031.stdout:7/311: rmdir d9/d15/d3a/d41 39 2024-04-22T23:45:41.062 INFO:tasks.workunit.client.0.smithi031.stdout:1/280: dwrite d1/de/f11 [0,4194304] 0 2024-04-22T23:45:41.062 INFO:tasks.workunit.client.0.smithi031.stdout:7/312: rename d9/d15/d6d/d65 to d9/d15/d6d/d65/d70 22 2024-04-22T23:45:41.063 INFO:tasks.workunit.client.0.smithi031.stdout:2/347: dread d6/dc/d42/d28/f54 [0,4194304] 0 2024-04-22T23:45:41.064 INFO:tasks.workunit.client.0.smithi031.stdout:7/313: truncate d9/d15/f17 4337960 0 2024-04-22T23:45:41.065 INFO:tasks.workunit.client.0.smithi031.stdout:6/412: fsync d3/f11 0 2024-04-22T23:45:41.065 INFO:tasks.workunit.client.0.smithi031.stdout:8/336: truncate d5/f9 5381376 0 2024-04-22T23:45:41.065 INFO:tasks.workunit.client.0.smithi031.stdout:7/314: fsync d9/d15/d3a/f51 0 2024-04-22T23:45:41.066 INFO:tasks.workunit.client.0.smithi031.stdout:8/337: chown d5/da/d12/l24 6544 1 2024-04-22T23:45:41.069 INFO:tasks.workunit.client.0.smithi031.stdout:4/456: mkdir db/d14/d1d/d7b/d7f/d2a/da1 0 2024-04-22T23:45:41.071 INFO:tasks.workunit.client.0.smithi031.stdout:9/352: link d2/d47/d64/l3f d2/dc/d21/d3d/d40/l66 0 2024-04-22T23:45:41.072 INFO:tasks.workunit.client.0.smithi031.stdout:9/353: stat d2/d47/d64 0 2024-04-22T23:45:41.075 INFO:tasks.workunit.client.0.smithi031.stdout:5/327: fdatasync db/d16/d1d/f1f 0 2024-04-22T23:45:41.076 INFO:tasks.workunit.client.0.smithi031.stdout:5/328: write db/d16/d1d/d24/d27/f57 [4239920,38215] 0 2024-04-22T23:45:41.079 INFO:tasks.workunit.client.0.smithi031.stdout:2/348: creat d6/dc/d57/f6d x:0 0 0 2024-04-22T23:45:41.081 INFO:tasks.workunit.client.0.smithi031.stdout:8/338: dread d5/da/d12/d1d/d38/f3f [0,4194304] 0 2024-04-22T23:45:41.082 INFO:tasks.workunit.client.0.smithi031.stdout:7/315: mknod d9/d14/d22/c71 0 2024-04-22T23:45:41.082 INFO:tasks.workunit.client.0.smithi031.stdout:7/316: fdatasync d9/d14/f23 0 2024-04-22T23:45:41.083 INFO:tasks.workunit.client.0.smithi031.stdout:7/317: dread - d9/d15/f60 zero size 2024-04-22T23:45:41.086 INFO:tasks.workunit.client.0.smithi031.stdout:6/413: rename d3/d2a/d3c/d40/f57 to d3/d1f/d60/f8f 0 2024-04-22T23:45:41.087 INFO:tasks.workunit.client.0.smithi031.stdout:6/414: write d3/d2a/d3c/d40/f4e [3592807,80397] 0 2024-04-22T23:45:41.089 INFO:tasks.workunit.client.0.smithi031.stdout:4/457: fsync db/d14/d2d/d49/d59/d6a/f64 0 2024-04-22T23:45:41.089 INFO:tasks.workunit.client.0.smithi031.stdout:6/415: read d3/d2a/d3c/f5f [125050,94546] 0 2024-04-22T23:45:41.099 INFO:tasks.workunit.client.0.smithi031.stdout:1/281: symlink d1/l65 0 2024-04-22T23:45:41.099 INFO:tasks.workunit.client.0.smithi031.stdout:1/282: chown d1/d35/d36/d5a 127 1 2024-04-22T23:45:41.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/283: read f0 [5817212,108725] 0 2024-04-22T23:45:41.100 INFO:tasks.workunit.client.0.smithi031.stdout:2/349: fdatasync d6/dc/d42/d28/f54 0 2024-04-22T23:45:41.101 INFO:tasks.workunit.client.0.smithi031.stdout:2/350: write d6/dc/f4e [265607,124934] 0 2024-04-22T23:45:41.105 INFO:tasks.workunit.client.0.smithi031.stdout:1/284: dread d1/d21/d29/f1b [0,4194304] 0 2024-04-22T23:45:41.108 INFO:tasks.workunit.client.0.smithi031.stdout:7/318: dwrite f3 [0,4194304] 0 2024-04-22T23:45:41.109 INFO:tasks.workunit.client.0.smithi031.stdout:7/319: write d9/d15/f56 [78766,37143] 0 2024-04-22T23:45:41.110 INFO:tasks.workunit.client.0.smithi031.stdout:7/320: readlink d9/l4e 0 2024-04-22T23:45:41.112 INFO:tasks.workunit.client.0.smithi031.stdout:0/294: sync 2024-04-22T23:45:41.112 INFO:tasks.workunit.client.0.smithi031.stdout:3/294: sync 2024-04-22T23:45:41.114 INFO:tasks.workunit.client.0.smithi031.stdout:6/416: read d3/f34 [3097478,33687] 0 2024-04-22T23:45:41.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/417: fsync d3/d1f/f4f 0 2024-04-22T23:45:41.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/418: readlink d3/d1f/l3e 0 2024-04-22T23:45:41.121 INFO:tasks.workunit.client.0.smithi031.stdout:5/329: write db/f3f [865862,90478] 0 2024-04-22T23:45:41.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/351: symlink d6/dc/d48/d4b/l6e 0 2024-04-22T23:45:41.122 INFO:tasks.workunit.client.0.smithi031.stdout:6/419: dwrite d3/d1f/f64 [0,4194304] 0 2024-04-22T23:45:41.123 INFO:tasks.workunit.client.0.smithi031.stdout:5/330: readlink db/d16/l28 0 2024-04-22T23:45:41.127 INFO:tasks.workunit.client.0.smithi031.stdout:0/295: mkdir d0/d13/d18/d6c 0 2024-04-22T23:45:41.132 INFO:tasks.workunit.client.0.smithi031.stdout:8/339: dwrite d5/f9 [0,4194304] 0 2024-04-22T23:45:41.135 INFO:tasks.workunit.client.0.smithi031.stdout:8/340: write d5/da/d12/d37/d3b/f4b [1401047,125898] 0 2024-04-22T23:45:41.137 INFO:tasks.workunit.client.0.smithi031.stdout:7/321: dwrite f6 [0,4194304] 0 2024-04-22T23:45:41.142 INFO:tasks.workunit.client.0.smithi031.stdout:4/458: creat db/d14/d2d/d49/d59/fa2 x:0 0 0 2024-04-22T23:45:41.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/354: link d2/c1a d2/dc/d3e/c67 0 2024-04-22T23:45:41.144 INFO:tasks.workunit.client.0.smithi031.stdout:4/459: fsync db/d14/d2d/d49/f99 0 2024-04-22T23:45:41.147 INFO:tasks.workunit.client.0.smithi031.stdout:9/355: chown d2/dc/d21/d3d/d40/f39 46553110 1 2024-04-22T23:45:41.148 INFO:tasks.workunit.client.0.smithi031.stdout:8/341: dwrite d5/da/d12/d37/d49/f4f [0,4194304] 0 2024-04-22T23:45:41.148 INFO:tasks.workunit.client.0.smithi031.stdout:4/460: write db/d31/d40/d62/f77 [346868,40856] 0 2024-04-22T23:45:41.149 INFO:tasks.workunit.client.0.smithi031.stdout:6/420: mknod d3/d77/d8d/c90 0 2024-04-22T23:45:41.150 INFO:tasks.workunit.client.0.smithi031.stdout:7/322: creat d9/d15/d6d/d65/f72 x:0 0 0 2024-04-22T23:45:41.152 INFO:tasks.workunit.client.0.smithi031.stdout:8/342: dread - d5/da/d12/f44 zero size 2024-04-22T23:45:41.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/352: truncate d6/dc/d42/d24/d32/f3a 199205 0 2024-04-22T23:45:41.155 INFO:tasks.workunit.client.0.smithi031.stdout:1/285: truncate f0 437752 0 2024-04-22T23:45:41.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/343: fsync d5/da/d12/d37/d3b/f42 0 2024-04-22T23:45:41.157 INFO:tasks.workunit.client.0.smithi031.stdout:8/344: chown d5/f9 3291194 1 2024-04-22T23:45:41.158 INFO:tasks.workunit.client.0.smithi031.stdout:9/356: fdatasync d2/dc/f4b 0 2024-04-22T23:45:41.160 INFO:tasks.workunit.client.0.smithi031.stdout:4/461: creat db/d14/d1d/d7b/d7f/d2a/d47/fa3 x:0 0 0 2024-04-22T23:45:41.160 INFO:tasks.workunit.client.0.smithi031.stdout:5/331: dwrite db/d16/d1d/f20 [0,4194304] 0 2024-04-22T23:45:41.162 INFO:tasks.workunit.client.0.smithi031.stdout:6/421: rename d3/d2a/d3c/d40/f46 to d3/d35/d86/f91 0 2024-04-22T23:45:41.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/323: mknod d9/d15/c73 0 2024-04-22T23:45:41.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/357: dwrite d2/d47/d4a/f61 [0,4194304] 0 2024-04-22T23:45:41.173 INFO:tasks.workunit.client.0.smithi031.stdout:6/422: dwrite d3/f15 [4194304,4194304] 0 2024-04-22T23:45:41.176 INFO:tasks.workunit.client.0.smithi031.stdout:0/296: getdents d0/d1/d2/d36/d11/d1a 0 2024-04-22T23:45:41.182 INFO:tasks.workunit.client.0.smithi031.stdout:5/332: creat db/d16/d2a/d36/f6f x:0 0 0 2024-04-22T23:45:41.186 INFO:tasks.workunit.client.0.smithi031.stdout:9/358: creat d2/dc/d21/d3d/f68 x:0 0 0 2024-04-22T23:45:41.188 INFO:tasks.workunit.client.0.smithi031.stdout:2/353: creat d6/dc/d48/d4b/d6b/f6f x:0 0 0 2024-04-22T23:45:41.189 INFO:tasks.workunit.client.0.smithi031.stdout:6/423: symlink d3/d2a/d3c/l92 0 2024-04-22T23:45:41.190 INFO:tasks.workunit.client.0.smithi031.stdout:6/424: dread - d3/d1f/d60/f84 zero size 2024-04-22T23:45:41.191 INFO:tasks.workunit.client.0.smithi031.stdout:0/297: creat d0/d13/d18/f6d x:0 0 0 2024-04-22T23:45:41.193 INFO:tasks.workunit.client.0.smithi031.stdout:5/333: rename db/d16/d1d/l55 to db/d16/d1d/d24/d32/d62/l70 0 2024-04-22T23:45:41.196 INFO:tasks.workunit.client.0.smithi031.stdout:6/425: fsync d3/f25 0 2024-04-22T23:45:41.197 INFO:tasks.workunit.client.0.smithi031.stdout:9/359: dread d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:41.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/462: creat db/fa4 x:0 0 0 2024-04-22T23:45:41.204 INFO:tasks.workunit.client.0.smithi031.stdout:9/360: creat d2/d47/d4a/f69 x:0 0 0 2024-04-22T23:45:41.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/345: truncate d5/da/d12/f1c 4294030 0 2024-04-22T23:45:41.209 INFO:tasks.workunit.client.0.smithi031.stdout:8/346: rmdir d5/da/d12/d37 39 2024-04-22T23:45:41.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/354: chown d6/dc/d42/d24/d32/f3a 847 1 2024-04-22T23:45:41.212 INFO:tasks.workunit.client.0.smithi031.stdout:6/426: dread d3/fe [0,4194304] 0 2024-04-22T23:45:41.213 INFO:tasks.workunit.client.0.smithi031.stdout:7/324: truncate d9/d14/d22/f2d 622887 0 2024-04-22T23:45:41.215 INFO:tasks.workunit.client.0.smithi031.stdout:4/463: dread db/f24 [0,4194304] 0 2024-04-22T23:45:41.216 INFO:tasks.workunit.client.0.smithi031.stdout:8/347: mknod d5/da/d12/d2e/d30/c63 0 2024-04-22T23:45:41.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/348: readlink d5/da/d12/d2f/l32 0 2024-04-22T23:45:41.222 INFO:tasks.workunit.client.0.smithi031.stdout:7/325: link d9/d14/d1f/c26 d9/d14/d22/c74 0 2024-04-22T23:45:41.223 INFO:tasks.workunit.client.0.smithi031.stdout:5/334: dread f5 [0,4194304] 0 2024-04-22T23:45:41.223 INFO:tasks.workunit.client.0.smithi031.stdout:9/361: truncate d2/dc/d21/d3d/f51 261469 0 2024-04-22T23:45:41.225 INFO:tasks.workunit.client.0.smithi031.stdout:6/427: dwrite d3/d2a/d3c/f71 [0,4194304] 0 2024-04-22T23:45:41.226 INFO:tasks.workunit.client.0.smithi031.stdout:2/355: link d6/f40 d6/dc/d49/f70 0 2024-04-22T23:45:41.227 INFO:tasks.workunit.client.0.smithi031.stdout:0/298: dwrite d0/d1/d2/d36/f2f [0,4194304] 0 2024-04-22T23:45:41.227 INFO:tasks.workunit.client.0.smithi031.stdout:5/335: fsync db/d16/d1d/d24/d27/f5b 0 2024-04-22T23:45:41.230 INFO:tasks.workunit.client.0.smithi031.stdout:4/464: creat db/d14/d2d/d49/d59/d6a/d63/fa5 x:0 0 0 2024-04-22T23:45:41.235 INFO:tasks.workunit.client.0.smithi031.stdout:5/336: dwrite db/d16/d1d/d24/d27/f5b [0,4194304] 0 2024-04-22T23:45:41.237 INFO:tasks.workunit.client.0.smithi031.stdout:8/349: rename d5/da/df/l2d to d5/l64 0 2024-04-22T23:45:41.242 INFO:tasks.workunit.client.0.smithi031.stdout:9/362: mknod d2/d47/c6a 0 2024-04-22T23:45:41.246 INFO:tasks.workunit.client.0.smithi031.stdout:8/350: dwrite d5/da/d12/d2e/d4d/f52 [0,4194304] 0 2024-04-22T23:45:41.251 INFO:tasks.workunit.client.0.smithi031.stdout:0/299: unlink d0/d13/d18/d1e/f68 0 2024-04-22T23:45:41.251 INFO:tasks.workunit.client.0.smithi031.stdout:0/300: dread - d0/d13/f67 zero size 2024-04-22T23:45:41.262 INFO:tasks.workunit.client.0.smithi031.stdout:7/326: dwrite d9/d14/f18 [0,4194304] 0 2024-04-22T23:45:41.263 INFO:tasks.workunit.client.0.smithi031.stdout:7/327: write d9/d14/f23 [674392,114533] 0 2024-04-22T23:45:41.264 INFO:tasks.workunit.client.0.smithi031.stdout:6/428: creat d3/d1f/d4c/d8e/f93 x:0 0 0 2024-04-22T23:45:41.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/328: truncate f3 4439528 0 2024-04-22T23:45:41.268 INFO:tasks.workunit.client.0.smithi031.stdout:2/356: write d6/dc/d42/f5c [525882,124195] 0 2024-04-22T23:45:41.270 INFO:tasks.workunit.client.0.smithi031.stdout:4/465: link db/d14/d1d/d7b/d7f/f32 db/d31/d40/d9b/fa6 0 2024-04-22T23:45:41.273 INFO:tasks.workunit.client.0.smithi031.stdout:0/301: creat d0/d13/d18/d1e/f6e x:0 0 0 2024-04-22T23:45:41.274 INFO:tasks.workunit.client.0.smithi031.stdout:5/337: mknod db/d16/c71 0 2024-04-22T23:45:41.275 INFO:tasks.workunit.client.0.smithi031.stdout:9/363: symlink d2/dd/l6b 0 2024-04-22T23:45:41.280 INFO:tasks.workunit.client.0.smithi031.stdout:0/302: write d0/d13/d18/f6d [1038067,43366] 0 2024-04-22T23:45:41.282 INFO:tasks.workunit.client.0.smithi031.stdout:9/364: read - d2/dc/d21/d3d/f68 zero size 2024-04-22T23:45:41.282 INFO:tasks.workunit.client.0.smithi031.stdout:8/351: dwrite d5/da/d12/d37/d49/f4f [0,4194304] 0 2024-04-22T23:45:41.286 INFO:tasks.workunit.client.0.smithi031.stdout:7/329: dwrite d9/fa [0,4194304] 0 2024-04-22T23:45:41.286 INFO:tasks.workunit.client.0.smithi031.stdout:2/357: mkdir d6/dc/d49/d71 0 2024-04-22T23:45:41.286 INFO:tasks.workunit.client.0.smithi031.stdout:4/466: mkdir db/d14/d1d/d7b/d7f/d2a/da1/da7 0 2024-04-22T23:45:41.287 INFO:tasks.workunit.client.0.smithi031.stdout:6/429: dwrite d3/d35/d75/f89 [0,4194304] 0 2024-04-22T23:45:41.289 INFO:tasks.workunit.client.0.smithi031.stdout:9/365: write d2/dc/f25 [985784,19524] 0 2024-04-22T23:45:41.291 INFO:tasks.workunit.client.0.smithi031.stdout:0/303: creat d0/d13/d18/d1e/f6f x:0 0 0 2024-04-22T23:45:41.294 INFO:tasks.workunit.client.0.smithi031.stdout:0/304: write d0/d13/d18/d1e/f6a [776305,25060] 0 2024-04-22T23:45:41.296 INFO:tasks.workunit.client.0.smithi031.stdout:1/286: sync 2024-04-22T23:45:41.296 INFO:tasks.workunit.client.0.smithi031.stdout:3/295: sync 2024-04-22T23:45:41.297 INFO:tasks.workunit.client.0.smithi031.stdout:2/358: unlink d6/dc/d48/f55 0 2024-04-22T23:45:41.300 INFO:tasks.workunit.client.0.smithi031.stdout:2/359: chown d6/dc/d49 174976573 1 2024-04-22T23:45:41.300 INFO:tasks.workunit.client.0.smithi031.stdout:9/366: creat d2/dc/d3e/f6c x:0 0 0 2024-04-22T23:45:41.302 INFO:tasks.workunit.client.0.smithi031.stdout:4/467: symlink db/la8 0 2024-04-22T23:45:41.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/352: dwrite d5/f19 [0,4194304] 0 2024-04-22T23:45:41.306 INFO:tasks.workunit.client.0.smithi031.stdout:1/287: mkdir d1/de/d1e/d34/d66 0 2024-04-22T23:45:41.307 INFO:tasks.workunit.client.0.smithi031.stdout:4/468: dread - db/f5f zero size 2024-04-22T23:45:41.307 INFO:tasks.workunit.client.0.smithi031.stdout:2/360: mkdir d6/dc/d48/d72 0 2024-04-22T23:45:41.308 INFO:tasks.workunit.client.0.smithi031.stdout:5/338: dwrite f5 [0,4194304] 0 2024-04-22T23:45:41.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/367: mkdir d2/dc/d6d 0 2024-04-22T23:45:41.312 INFO:tasks.workunit.client.0.smithi031.stdout:3/296: mknod d4/d54/d59/c5d 0 2024-04-22T23:45:41.313 INFO:tasks.workunit.client.0.smithi031.stdout:8/353: dread - d5/da/d12/d37/d3b/f40 zero size 2024-04-22T23:45:41.314 INFO:tasks.workunit.client.0.smithi031.stdout:0/305: rename d0/d1/l5a to d0/d1/d3f/l70 0 2024-04-22T23:45:41.316 INFO:tasks.workunit.client.0.smithi031.stdout:0/306: readlink d0/d1/d2/d36/d11/d1a/l34 0 2024-04-22T23:45:41.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/430: getdents d3/d1f 0 2024-04-22T23:45:41.319 INFO:tasks.workunit.client.0.smithi031.stdout:2/361: dread d6/dc/d42/d28/f2a [0,4194304] 0 2024-04-22T23:45:41.320 INFO:tasks.workunit.client.0.smithi031.stdout:4/469: creat db/d14/d1d/d7b/d7f/d2a/da1/fa9 x:0 0 0 2024-04-22T23:45:41.324 INFO:tasks.workunit.client.0.smithi031.stdout:4/470: fdatasync db/d14/d2d/d49/d59/d83/d9c/f55 0 2024-04-22T23:45:41.328 INFO:tasks.workunit.client.0.smithi031.stdout:0/307: dwrite d0/d1/d28/f55 [0,4194304] 0 2024-04-22T23:45:41.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/431: creat d3/d35/f94 x:0 0 0 2024-04-22T23:45:41.330 INFO:tasks.workunit.client.0.smithi031.stdout:9/368: truncate d2/d47/d64/d5d/f57 1012976 0 2024-04-22T23:45:41.331 INFO:tasks.workunit.client.0.smithi031.stdout:7/330: dread d9/ff [0,4194304] 0 2024-04-22T23:45:41.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/362: dread d6/d45/d50/f58 [0,4194304] 0 2024-04-22T23:45:41.332 INFO:tasks.workunit.client.0.smithi031.stdout:0/308: fsync d0/d1/d2/d36/fd 0 2024-04-22T23:45:41.333 INFO:tasks.workunit.client.0.smithi031.stdout:6/432: write d3/d2a/d3c/d40/f44 [3375243,83776] 0 2024-04-22T23:45:41.333 INFO:tasks.workunit.client.0.smithi031.stdout:2/363: readlink d6/dc/d48/d4b/l6e 0 2024-04-22T23:45:41.340 INFO:tasks.workunit.client.0.smithi031.stdout:1/288: link d1/de/d1e/d34/c53 d1/d35/d36/d5a/c67 0 2024-04-22T23:45:41.344 INFO:tasks.workunit.client.0.smithi031.stdout:1/289: dread d1/de/d1e/d34/d41/f4c [0,4194304] 0 2024-04-22T23:45:41.345 INFO:tasks.workunit.client.0.smithi031.stdout:5/339: dwrite db/d16/d2a/f2c [0,4194304] 0 2024-04-22T23:45:41.347 INFO:tasks.workunit.client.0.smithi031.stdout:5/340: write db/d16/d1d/d24/d32/d62/f67 [699487,25183] 0 2024-04-22T23:45:41.347 INFO:tasks.workunit.client.0.smithi031.stdout:6/433: mknod d3/d2a/c95 0 2024-04-22T23:45:41.351 INFO:tasks.workunit.client.0.smithi031.stdout:2/364: rmdir d6/dc/d48/d4b/d6b 39 2024-04-22T23:45:41.352 INFO:tasks.workunit.client.0.smithi031.stdout:4/471: write db/d31/d40/f68 [774177,91540] 0 2024-04-22T23:45:41.353 INFO:tasks.workunit.client.0.smithi031.stdout:4/472: readlink db/d14/d1d/d7b/d7f/l17 0 2024-04-22T23:45:41.355 INFO:tasks.workunit.client.0.smithi031.stdout:4/473: dread db/d14/d1d/d7b/d7f/d2a/d30/f42 [0,4194304] 0 2024-04-22T23:45:41.357 INFO:tasks.workunit.client.0.smithi031.stdout:7/331: fdatasync d9/d14/d22/f2d 0 2024-04-22T23:45:41.359 INFO:tasks.workunit.client.0.smithi031.stdout:5/341: creat db/d16/d2a/d59/f72 x:0 0 0 2024-04-22T23:45:41.360 INFO:tasks.workunit.client.0.smithi031.stdout:0/309: symlink d0/d1/d2/d36/d11/d1a/d56/l71 0 2024-04-22T23:45:41.361 INFO:tasks.workunit.client.0.smithi031.stdout:2/365: creat d6/d45/f73 x:0 0 0 2024-04-22T23:45:41.362 INFO:tasks.workunit.client.0.smithi031.stdout:0/310: write d0/d1/f6b [890811,94091] 0 2024-04-22T23:45:41.365 INFO:tasks.workunit.client.0.smithi031.stdout:5/342: fsync db/d16/d1d/f6b 0 2024-04-22T23:45:41.366 INFO:tasks.workunit.client.0.smithi031.stdout:0/311: chown d0/d1/d2/fa 246498941 1 2024-04-22T23:45:41.366 INFO:tasks.workunit.client.0.smithi031.stdout:4/474: dwrite db/d14/d2d/f37 [4194304,4194304] 0 2024-04-22T23:45:41.366 INFO:tasks.workunit.client.0.smithi031.stdout:6/434: dread d3/f18 [0,4194304] 0 2024-04-22T23:45:41.367 INFO:tasks.workunit.client.0.smithi031.stdout:8/354: rename d5/da/df/f1b to d5/da/df/d33/f65 0 2024-04-22T23:45:41.369 INFO:tasks.workunit.client.0.smithi031.stdout:6/435: truncate d3/d1f/f4f 778104 0 2024-04-22T23:45:41.372 INFO:tasks.workunit.client.0.smithi031.stdout:4/475: dread db/f24 [0,4194304] 0 2024-04-22T23:45:41.373 INFO:tasks.workunit.client.0.smithi031.stdout:1/290: dwrite d1/f6 [0,4194304] 0 2024-04-22T23:45:41.377 INFO:tasks.workunit.client.0.smithi031.stdout:4/476: dread db/d14/d2d/d49/d59/d83/d9c/fa0 [0,4194304] 0 2024-04-22T23:45:41.378 INFO:tasks.workunit.client.0.smithi031.stdout:5/343: creat db/d16/d1d/d24/d32/f73 x:0 0 0 2024-04-22T23:45:41.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/366: symlink d6/dc/d17/d69/l74 0 2024-04-22T23:45:41.383 INFO:tasks.workunit.client.0.smithi031.stdout:2/367: fsync d6/dc/d42/d24/f39 0 2024-04-22T23:45:41.384 INFO:tasks.workunit.client.0.smithi031.stdout:1/291: symlink d1/de/l68 0 2024-04-22T23:45:41.388 INFO:tasks.workunit.client.0.smithi031.stdout:5/344: mkdir db/d16/d2a/d36/d74 0 2024-04-22T23:45:41.390 INFO:tasks.workunit.client.0.smithi031.stdout:2/368: dwrite d6/dc/d42/d24/f34 [0,4194304] 0 2024-04-22T23:45:41.390 INFO:tasks.workunit.client.0.smithi031.stdout:8/355: symlink d5/da/d12/l66 0 2024-04-22T23:45:41.392 INFO:tasks.workunit.client.0.smithi031.stdout:1/292: mkdir d1/d21/d69 0 2024-04-22T23:45:41.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/293: dread - d1/db/d16/d2f/f50 zero size 2024-04-22T23:45:41.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/294: read - d1/de/d1e/d34/d41/f44 zero size 2024-04-22T23:45:41.398 INFO:tasks.workunit.client.0.smithi031.stdout:5/345: creat db/d16/d2a/d59/f75 x:0 0 0 2024-04-22T23:45:41.403 INFO:tasks.workunit.client.0.smithi031.stdout:1/295: mknod d1/de/d1e/c6a 0 2024-04-22T23:45:41.405 INFO:tasks.workunit.client.0.smithi031.stdout:9/369: rename d2/dc/f60 to d2/f6e 0 2024-04-22T23:45:41.407 INFO:tasks.workunit.client.0.smithi031.stdout:2/369: dwrite d6/d45/d50/f58 [0,4194304] 0 2024-04-22T23:45:41.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/356: creat d5/da/d12/d37/d3b/d5b/f67 x:0 0 0 2024-04-22T23:45:41.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/477: rename db/d14/d2d/d49/c7c to db/d31/d40/d43/caa 0 2024-04-22T23:45:41.409 INFO:tasks.workunit.client.0.smithi031.stdout:2/370: chown d6/dc/d42/d28/f54 0 1 2024-04-22T23:45:41.411 INFO:tasks.workunit.client.0.smithi031.stdout:8/357: fdatasync d5/da/d12/d2e/d4d/d56/f5a 0 2024-04-22T23:45:41.417 INFO:tasks.workunit.client.0.smithi031.stdout:9/370: mknod d2/dc/d3e/c6f 0 2024-04-22T23:45:41.418 INFO:tasks.workunit.client.0.smithi031.stdout:4/478: write db/d14/d2d/d49/f99 [512748,90999] 0 2024-04-22T23:45:41.418 INFO:tasks.workunit.client.0.smithi031.stdout:0/312: dread d0/d1/d2/d36/fd [0,4194304] 0 2024-04-22T23:45:41.418 INFO:tasks.workunit.client.0.smithi031.stdout:5/346: getdents db/d16/d1d/d24/d32/d62 0 2024-04-22T23:45:41.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/358: truncate d5/da/d12/d37/d34/f4c 905371 0 2024-04-22T23:45:41.421 INFO:tasks.workunit.client.0.smithi031.stdout:4/479: symlink db/d31/d40/d62/lab 0 2024-04-22T23:45:41.424 INFO:tasks.workunit.client.0.smithi031.stdout:0/313: read d0/d1/d2/fa [4892970,120942] 0 2024-04-22T23:45:41.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/371: read d2/dc/f4b [3735971,117038] 0 2024-04-22T23:45:41.424 INFO:tasks.workunit.client.0.smithi031.stdout:2/371: rename d6/dc/d17/l3f to d6/dc/d48/d4b/d6b/l75 0 2024-04-22T23:45:41.426 INFO:tasks.workunit.client.0.smithi031.stdout:5/347: creat db/d16/d1d/d24/f76 x:0 0 0 2024-04-22T23:45:41.427 INFO:tasks.workunit.client.0.smithi031.stdout:1/296: dwrite d1/d21/d29/f1b [4194304,4194304] 0 2024-04-22T23:45:41.428 INFO:tasks.workunit.client.0.smithi031.stdout:5/348: chown db/c68 54 1 2024-04-22T23:45:41.429 INFO:tasks.workunit.client.0.smithi031.stdout:8/359: symlink d5/da/d12/d37/d34/l68 0 2024-04-22T23:45:41.433 INFO:tasks.workunit.client.0.smithi031.stdout:4/480: symlink db/lac 0 2024-04-22T23:45:41.434 INFO:tasks.workunit.client.0.smithi031.stdout:2/372: chown d6/dc/d42/d24/c36 3504 1 2024-04-22T23:45:41.435 INFO:tasks.workunit.client.0.smithi031.stdout:9/372: mknod d2/dc/d21/d3d/c70 0 2024-04-22T23:45:41.438 INFO:tasks.workunit.client.0.smithi031.stdout:5/349: creat db/d16/d1d/d24/d58/d5d/f77 x:0 0 0 2024-04-22T23:45:41.438 INFO:tasks.workunit.client.0.smithi031.stdout:2/373: fdatasync d6/d45/d50/f58 0 2024-04-22T23:45:41.441 INFO:tasks.workunit.client.0.smithi031.stdout:9/373: mknod d2/d47/c71 0 2024-04-22T23:45:41.442 INFO:tasks.workunit.client.0.smithi031.stdout:0/314: dwrite d0/d1/f6b [0,4194304] 0 2024-04-22T23:45:41.446 INFO:tasks.workunit.client.0.smithi031.stdout:9/374: write d2/f20 [245105,51150] 0 2024-04-22T23:45:41.446 INFO:tasks.workunit.client.0.smithi031.stdout:4/481: rename db/d14/d1d/d7b/d7f/d2a/da1/fa9 to db/d14/fad 0 2024-04-22T23:45:41.450 INFO:tasks.workunit.client.0.smithi031.stdout:8/360: dwrite d5/f26 [0,4194304] 0 2024-04-22T23:45:41.451 INFO:tasks.workunit.client.0.smithi031.stdout:5/350: mkdir db/d16/d1d/d24/d58/d78 0 2024-04-22T23:45:41.451 INFO:tasks.workunit.client.0.smithi031.stdout:1/297: dread d1/d21/d29/d28/f2a [0,4194304] 0 2024-04-22T23:45:41.452 INFO:tasks.workunit.client.0.smithi031.stdout:2/374: creat d6/d45/d68/d6c/f76 x:0 0 0 2024-04-22T23:45:41.457 INFO:tasks.workunit.client.0.smithi031.stdout:5/351: write db/d16/d2a/d36/f38 [1908675,39990] 0 2024-04-22T23:45:41.457 INFO:tasks.workunit.client.0.smithi031.stdout:0/315: write d0/d1/d2/d36/d11/d1a/d3c/f40 [1814713,16389] 0 2024-04-22T23:45:41.458 INFO:tasks.workunit.client.0.smithi031.stdout:2/375: write d6/dc/d42/f61 [951021,123396] 0 2024-04-22T23:45:41.459 INFO:tasks.workunit.client.0.smithi031.stdout:1/298: fdatasync d1/d21/d29/d28/f2a 0 2024-04-22T23:45:41.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/375: read d2/d47/d4a/f49 [198218,114310] 0 2024-04-22T23:45:41.463 INFO:tasks.workunit.client.0.smithi031.stdout:4/482: unlink db/d14/d2d/d49/d59/d6a/d41/f7e 0 2024-04-22T23:45:41.466 INFO:tasks.workunit.client.0.smithi031.stdout:2/376: creat d6/d45/f77 x:0 0 0 2024-04-22T23:45:41.470 INFO:tasks.workunit.client.0.smithi031.stdout:2/377: dread d6/dc/d57/f4d [0,4194304] 0 2024-04-22T23:45:41.470 INFO:tasks.workunit.client.0.smithi031.stdout:1/299: unlink d1/d21/l2d 0 2024-04-22T23:45:41.471 INFO:tasks.workunit.client.0.smithi031.stdout:5/352: creat db/d16/d1d/d24/d32/f79 x:0 0 0 2024-04-22T23:45:41.474 INFO:tasks.workunit.client.0.smithi031.stdout:4/483: creat db/d14/d1d/d7b/d7f/fae x:0 0 0 2024-04-22T23:45:41.476 INFO:tasks.workunit.client.0.smithi031.stdout:4/484: truncate db/d14/d1d/d7b/d7f/f32 4858660 0 2024-04-22T23:45:41.478 INFO:tasks.workunit.client.0.smithi031.stdout:2/378: write f3 [4537738,17423] 0 2024-04-22T23:45:41.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/300: symlink d1/d3f/d54/l6b 0 2024-04-22T23:45:41.483 INFO:tasks.workunit.client.0.smithi031.stdout:4/485: mknod db/d31/d40/d62/caf 0 2024-04-22T23:45:41.487 INFO:tasks.workunit.client.0.smithi031.stdout:1/301: rename d1/db/c24 to d1/de/d1e/d34/d4d/c6c 0 2024-04-22T23:45:41.490 INFO:tasks.workunit.client.0.smithi031.stdout:5/353: symlink db/d16/d1d/d24/d58/d78/l7a 0 2024-04-22T23:45:41.491 INFO:tasks.workunit.client.0.smithi031.stdout:5/354: chown db/d16/f3e 20749 1 2024-04-22T23:45:41.491 INFO:tasks.workunit.client.0.smithi031.stdout:4/486: creat db/d14/d1d/fb0 x:0 0 0 2024-04-22T23:45:41.493 INFO:tasks.workunit.client.0.smithi031.stdout:0/316: getdents d0/d1/d2/d36 0 2024-04-22T23:45:41.497 INFO:tasks.workunit.client.0.smithi031.stdout:9/376: getdents d2/d47/d4a 0 2024-04-22T23:45:41.502 INFO:tasks.workunit.client.0.smithi031.stdout:5/355: rename db/d16/d2a/f37 to db/d16/d1d/d24/d58/d78/f7b 0 2024-04-22T23:45:41.504 INFO:tasks.workunit.client.0.smithi031.stdout:4/487: dwrite db/f24 [0,4194304] 0 2024-04-22T23:45:41.504 INFO:tasks.workunit.client.0.smithi031.stdout:1/302: creat d1/d47/d63/f6d x:0 0 0 2024-04-22T23:45:41.505 INFO:tasks.workunit.client.0.smithi031.stdout:9/377: mknod d2/d15/d5a/c72 0 2024-04-22T23:45:41.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/317: unlink d0/ff 0 2024-04-22T23:45:41.507 INFO:tasks.workunit.client.0.smithi031.stdout:8/361: write d5/da/d12/d1d/d38/f3f [567163,16761] 0 2024-04-22T23:45:41.509 INFO:tasks.workunit.client.0.smithi031.stdout:2/379: getdents d6/dc/d42 0 2024-04-22T23:45:41.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/356: mkdir db/d16/d2a/d7c 0 2024-04-22T23:45:41.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/318: creat d0/d1/d2/d36/d11/d1c/d54/f72 x:0 0 0 2024-04-22T23:45:41.516 INFO:tasks.workunit.client.0.smithi031.stdout:8/362: dwrite d5/f26 [4194304,4194304] 0 2024-04-22T23:45:41.518 INFO:tasks.workunit.client.0.smithi031.stdout:2/380: mkdir d6/dc/d17/d69/d78 0 2024-04-22T23:45:41.521 INFO:tasks.workunit.client.0.smithi031.stdout:9/378: rename d2/ce to d2/d15/d5a/c73 0 2024-04-22T23:45:41.524 INFO:tasks.workunit.client.0.smithi031.stdout:8/363: dwrite d5/da/d12/d37/d49/f4f [0,4194304] 0 2024-04-22T23:45:41.525 INFO:tasks.workunit.client.0.smithi031.stdout:1/303: dread d1/f1d [0,4194304] 0 2024-04-22T23:45:41.526 INFO:tasks.workunit.client.0.smithi031.stdout:1/304: stat d1/d35/d36/c59 0 2024-04-22T23:45:41.527 INFO:tasks.workunit.client.0.smithi031.stdout:2/381: mkdir d6/dc/d42/d24/d32/d79 0 2024-04-22T23:45:41.528 INFO:tasks.workunit.client.0.smithi031.stdout:0/319: symlink d0/d13/d18/d6c/l73 0 2024-04-22T23:45:41.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/320: write d0/d13/d18/f6d [1271442,130025] 0 2024-04-22T23:45:41.532 INFO:tasks.workunit.client.0.smithi031.stdout:4/488: getdents db 0 2024-04-22T23:45:41.534 INFO:tasks.workunit.client.0.smithi031.stdout:2/382: mknod d6/d45/c7a 0 2024-04-22T23:45:41.538 INFO:tasks.workunit.client.0.smithi031.stdout:9/379: dwrite d2/f2e [0,4194304] 0 2024-04-22T23:45:41.541 INFO:tasks.workunit.client.0.smithi031.stdout:4/489: dwrite db/d14/d2d/f35 [0,4194304] 0 2024-04-22T23:45:41.545 INFO:tasks.workunit.client.0.smithi031.stdout:1/305: mknod d1/d35/d36/d5a/c6e 0 2024-04-22T23:45:41.548 INFO:tasks.workunit.client.0.smithi031.stdout:4/490: unlink db/d14/d2d/d49/d59/d83/d9c/f6f 0 2024-04-22T23:45:41.550 INFO:tasks.workunit.client.0.smithi031.stdout:1/306: mkdir d1/d21/d4b/d6f 0 2024-04-22T23:45:41.554 INFO:tasks.workunit.client.0.smithi031.stdout:1/307: mknod d1/de/d1e/d34/c70 0 2024-04-22T23:45:41.559 INFO:tasks.workunit.client.0.smithi031.stdout:4/491: dwrite db/d14/d2d/d49/d59/d6a/f92 [4194304,4194304] 0 2024-04-22T23:45:41.561 INFO:tasks.workunit.client.0.smithi031.stdout:1/308: getdents d1/d35/d36 0 2024-04-22T23:45:41.564 INFO:tasks.workunit.client.0.smithi031.stdout:4/492: creat db/d14/d1d/d7b/d7f/d2a/fb1 x:0 0 0 2024-04-22T23:45:41.565 INFO:tasks.workunit.client.0.smithi031.stdout:9/380: dread d2/d47/d4a/f49 [0,4194304] 0 2024-04-22T23:45:41.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/493: creat db/d31/d40/fb2 x:0 0 0 2024-04-22T23:45:41.568 INFO:tasks.workunit.client.0.smithi031.stdout:8/364: dread d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:41.571 INFO:tasks.workunit.client.0.smithi031.stdout:9/381: creat d2/dc/d6d/f74 x:0 0 0 2024-04-22T23:45:41.573 INFO:tasks.workunit.client.0.smithi031.stdout:0/321: dread d0/d1/d2/d36/d11/d1a/d3c/f4d [0,4194304] 0 2024-04-22T23:45:41.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/309: dwrite d1/db/d16/d2f/f50 [0,4194304] 0 2024-04-22T23:45:41.575 INFO:tasks.workunit.client.0.smithi031.stdout:1/310: readlink d1/d21/d29/l5b 0 2024-04-22T23:45:41.580 INFO:tasks.workunit.client.0.smithi031.stdout:0/322: creat d0/d13/d18/d6c/f74 x:0 0 0 2024-04-22T23:45:41.580 INFO:tasks.workunit.client.0.smithi031.stdout:8/365: link d5/da/d12/d1d/l22 d5/l69 0 2024-04-22T23:45:41.585 INFO:tasks.workunit.client.0.smithi031.stdout:9/382: rename d2/d47 to d2/dd/d75 0 2024-04-22T23:45:41.586 INFO:tasks.workunit.client.0.smithi031.stdout:0/323: rename d0/d1/d2/d36/d11/d1a/d3c to d0/d1/d2/d36/d11/d1a/d3c/d75 22 2024-04-22T23:45:41.586 INFO:tasks.workunit.client.0.smithi031.stdout:9/383: truncate d2/f6e 666021 0 2024-04-22T23:45:41.587 INFO:tasks.workunit.client.0.smithi031.stdout:1/311: creat d1/d21/d4e/f71 x:0 0 0 2024-04-22T23:45:41.589 INFO:tasks.workunit.client.0.smithi031.stdout:9/384: truncate d2/dc/d21/f58 525362 0 2024-04-22T23:45:41.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/324: fsync d0/d1/d2/fa 0 2024-04-22T23:45:41.590 INFO:tasks.workunit.client.0.smithi031.stdout:9/385: write d2/dd/d75/d4a/f34 [1275706,123764] 0 2024-04-22T23:45:41.592 INFO:tasks.workunit.client.0.smithi031.stdout:8/366: dread d5/da/d12/f14 [0,4194304] 0 2024-04-22T23:45:41.596 INFO:tasks.workunit.client.0.smithi031.stdout:1/312: dwrite d1/de/d1e/f64 [0,4194304] 0 2024-04-22T23:45:41.596 INFO:tasks.workunit.client.0.smithi031.stdout:8/367: read d5/da/df/f13 [22398,43104] 0 2024-04-22T23:45:41.634 INFO:tasks.workunit.client.0.smithi031.stdout:4/494: truncate db/d14/d1d/f25 156950 0 2024-04-22T23:45:41.635 INFO:tasks.workunit.client.0.smithi031.stdout:9/386: creat d2/dc/d21/d3d/d40/f76 x:0 0 0 2024-04-22T23:45:41.636 INFO:tasks.workunit.client.0.smithi031.stdout:9/387: fdatasync d2/dd/d75/f5e 0 2024-04-22T23:45:41.638 INFO:tasks.workunit.client.0.smithi031.stdout:1/313: rename d1/d35/d36/d5a/c67 to d1/d21/d69/c72 0 2024-04-22T23:45:41.641 INFO:tasks.workunit.client.0.smithi031.stdout:1/314: chown d1/db/d16/c37 4390 1 2024-04-22T23:45:41.642 INFO:tasks.workunit.client.0.smithi031.stdout:4/495: getdents db/d14/d2d/d49/d59 0 2024-04-22T23:45:41.645 INFO:tasks.workunit.client.0.smithi031.stdout:1/315: symlink d1/d47/d63/l73 0 2024-04-22T23:45:41.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/316: dwrite d1/d47/d63/f6d [0,4194304] 0 2024-04-22T23:45:41.652 INFO:tasks.workunit.client.0.smithi031.stdout:4/496: rmdir db/d14/d1d/d7b/d7f 39 2024-04-22T23:45:41.654 INFO:tasks.workunit.client.0.smithi031.stdout:9/388: link d2/dc/f10 d2/dd/d75/d64/d5d/f77 0 2024-04-22T23:45:41.662 INFO:tasks.workunit.client.0.smithi031.stdout:9/389: creat d2/dd/d75/d64/d5d/d59/f78 x:0 0 0 2024-04-22T23:45:41.663 INFO:tasks.workunit.client.0.smithi031.stdout:1/317: rename d1/l4 to d1/db/d16/l74 0 2024-04-22T23:45:41.667 INFO:tasks.workunit.client.0.smithi031.stdout:4/497: rename db/fa4 to db/d14/d2d/d49/fb3 0 2024-04-22T23:45:41.678 INFO:tasks.workunit.client.0.smithi031.stdout:9/390: dread d2/dc/f1e [0,4194304] 0 2024-04-22T23:45:41.679 INFO:tasks.workunit.client.0.smithi031.stdout:4/498: dwrite f9 [4194304,4194304] 0 2024-04-22T23:45:41.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/297: sync 2024-04-22T23:45:41.687 INFO:tasks.workunit.client.0.smithi031.stdout:1/318: dwrite d1/d21/d29/f23 [0,4194304] 0 2024-04-22T23:45:41.687 INFO:tasks.workunit.client.0.smithi031.stdout:3/298: creat d4/d49/f5e x:0 0 0 2024-04-22T23:45:41.694 INFO:tasks.workunit.client.0.smithi031.stdout:4/499: unlink db/d14/d1d/f20 0 2024-04-22T23:45:41.695 INFO:tasks.workunit.client.0.smithi031.stdout:9/391: dwrite d2/dc/f48 [0,4194304] 0 2024-04-22T23:45:41.698 INFO:tasks.workunit.client.0.smithi031.stdout:1/319: dread d1/db/d16/d2f/f50 [0,4194304] 0 2024-04-22T23:45:41.706 INFO:tasks.workunit.client.0.smithi031.stdout:1/320: fsync d1/db/d16/d2f/f50 0 2024-04-22T23:45:41.709 INFO:tasks.workunit.client.0.smithi031.stdout:3/299: dwrite d4/f1e [0,4194304] 0 2024-04-22T23:45:41.711 INFO:tasks.workunit.client.0.smithi031.stdout:1/321: dread d1/d21/d29/f1b [0,4194304] 0 2024-04-22T23:45:41.715 INFO:tasks.workunit.client.0.smithi031.stdout:3/300: rename d4/d6/d4c/d4d/d5b to d4/d6/d4c/d4d/d5f 0 2024-04-22T23:45:41.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/322: creat d1/db/d16/d2f/f75 x:0 0 0 2024-04-22T23:45:41.719 INFO:tasks.workunit.client.0.smithi031.stdout:1/323: mknod d1/d3f/d54/c76 0 2024-04-22T23:45:41.720 INFO:tasks.workunit.client.0.smithi031.stdout:9/392: dwrite d2/dd/f2b [0,4194304] 0 2024-04-22T23:45:41.721 INFO:tasks.workunit.client.0.smithi031.stdout:3/301: creat d4/d10/d2e/d45/f60 x:0 0 0 2024-04-22T23:45:41.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/324: symlink d1/de/d1e/d34/d41/l77 0 2024-04-22T23:45:41.726 INFO:tasks.workunit.client.0.smithi031.stdout:3/302: mkdir d4/d54/d56/d61 0 2024-04-22T23:45:41.727 INFO:tasks.workunit.client.0.smithi031.stdout:1/325: fsync d1/de/d1e/f64 0 2024-04-22T23:45:41.729 INFO:tasks.workunit.client.0.smithi031.stdout:4/500: dread db/d14/d1d/f76 [0,4194304] 0 2024-04-22T23:45:41.733 INFO:tasks.workunit.client.0.smithi031.stdout:9/393: dwrite d2/f20 [0,4194304] 0 2024-04-22T23:45:41.734 INFO:tasks.workunit.client.0.smithi031.stdout:1/326: dread d1/d47/d63/f6d [0,4194304] 0 2024-04-22T23:45:41.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/394: dwrite d2/dd/d75/d64/d5d/f57 [0,4194304] 0 2024-04-22T23:45:41.743 INFO:tasks.workunit.client.0.smithi031.stdout:9/395: write d2/dc/d21/d3d/d40/f5c [198743,56143] 0 2024-04-22T23:45:41.743 INFO:tasks.workunit.client.0.smithi031.stdout:3/303: link d4/f1e d4/d10/f62 0 2024-04-22T23:45:41.744 INFO:tasks.workunit.client.0.smithi031.stdout:1/327: symlink d1/d21/d29/d28/l78 0 2024-04-22T23:45:41.748 INFO:tasks.workunit.client.0.smithi031.stdout:4/501: rename db/d14/d2d/d49/d59/d83/d9c/l5a to db/d31/d40/d62/lb4 0 2024-04-22T23:45:41.753 INFO:tasks.workunit.client.0.smithi031.stdout:1/328: truncate d1/f46 20983 0 2024-04-22T23:45:41.754 INFO:tasks.workunit.client.0.smithi031.stdout:9/396: symlink d2/dd/d75/d64/l79 0 2024-04-22T23:45:41.755 INFO:tasks.workunit.client.0.smithi031.stdout:7/332: sync 2024-04-22T23:45:41.757 INFO:tasks.workunit.client.0.smithi031.stdout:1/329: truncate d1/d21/d4b/f5d 802191 0 2024-04-22T23:45:41.757 INFO:tasks.workunit.client.0.smithi031.stdout:9/397: symlink d2/dd/l7a 0 2024-04-22T23:45:41.757 INFO:tasks.workunit.client.0.smithi031.stdout:7/333: truncate d9/f4b 920674 0 2024-04-22T23:45:41.758 INFO:tasks.workunit.client.0.smithi031.stdout:4/502: rename db/d14/d1d/d7b/d7f/d2a/d30/l34 to db/d14/d1d/d7b/d7f/lb5 0 2024-04-22T23:45:41.759 INFO:tasks.workunit.client.0.smithi031.stdout:1/330: mknod d1/d21/c79 0 2024-04-22T23:45:41.760 INFO:tasks.workunit.client.0.smithi031.stdout:9/398: truncate d2/dc/d21/d3d/d40/f76 461085 0 2024-04-22T23:45:41.760 INFO:tasks.workunit.client.0.smithi031.stdout:7/334: creat d9/d14/f75 x:0 0 0 2024-04-22T23:45:41.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/331: symlink d1/de/d1e/d34/l7a 0 2024-04-22T23:45:41.768 INFO:tasks.workunit.client.0.smithi031.stdout:7/335: dwrite d9/fa [4194304,4194304] 0 2024-04-22T23:45:41.769 INFO:tasks.workunit.client.0.smithi031.stdout:4/503: link db/d14/d2d/d49/d59/d83/d9c/f55 db/d14/d2d/fb6 0 2024-04-22T23:45:41.769 INFO:tasks.workunit.client.0.smithi031.stdout:1/332: dwrite d1/de/d1e/d34/d41/f44 [0,4194304] 0 2024-04-22T23:45:41.771 INFO:tasks.workunit.client.0.smithi031.stdout:7/336: write f2 [2821559,44465] 0 2024-04-22T23:45:41.771 INFO:tasks.workunit.client.0.smithi031.stdout:9/399: symlink d2/dd/d65/l7b 0 2024-04-22T23:45:41.773 INFO:tasks.workunit.client.0.smithi031.stdout:4/504: chown db/d14/d1d/d7b/d7f/d2a/d30/c94 23 1 2024-04-22T23:45:41.774 INFO:tasks.workunit.client.0.smithi031.stdout:7/337: chown d9/d14/d22/d5c/d5f 0 1 2024-04-22T23:45:41.776 INFO:tasks.workunit.client.0.smithi031.stdout:1/333: read d1/de/d1e/d34/d41/f42 [1391154,125398] 0 2024-04-22T23:45:41.781 INFO:tasks.workunit.client.0.smithi031.stdout:7/338: dwrite d9/d15/f49 [0,4194304] 0 2024-04-22T23:45:41.784 INFO:tasks.workunit.client.0.smithi031.stdout:3/304: dwrite d4/d6/d11/f19 [0,4194304] 0 2024-04-22T23:45:41.786 INFO:tasks.workunit.client.0.smithi031.stdout:1/334: creat d1/d35/f7b x:0 0 0 2024-04-22T23:45:41.790 INFO:tasks.workunit.client.0.smithi031.stdout:3/305: chown d4/d6/d11 3420 1 2024-04-22T23:45:41.791 INFO:tasks.workunit.client.0.smithi031.stdout:7/339: creat d9/d14/d1f/d45/f76 x:0 0 0 2024-04-22T23:45:41.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/340: dread - d9/d14/d22/d5b/f6e zero size 2024-04-22T23:45:41.802 INFO:tasks.workunit.client.0.smithi031.stdout:7/341: symlink d9/l77 0 2024-04-22T23:45:41.803 INFO:tasks.workunit.client.0.smithi031.stdout:7/342: write d9/d15/f49 [3701692,84998] 0 2024-04-22T23:45:41.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/306: creat d4/d6/d4c/d4d/d5f/f63 x:0 0 0 2024-04-22T23:45:41.807 INFO:tasks.workunit.client.0.smithi031.stdout:9/400: write d2/dd/d75/d64/f55 [3963538,4787] 0 2024-04-22T23:45:41.809 INFO:tasks.workunit.client.0.smithi031.stdout:7/343: creat d9/d14/d22/f78 x:0 0 0 2024-04-22T23:45:41.814 INFO:tasks.workunit.client.0.smithi031.stdout:7/344: mknod d9/d14/d1f/d45/c79 0 2024-04-22T23:45:41.815 INFO:tasks.workunit.client.0.smithi031.stdout:9/401: link d2/dc/d21/d3d/d40/f39 d2/d15/f7c 0 2024-04-22T23:45:41.816 INFO:tasks.workunit.client.0.smithi031.stdout:1/335: dwrite d1/d21/d29/d28/f2a [0,4194304] 0 2024-04-22T23:45:41.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/402: symlink d2/dd/d75/d4a/l7d 0 2024-04-22T23:45:41.821 INFO:tasks.workunit.client.0.smithi031.stdout:9/403: dread - d2/dd/d75/d4a/f27 zero size 2024-04-22T23:45:41.825 INFO:tasks.workunit.client.0.smithi031.stdout:3/307: dread d4/d10/f1b [0,4194304] 0 2024-04-22T23:45:41.826 INFO:tasks.workunit.client.0.smithi031.stdout:3/308: stat d4/d10/f1b 0 2024-04-22T23:45:41.829 INFO:tasks.workunit.client.0.smithi031.stdout:9/404: creat d2/dd/d75/f7e x:0 0 0 2024-04-22T23:45:41.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/336: fsync d1/d2e/f3e 0 2024-04-22T23:45:41.833 INFO:tasks.workunit.client.0.smithi031.stdout:3/309: mknod d4/d6/d4c/d4d/d5f/c64 0 2024-04-22T23:45:41.834 INFO:tasks.workunit.client.0.smithi031.stdout:3/310: truncate d4/d10/d2e/f4f 933428 0 2024-04-22T23:45:41.834 INFO:tasks.workunit.client.0.smithi031.stdout:9/405: mknod d2/dd/d75/c7f 0 2024-04-22T23:45:41.835 INFO:tasks.workunit.client.0.smithi031.stdout:9/406: chown d2/dd/d75 2077260 1 2024-04-22T23:45:41.839 INFO:tasks.workunit.client.0.smithi031.stdout:1/337: unlink d1/c57 0 2024-04-22T23:45:41.843 INFO:tasks.workunit.client.0.smithi031.stdout:7/345: getdents d9/d14 0 2024-04-22T23:45:41.847 INFO:tasks.workunit.client.0.smithi031.stdout:9/407: mkdir d2/dd/d75/d80 0 2024-04-22T23:45:41.847 INFO:tasks.workunit.client.0.smithi031.stdout:3/311: chown d4/d10/d2e/c4b 0 1 2024-04-22T23:45:41.849 INFO:tasks.workunit.client.0.smithi031.stdout:3/312: write d4/d10/f2d [1438652,95075] 0 2024-04-22T23:45:41.849 INFO:tasks.workunit.client.0.smithi031.stdout:1/338: read d1/d21/d29/f1a [3975897,72884] 0 2024-04-22T23:45:41.856 INFO:tasks.workunit.client.0.smithi031.stdout:3/313: creat d4/d54/d56/f65 x:0 0 0 2024-04-22T23:45:41.858 INFO:tasks.workunit.client.0.smithi031.stdout:1/339: mknod d1/d3f/c7c 0 2024-04-22T23:45:41.859 INFO:tasks.workunit.client.0.smithi031.stdout:9/408: symlink d2/dd/d75/d80/l81 0 2024-04-22T23:45:41.859 INFO:tasks.workunit.client.0.smithi031.stdout:9/409: stat d2/d15/c4e 0 2024-04-22T23:45:41.860 INFO:tasks.workunit.client.0.smithi031.stdout:7/346: dread d9/d15/f21 [0,4194304] 0 2024-04-22T23:45:41.861 INFO:tasks.workunit.client.0.smithi031.stdout:1/340: write d1/db/d16/d1f/f26 [1939188,40954] 0 2024-04-22T23:45:41.861 INFO:tasks.workunit.client.0.smithi031.stdout:7/347: write d9/d14/d22/f52 [842026,105407] 0 2024-04-22T23:45:41.866 INFO:tasks.workunit.client.0.smithi031.stdout:1/341: mkdir d1/d47/d7d 0 2024-04-22T23:45:41.867 INFO:tasks.workunit.client.0.smithi031.stdout:7/348: chown d9/d14/d1f/l4d 108778 1 2024-04-22T23:45:41.868 INFO:tasks.workunit.client.0.smithi031.stdout:3/314: link d4/d54/c57 d4/d10/d2e/d45/c66 0 2024-04-22T23:45:41.868 INFO:tasks.workunit.client.0.smithi031.stdout:3/315: stat d4 0 2024-04-22T23:45:41.878 INFO:tasks.workunit.client.0.smithi031.stdout:5/357: sync 2024-04-22T23:45:41.878 INFO:tasks.workunit.client.0.smithi031.stdout:6/436: sync 2024-04-22T23:45:41.878 INFO:tasks.workunit.client.0.smithi031.stdout:8/368: sync 2024-04-22T23:45:41.878 INFO:tasks.workunit.client.0.smithi031.stdout:2/383: sync 2024-04-22T23:45:41.878 INFO:tasks.workunit.client.0.smithi031.stdout:0/325: sync 2024-04-22T23:45:41.892 INFO:tasks.workunit.client.0.smithi031.stdout:3/316: creat d4/d6/da/f67 x:0 0 0 2024-04-22T23:45:41.892 INFO:tasks.workunit.client.0.smithi031.stdout:7/349: chown d9/d15/l43 20133861 1 2024-04-22T23:45:41.894 INFO:tasks.workunit.client.0.smithi031.stdout:7/350: stat d9/d15/d3a 0 2024-04-22T23:45:41.897 INFO:tasks.workunit.client.0.smithi031.stdout:3/317: fdatasync d4/d10/f2b 0 2024-04-22T23:45:41.949 INFO:tasks.workunit.client.0.smithi031.stdout:0/326: dread d0/d13/d18/d1e/f6a [0,4194304] 0 2024-04-22T23:45:41.950 INFO:tasks.workunit.client.0.smithi031.stdout:1/342: rmdir d1 39 2024-04-22T23:45:41.951 INFO:tasks.workunit.client.0.smithi031.stdout:5/358: mkdir db/d16/d1d/d24/d7d 0 2024-04-22T23:45:41.951 INFO:tasks.workunit.client.0.smithi031.stdout:7/351: mkdir d9/d14/d22/d28/d7a 0 2024-04-22T23:45:41.951 INFO:tasks.workunit.client.0.smithi031.stdout:0/327: readlink d0/d13/d18/l24 0 2024-04-22T23:45:41.952 INFO:tasks.workunit.client.0.smithi031.stdout:8/369: dwrite d5/da/d12/d37/d49/f4f [0,4194304] 0 2024-04-22T23:45:41.952 INFO:tasks.workunit.client.0.smithi031.stdout:6/437: mkdir d3/d2a/d85/d96 0 2024-04-22T23:45:41.952 INFO:tasks.workunit.client.0.smithi031.stdout:3/318: dread - d4/d6/d4c/d4d/d5f/f63 zero size 2024-04-22T23:45:41.958 INFO:tasks.workunit.client.0.smithi031.stdout:6/438: mknod d3/d35/d75/d7d/c97 0 2024-04-22T23:45:41.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/370: symlink d5/da/d12/d2f/l6a 0 2024-04-22T23:45:41.959 INFO:tasks.workunit.client.0.smithi031.stdout:3/319: creat d4/d6/d4c/d4d/d5f/f68 x:0 0 0 2024-04-22T23:45:41.962 INFO:tasks.workunit.client.0.smithi031.stdout:1/343: truncate d1/d21/d29/f1a 453163 0 2024-04-22T23:45:41.963 INFO:tasks.workunit.client.0.smithi031.stdout:6/439: creat d3/d2a/d3c/f98 x:0 0 0 2024-04-22T23:45:41.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/371: unlink d5/f26 0 2024-04-22T23:45:41.967 INFO:tasks.workunit.client.0.smithi031.stdout:3/320: creat d4/d6/d4c/d4d/f69 x:0 0 0 2024-04-22T23:45:41.967 INFO:tasks.workunit.client.0.smithi031.stdout:3/321: chown d4/d6/d11/d3e 0 1 2024-04-22T23:45:41.973 INFO:tasks.workunit.client.0.smithi031.stdout:8/372: readlink d5/da/d12/d1d/l22 0 2024-04-22T23:45:41.975 INFO:tasks.workunit.client.0.smithi031.stdout:7/352: dread d9/d15/f17 [0,4194304] 0 2024-04-22T23:45:41.975 INFO:tasks.workunit.client.0.smithi031.stdout:3/322: dwrite d4/d6/d4c/d4d/f69 [0,4194304] 0 2024-04-22T23:45:41.985 INFO:tasks.workunit.client.0.smithi031.stdout:7/353: rename d9/d14/d1f/d47/l6f to d9/d14/d22/d28/d7a/l7b 0 2024-04-22T23:45:41.986 INFO:tasks.workunit.client.0.smithi031.stdout:8/373: mkdir d5/d48/d61/d6b 0 2024-04-22T23:45:41.987 INFO:tasks.workunit.client.0.smithi031.stdout:6/440: dread d3/f1a [0,4194304] 0 2024-04-22T23:45:41.991 INFO:tasks.workunit.client.0.smithi031.stdout:3/323: unlink d4/d54/c57 0 2024-04-22T23:45:41.996 INFO:tasks.workunit.client.0.smithi031.stdout:7/354: creat d9/d14/d1f/d47/f7c x:0 0 0 2024-04-22T23:45:41.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/505: sync 2024-04-22T23:45:41.998 INFO:tasks.workunit.client.0.smithi031.stdout:9/410: sync 2024-04-22T23:45:41.998 INFO:tasks.workunit.client.0.smithi031.stdout:8/374: fsync d5/fb 0 2024-04-22T23:45:41.998 INFO:tasks.workunit.client.0.smithi031.stdout:3/324: dwrite d4/d6/d4c/d4d/d5f/f63 [0,4194304] 0 2024-04-22T23:45:42.001 INFO:tasks.workunit.client.0.smithi031.stdout:6/441: dread d3/d2a/d3c/d40/f44 [0,4194304] 0 2024-04-22T23:45:42.005 INFO:tasks.workunit.client.0.smithi031.stdout:4/506: mkdir db/d14/d2d/db7 0 2024-04-22T23:45:42.009 INFO:tasks.workunit.client.0.smithi031.stdout:6/442: symlink d3/d2a/d3c/d65/l99 0 2024-04-22T23:45:42.011 INFO:tasks.workunit.client.0.smithi031.stdout:9/411: fdatasync d2/dc/f12 0 2024-04-22T23:45:42.012 INFO:tasks.workunit.client.0.smithi031.stdout:9/412: dread - d2/dd/d75/f7e zero size 2024-04-22T23:45:42.013 INFO:tasks.workunit.client.0.smithi031.stdout:7/355: dread d9/f1c [0,4194304] 0 2024-04-22T23:45:42.013 INFO:tasks.workunit.client.0.smithi031.stdout:9/413: chown d2/dc/d3e/f6c 0 1 2024-04-22T23:45:42.013 INFO:tasks.workunit.client.0.smithi031.stdout:8/375: mkdir d5/d48/d61/d6b/d6c 0 2024-04-22T23:45:42.014 INFO:tasks.workunit.client.0.smithi031.stdout:7/356: dread - d9/d15/d6d/d65/f72 zero size 2024-04-22T23:45:42.016 INFO:tasks.workunit.client.0.smithi031.stdout:3/325: creat d4/f6a x:0 0 0 2024-04-22T23:45:42.017 INFO:tasks.workunit.client.0.smithi031.stdout:6/443: creat d3/d2a/d3c/d40/d7a/f9a x:0 0 0 2024-04-22T23:45:42.020 INFO:tasks.workunit.client.0.smithi031.stdout:4/507: link db/d14/d1d/d7b/d7f/d2a/d30/f5e db/d14/d2d/db7/fb8 0 2024-04-22T23:45:42.020 INFO:tasks.workunit.client.0.smithi031.stdout:7/357: dwrite d9/d15/d6d/d65/f6b [0,4194304] 0 2024-04-22T23:45:42.021 INFO:tasks.workunit.client.0.smithi031.stdout:8/376: creat d5/da/d12/d37/d3b/f6d x:0 0 0 2024-04-22T23:45:42.021 INFO:tasks.workunit.client.0.smithi031.stdout:7/358: stat d9/d14/d22/d28/f36 0 2024-04-22T23:45:42.022 INFO:tasks.workunit.client.0.smithi031.stdout:7/359: write d9/f1c [5167013,75129] 0 2024-04-22T23:45:42.023 INFO:tasks.workunit.client.0.smithi031.stdout:8/377: chown d5/da/d12/d37/d34/f4c 1921342321 1 2024-04-22T23:45:42.026 INFO:tasks.workunit.client.0.smithi031.stdout:3/326: symlink d4/d6/d11/d25/d44/l6b 0 2024-04-22T23:45:42.029 INFO:tasks.workunit.client.0.smithi031.stdout:6/444: mknod d3/d2a/d3c/d40/d5b/c9b 0 2024-04-22T23:45:42.029 INFO:tasks.workunit.client.0.smithi031.stdout:3/327: write d4/d6/d11/d25/f36 [362862,29564] 0 2024-04-22T23:45:42.030 INFO:tasks.workunit.client.0.smithi031.stdout:7/360: dwrite d9/d14/f20 [0,4194304] 0 2024-04-22T23:45:42.030 INFO:tasks.workunit.client.0.smithi031.stdout:6/445: dread d3/d2a/d3c/d40/d5b/f63 [0,4194304] 0 2024-04-22T23:45:42.030 INFO:tasks.workunit.client.0.smithi031.stdout:4/508: mknod db/d14/d1d/d7b/d7f/cb9 0 2024-04-22T23:45:42.031 INFO:tasks.workunit.client.0.smithi031.stdout:6/446: fdatasync d3/d1f/f64 0 2024-04-22T23:45:42.032 INFO:tasks.workunit.client.0.smithi031.stdout:9/414: link d2/f20 d2/dd/d75/d80/f82 0 2024-04-22T23:45:42.035 INFO:tasks.workunit.client.0.smithi031.stdout:8/378: mknod d5/da/d12/d37/d3b/d5b/c6e 0 2024-04-22T23:45:42.036 INFO:tasks.workunit.client.0.smithi031.stdout:3/328: creat d4/d10/d2e/d45/f6c x:0 0 0 2024-04-22T23:45:42.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/361: creat d9/d14/d22/d28/d7a/f7d x:0 0 0 2024-04-22T23:45:42.043 INFO:tasks.workunit.client.0.smithi031.stdout:3/329: dwrite d4/f6a [0,4194304] 0 2024-04-22T23:45:42.046 INFO:tasks.workunit.client.0.smithi031.stdout:7/362: write d9/d15/f17 [1761412,12701] 0 2024-04-22T23:45:42.048 INFO:tasks.workunit.client.0.smithi031.stdout:7/363: read - d9/d14/d22/d5b/f6e zero size 2024-04-22T23:45:42.052 INFO:tasks.workunit.client.0.smithi031.stdout:0/328: write d0/d1/f6b [4781617,65909] 0 2024-04-22T23:45:42.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/415: rename d2/dd/d75/d4a/f69 to d2/dc/f83 0 2024-04-22T23:45:42.055 INFO:tasks.workunit.client.0.smithi031.stdout:5/359: write db/d16/f15 [1031749,19383] 0 2024-04-22T23:45:42.056 INFO:tasks.workunit.client.0.smithi031.stdout:7/364: mknod d9/d14/d1f/d47/c7e 0 2024-04-22T23:45:42.057 INFO:tasks.workunit.client.0.smithi031.stdout:5/360: write db/f1e [3605594,2201] 0 2024-04-22T23:45:42.061 INFO:tasks.workunit.client.0.smithi031.stdout:3/330: rename d4/d54/d56/d61 to d4/d54/d6d 0 2024-04-22T23:45:42.064 INFO:tasks.workunit.client.0.smithi031.stdout:5/361: dwrite db/d16/f3e [4194304,4194304] 0 2024-04-22T23:45:42.066 INFO:tasks.workunit.client.0.smithi031.stdout:9/416: symlink d2/d15/d5a/l84 0 2024-04-22T23:45:42.067 INFO:tasks.workunit.client.0.smithi031.stdout:9/417: chown d2/dd/d75/d4a 136 1 2024-04-22T23:45:42.067 INFO:tasks.workunit.client.0.smithi031.stdout:6/447: getdents d3/d2a/d3c 0 2024-04-22T23:45:42.068 INFO:tasks.workunit.client.0.smithi031.stdout:6/448: chown d3/f8 81 1 2024-04-22T23:45:42.073 INFO:tasks.workunit.client.0.smithi031.stdout:6/449: dwrite d3/f18 [0,4194304] 0 2024-04-22T23:45:42.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/329: rename d0/d1/d2/d36/l16 to d0/d13/d18/d1e/d25/l76 0 2024-04-22T23:45:42.081 INFO:tasks.workunit.client.0.smithi031.stdout:1/344: dwrite d1/f1d [0,4194304] 0 2024-04-22T23:45:42.090 INFO:tasks.workunit.client.0.smithi031.stdout:6/450: symlink d3/d1f/d4c/d8e/l9c 0 2024-04-22T23:45:42.098 INFO:tasks.workunit.client.0.smithi031.stdout:1/345: rmdir d1/de/d1e/d34 39 2024-04-22T23:45:42.102 INFO:tasks.workunit.client.0.smithi031.stdout:6/451: creat d3/d2a/d85/f9d x:0 0 0 2024-04-22T23:45:42.108 INFO:tasks.workunit.client.0.smithi031.stdout:1/346: rename d1/d21/d29/f23 to d1/f7e 0 2024-04-22T23:45:42.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/379: truncate d5/da/d12/d1d/d38/f3f 3263437 0 2024-04-22T23:45:42.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/384: sync 2024-04-22T23:45:42.125 INFO:tasks.workunit.client.0.smithi031.stdout:2/385: symlink d6/dc/d42/l7b 0 2024-04-22T23:45:42.130 INFO:tasks.workunit.client.0.smithi031.stdout:7/365: write d9/f37 [238994,111482] 0 2024-04-22T23:45:42.132 INFO:tasks.workunit.client.0.smithi031.stdout:7/366: write d9/f3b [3607562,53782] 0 2024-04-22T23:45:42.132 INFO:tasks.workunit.client.0.smithi031.stdout:7/367: fsync d9/d15/d3a/f51 0 2024-04-22T23:45:42.134 INFO:tasks.workunit.client.0.smithi031.stdout:7/368: dread - d9/d14/d22/d5b/f6e zero size 2024-04-22T23:45:42.138 INFO:tasks.workunit.client.0.smithi031.stdout:7/369: rename d9/d53 to d9/d14/d1f/d7f 0 2024-04-22T23:45:42.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/331: truncate d4/d6/da/f29 3070403 0 2024-04-22T23:45:42.141 INFO:tasks.workunit.client.0.smithi031.stdout:5/362: dwrite db/d16/d2a/f30 [0,4194304] 0 2024-04-22T23:45:42.142 INFO:tasks.workunit.client.0.smithi031.stdout:9/418: dwrite d2/dd/d75/d4a/f36 [0,4194304] 0 2024-04-22T23:45:42.143 INFO:tasks.workunit.client.0.smithi031.stdout:0/330: write d0/d1/d2/d36/d11/d1a/d3c/f4d [4317350,49756] 0 2024-04-22T23:45:42.143 INFO:tasks.workunit.client.0.smithi031.stdout:7/370: write d9/d14/d22/d28/d7a/f7d [957337,68932] 0 2024-04-22T23:45:42.145 INFO:tasks.workunit.client.0.smithi031.stdout:3/332: fdatasync d4/d6/d4c/d4d/d5f/f63 0 2024-04-22T23:45:42.145 INFO:tasks.workunit.client.0.smithi031.stdout:9/419: dread d2/f6e [0,4194304] 0 2024-04-22T23:45:42.149 INFO:tasks.workunit.client.0.smithi031.stdout:7/371: dread - d9/d14/d1f/d47/f7c zero size 2024-04-22T23:45:42.153 INFO:tasks.workunit.client.0.smithi031.stdout:5/363: dwrite db/f3f [0,4194304] 0 2024-04-22T23:45:42.155 INFO:tasks.workunit.client.0.smithi031.stdout:5/364: write db/d16/d2a/d36/f6f [1003274,32243] 0 2024-04-22T23:45:42.155 INFO:tasks.workunit.client.0.smithi031.stdout:1/347: truncate d1/d21/d29/f1b 5430265 0 2024-04-22T23:45:42.159 INFO:tasks.workunit.client.0.smithi031.stdout:7/372: dread d9/d14/f23 [0,4194304] 0 2024-04-22T23:45:42.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/348: rmdir d1/d21 39 2024-04-22T23:45:42.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/333: fdatasync d4/d10/d2e/d45/f6c 0 2024-04-22T23:45:42.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/380: dread d5/da/d12/d37/f3c [0,4194304] 0 2024-04-22T23:45:42.178 INFO:tasks.workunit.client.0.smithi031.stdout:3/334: dread d4/d6/d11/d25/f36 [0,4194304] 0 2024-04-22T23:45:42.180 INFO:tasks.workunit.client.0.smithi031.stdout:2/386: chown d6/dc/d42/f61 116477373 1 2024-04-22T23:45:42.185 INFO:tasks.workunit.client.0.smithi031.stdout:5/365: dread db/d16/d2a/f2c [0,4194304] 0 2024-04-22T23:45:42.190 INFO:tasks.workunit.client.0.smithi031.stdout:3/335: creat d4/d54/d59/f6e x:0 0 0 2024-04-22T23:45:42.193 INFO:tasks.workunit.client.0.smithi031.stdout:0/331: getdents d0/d1/d2/d36/d11/d1a/d3c 0 2024-04-22T23:45:42.194 INFO:tasks.workunit.client.0.smithi031.stdout:5/366: dwrite db/d16/f15 [0,4194304] 0 2024-04-22T23:45:42.197 INFO:tasks.workunit.client.0.smithi031.stdout:2/387: dread d6/dc/d49/f70 [0,4194304] 0 2024-04-22T23:45:42.197 INFO:tasks.workunit.client.0.smithi031.stdout:4/509: sync 2024-04-22T23:45:42.198 INFO:tasks.workunit.client.0.smithi031.stdout:5/367: fdatasync db/d16/d1d/f20 0 2024-04-22T23:45:42.199 INFO:tasks.workunit.client.0.smithi031.stdout:9/420: dwrite d2/dc/f10 [8388608,4194304] 0 2024-04-22T23:45:42.199 INFO:tasks.workunit.client.0.smithi031.stdout:7/373: link d9/d15/d3a/d41/f54 d9/d14/d22/d5c/f80 0 2024-04-22T23:45:42.199 INFO:tasks.workunit.client.0.smithi031.stdout:2/388: chown d6/d45/d50/f58 27423 1 2024-04-22T23:45:42.200 INFO:tasks.workunit.client.0.smithi031.stdout:9/421: fsync d2/f35 0 2024-04-22T23:45:42.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/510: write db/d31/d40/d62/f77 [325716,78479] 0 2024-04-22T23:45:42.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/374: readlink d9/d14/d1f/d45/l62 0 2024-04-22T23:45:42.201 INFO:tasks.workunit.client.0.smithi031.stdout:2/389: chown d6/d45/d68 1930734975 1 2024-04-22T23:45:42.203 INFO:tasks.workunit.client.0.smithi031.stdout:5/368: read db/d16/d1d/d24/f2e [217801,72823] 0 2024-04-22T23:45:42.203 INFO:tasks.workunit.client.0.smithi031.stdout:3/336: unlink d4/d54/c40 0 2024-04-22T23:45:42.205 INFO:tasks.workunit.client.0.smithi031.stdout:5/369: chown db/d16/d2a/d36/f6f 0 1 2024-04-22T23:45:42.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/381: rename d5/da/d12/d37/d3b to d5/da/d12/d37/d34/d6f 0 2024-04-22T23:45:42.209 INFO:tasks.workunit.client.0.smithi031.stdout:8/382: stat d5/da/d12/d37/d49/f4f 0 2024-04-22T23:45:42.211 INFO:tasks.workunit.client.0.smithi031.stdout:2/390: dwrite d6/dc/f1a [0,4194304] 0 2024-04-22T23:45:42.212 INFO:tasks.workunit.client.0.smithi031.stdout:0/332: symlink d0/d13/d18/d1e/d25/d4c/l77 0 2024-04-22T23:45:42.214 INFO:tasks.workunit.client.0.smithi031.stdout:0/333: dread d0/d13/d18/d1e/f6a [0,4194304] 0 2024-04-22T23:45:42.219 INFO:tasks.workunit.client.0.smithi031.stdout:4/511: creat db/d31/d40/d62/fba x:0 0 0 2024-04-22T23:45:42.222 INFO:tasks.workunit.client.0.smithi031.stdout:7/375: symlink d9/d14/d1f/d7f/l81 0 2024-04-22T23:45:42.224 INFO:tasks.workunit.client.0.smithi031.stdout:7/376: write d9/d15/d6d/d65/f6b [1622710,44759] 0 2024-04-22T23:45:42.225 INFO:tasks.workunit.client.0.smithi031.stdout:1/349: fdatasync d1/d21/d29/f1b 0 2024-04-22T23:45:42.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/377: write d9/d15/f56 [116890,106393] 0 2024-04-22T23:45:42.227 INFO:tasks.workunit.client.0.smithi031.stdout:9/422: rename d2/dc/d21/d3d/d40/l3b to d2/dd/d75/l85 0 2024-04-22T23:45:42.227 INFO:tasks.workunit.client.0.smithi031.stdout:7/378: readlink d9/l4e 0 2024-04-22T23:45:42.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/370: creat db/d16/d1d/d24/d58/d5d/f7e x:0 0 0 2024-04-22T23:45:42.228 INFO:tasks.workunit.client.0.smithi031.stdout:4/512: dwrite db/d31/d40/d62/f77 [0,4194304] 0 2024-04-22T23:45:42.231 INFO:tasks.workunit.client.0.smithi031.stdout:4/513: chown db/d31/d40/d62/caf 479 1 2024-04-22T23:45:42.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/371: read db/d16/d1d/d24/d58/f51 [3660796,111705] 0 2024-04-22T23:45:42.232 INFO:tasks.workunit.client.0.smithi031.stdout:0/334: mknod d0/d1/d2/d17/c78 0 2024-04-22T23:45:42.241 INFO:tasks.workunit.client.0.smithi031.stdout:1/350: truncate d1/db/f19 1205876 0 2024-04-22T23:45:42.242 INFO:tasks.workunit.client.0.smithi031.stdout:7/379: creat d9/d14/d22/d28/f82 x:0 0 0 2024-04-22T23:45:42.243 INFO:tasks.workunit.client.0.smithi031.stdout:9/423: dread d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:42.244 INFO:tasks.workunit.client.0.smithi031.stdout:9/424: chown d2/f2e 5 1 2024-04-22T23:45:42.245 INFO:tasks.workunit.client.0.smithi031.stdout:2/391: mkdir d6/dc/d42/d28/d7c 0 2024-04-22T23:45:42.248 INFO:tasks.workunit.client.0.smithi031.stdout:5/372: creat db/d16/d2a/f7f x:0 0 0 2024-04-22T23:45:42.249 INFO:tasks.workunit.client.0.smithi031.stdout:3/337: dwrite d4/d10/f2b [4194304,4194304] 0 2024-04-22T23:45:42.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/335: write d0/d13/d18/d1e/f6a [1705803,47944] 0 2024-04-22T23:45:42.250 INFO:tasks.workunit.client.0.smithi031.stdout:5/373: fsync db/d16/d1d/d24/d27/f5b 0 2024-04-22T23:45:42.251 INFO:tasks.workunit.client.0.smithi031.stdout:5/374: stat db/d16/d1d/d24/d27/l64 0 2024-04-22T23:45:42.251 INFO:tasks.workunit.client.0.smithi031.stdout:0/336: write d0/d1/d28/f55 [4434534,121404] 0 2024-04-22T23:45:42.252 INFO:tasks.workunit.client.0.smithi031.stdout:0/337: read - d0/d1/d28/d2b/f51 zero size 2024-04-22T23:45:42.253 INFO:tasks.workunit.client.0.smithi031.stdout:0/338: read - d0/d13/f67 zero size 2024-04-22T23:45:42.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/351: unlink d1/f7e 0 2024-04-22T23:45:42.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/352: fdatasync d1/de/d1e/f64 0 2024-04-22T23:45:42.261 INFO:tasks.workunit.client.0.smithi031.stdout:8/383: link d5/da/d12/d37/d34/f4c d5/d48/d61/d6b/d6c/f70 0 2024-04-22T23:45:42.262 INFO:tasks.workunit.client.0.smithi031.stdout:9/425: fdatasync d2/d15/f7c 0 2024-04-22T23:45:42.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/426: chown d2/dc/d21/d3d/f68 100 1 2024-04-22T23:45:42.269 INFO:tasks.workunit.client.0.smithi031.stdout:4/514: write db/d14/d2d/d49/d59/d83/d9c/fa0 [2524896,5630] 0 2024-04-22T23:45:42.269 INFO:tasks.workunit.client.0.smithi031.stdout:2/392: dwrite d6/dc/d42/d24/f2b [0,4194304] 0 2024-04-22T23:45:42.271 INFO:tasks.workunit.client.0.smithi031.stdout:3/338: unlink f3 0 2024-04-22T23:45:42.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/339: chown d4/d6/d4c/d4d/d5f 0 1 2024-04-22T23:45:42.276 INFO:tasks.workunit.client.0.smithi031.stdout:0/339: rename d0/d13/d18/f39 to d0/d1/d28/f79 0 2024-04-22T23:45:42.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/353: write d1/db/d16/f25 [1816579,45112] 0 2024-04-22T23:45:42.279 INFO:tasks.workunit.client.0.smithi031.stdout:0/340: read d0/d1/d2/d36/f2f [1392154,112273] 0 2024-04-22T23:45:42.280 INFO:tasks.workunit.client.0.smithi031.stdout:6/452: sync 2024-04-22T23:45:42.282 INFO:tasks.workunit.client.0.smithi031.stdout:6/453: write d3/d35/d75/f89 [4546328,114904] 0 2024-04-22T23:45:42.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/380: dwrite d9/d14/d22/d5b/f5d [0,4194304] 0 2024-04-22T23:45:42.286 INFO:tasks.workunit.client.0.smithi031.stdout:9/427: chown d2/f8 6873 1 2024-04-22T23:45:42.292 INFO:tasks.workunit.client.0.smithi031.stdout:4/515: creat db/d14/d2d/d49/d59/d6a/fbb x:0 0 0 2024-04-22T23:45:42.293 INFO:tasks.workunit.client.0.smithi031.stdout:4/516: truncate db/d14/d2d/d49/d59/fa2 230121 0 2024-04-22T23:45:42.294 INFO:tasks.workunit.client.0.smithi031.stdout:8/384: write d5/da/d12/f14 [1136160,83944] 0 2024-04-22T23:45:42.302 INFO:tasks.workunit.client.0.smithi031.stdout:0/341: creat d0/d1/d2/f7a x:0 0 0 2024-04-22T23:45:42.303 INFO:tasks.workunit.client.0.smithi031.stdout:5/375: truncate db/d16/f2b 3545354 0 2024-04-22T23:45:42.304 INFO:tasks.workunit.client.0.smithi031.stdout:5/376: write db/d16/d1d/d24/f63 [1337190,81226] 0 2024-04-22T23:45:42.305 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:41 smithi031 ceph-mon[26721]: pgmap v27: 65 pgs: 65 active+clean; 662 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.8 MiB/s rd, 81 MiB/s wr, 383 op/s 2024-04-22T23:45:42.305 INFO:tasks.workunit.client.0.smithi031.stdout:5/377: write db/d16/d2a/d36/f38 [1448072,113937] 0 2024-04-22T23:45:42.307 INFO:tasks.workunit.client.0.smithi031.stdout:5/378: write db/d16/d1d/f47 [138970,113404] 0 2024-04-22T23:45:42.308 INFO:tasks.workunit.client.0.smithi031.stdout:7/381: creat d9/d15/d3a/d41/f83 x:0 0 0 2024-04-22T23:45:42.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/428: rmdir d2/dd/d75/d4a 39 2024-04-22T23:45:42.314 INFO:tasks.workunit.client.0.smithi031.stdout:4/517: creat db/d14/d1d/d7b/d7f/d2a/da1/fbc x:0 0 0 2024-04-22T23:45:42.316 INFO:tasks.workunit.client.0.smithi031.stdout:9/429: dwrite d2/dc/d3e/f56 [0,4194304] 0 2024-04-22T23:45:42.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/385: rename d5/d48/f5f to d5/da/d12/d2e/d4d/d56/f71 0 2024-04-22T23:45:42.316 INFO:tasks.workunit.client.0.smithi031.stdout:4/518: write db/d31/d40/d43/f51 [330959,40817] 0 2024-04-22T23:45:42.318 INFO:tasks.workunit.client.0.smithi031.stdout:4/519: dread - db/d14/d1d/d7b/d7f/fae zero size 2024-04-22T23:45:42.319 INFO:tasks.workunit.client.0.smithi031.stdout:4/520: read - db/d14/d2d/d49/fb3 zero size 2024-04-22T23:45:42.320 INFO:tasks.workunit.client.0.smithi031.stdout:0/342: fdatasync d0/d1/d28/f79 0 2024-04-22T23:45:42.321 INFO:tasks.workunit.client.0.smithi031.stdout:4/521: write db/d14/d1d/d7b/d7f/d2a/da1/fbc [403554,53075] 0 2024-04-22T23:45:42.328 INFO:tasks.workunit.client.0.smithi031.stdout:4/522: dwrite db/d14/d1d/fb0 [0,4194304] 0 2024-04-22T23:45:42.328 INFO:tasks.workunit.client.0.smithi031.stdout:6/454: link d3/f39 d3/d2a/d85/d96/f9e 0 2024-04-22T23:45:42.328 INFO:tasks.workunit.client.0.smithi031.stdout:1/354: write d1/db/d16/d1f/d52/f5c [14835,102358] 0 2024-04-22T23:45:42.329 INFO:tasks.workunit.client.0.smithi031.stdout:7/382: symlink d9/d14/d1f/d45/l84 0 2024-04-22T23:45:42.332 INFO:tasks.workunit.client.0.smithi031.stdout:2/393: creat d6/dc/d48/d4b/f7d x:0 0 0 2024-04-22T23:45:42.333 INFO:tasks.workunit.client.0.smithi031.stdout:4/523: write db/d14/d1d/d7b/d7f/d2a/d47/f7a [329252,5974] 0 2024-04-22T23:45:42.335 INFO:tasks.workunit.client.0.smithi031.stdout:7/383: write d9/d15/d6d/d65/f72 [717373,93661] 0 2024-04-22T23:45:42.336 INFO:tasks.workunit.client.0.smithi031.stdout:0/343: chown d0/d1/d28/d2b/l4b 782 1 2024-04-22T23:45:42.336 INFO:tasks.workunit.client.0.smithi031.stdout:6/455: symlink d3/d35/d86/l9f 0 2024-04-22T23:45:42.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/340: link d4/d6/d11/f1c d4/d49/f6f 0 2024-04-22T23:45:42.339 INFO:tasks.workunit.client.0.smithi031.stdout:3/341: chown d4/d6/d11/d3e/f43 67 1 2024-04-22T23:45:42.343 INFO:tasks.workunit.client.0.smithi031.stdout:2/394: rename d6/d45/d50/f5b to d6/d45/d68/d6c/f7e 0 2024-04-22T23:45:42.344 INFO:tasks.workunit.client.0.smithi031.stdout:6/456: creat d3/d77/d8d/fa0 x:0 0 0 2024-04-22T23:45:42.346 INFO:tasks.workunit.client.0.smithi031.stdout:4/524: symlink db/d14/d1d/d7b/d7f/d2a/d47/lbd 0 2024-04-22T23:45:42.347 INFO:tasks.workunit.client.0.smithi031.stdout:0/344: symlink d0/d1/d28/d2b/l7b 0 2024-04-22T23:45:42.349 INFO:tasks.workunit.client.0.smithi031.stdout:1/355: link d1/de/d1e/c61 d1/d47/d63/c7f 0 2024-04-22T23:45:42.351 INFO:tasks.workunit.client.0.smithi031.stdout:9/430: write d2/dc/d21/d3d/f51 [1023399,45932] 0 2024-04-22T23:45:42.352 INFO:tasks.workunit.client.0.smithi031.stdout:2/395: creat d6/dc/d17/d69/f7f x:0 0 0 2024-04-22T23:45:42.353 INFO:tasks.workunit.client.0.smithi031.stdout:6/457: rmdir d3/d2a/d3c/d5e 39 2024-04-22T23:45:42.356 INFO:tasks.workunit.client.0.smithi031.stdout:0/345: symlink d0/d1/d2/d36/l7c 0 2024-04-22T23:45:42.357 INFO:tasks.workunit.client.0.smithi031.stdout:3/342: truncate d4/d6/f16 2128243 0 2024-04-22T23:45:42.358 INFO:tasks.workunit.client.0.smithi031.stdout:3/343: chown d4/d6/d4c/c4e 2503540 1 2024-04-22T23:45:42.360 INFO:tasks.workunit.client.0.smithi031.stdout:7/384: link d9/d15/d6d/l66 d9/d14/d1f/l85 0 2024-04-22T23:45:42.363 INFO:tasks.workunit.client.0.smithi031.stdout:2/396: unlink d6/dc/d42/d24/f2b 0 2024-04-22T23:45:42.364 INFO:tasks.workunit.client.0.smithi031.stdout:4/525: dread db/d14/d2d/d49/d59/d6a/f92 [4194304,4194304] 0 2024-04-22T23:45:42.369 INFO:tasks.workunit.client.0.smithi031.stdout:8/386: getdents d5/da/df/d33 0 2024-04-22T23:45:42.370 INFO:tasks.workunit.client.0.smithi031.stdout:1/356: symlink d1/d47/d7d/l80 0 2024-04-22T23:45:42.370 INFO:tasks.workunit.client.0.smithi031.stdout:1/357: fdatasync d1/db/d16/d2f/f75 0 2024-04-22T23:45:42.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/431: write d2/dc/f17 [53483,96700] 0 2024-04-22T23:45:42.374 INFO:tasks.workunit.client.0.smithi031.stdout:3/344: fdatasync d4/d10/f62 0 2024-04-22T23:45:42.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/458: write d3/fe [1417870,53981] 0 2024-04-22T23:45:42.378 INFO:tasks.workunit.client.0.smithi031.stdout:7/385: creat d9/d15/d6d/d65/f86 x:0 0 0 2024-04-22T23:45:42.379 INFO:tasks.workunit.client.0.smithi031.stdout:7/386: fsync d9/d14/f18 0 2024-04-22T23:45:42.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/358: rename d1/d35/d36/d5a to d1/db/d16/d2f/d81 0 2024-04-22T23:45:42.387 INFO:tasks.workunit.client.0.smithi031.stdout:9/432: rmdir d2/d15 39 2024-04-22T23:45:42.388 INFO:tasks.workunit.client.0.smithi031.stdout:3/345: symlink d4/d6/d22/l70 0 2024-04-22T23:45:42.392 INFO:tasks.workunit.client.0.smithi031.stdout:8/387: symlink d5/da/d12/d37/d4e/l72 0 2024-04-22T23:45:42.393 INFO:tasks.workunit.client.0.smithi031.stdout:6/459: rename d3/c6 to d3/d1f/ca1 0 2024-04-22T23:45:42.393 INFO:tasks.workunit.client.0.smithi031.stdout:8/388: chown d5/da/d12/f44 7485882 1 2024-04-22T23:45:42.394 INFO:tasks.workunit.client.0.smithi031.stdout:1/359: symlink d1/db/l82 0 2024-04-22T23:45:42.396 INFO:tasks.workunit.client.0.smithi031.stdout:2/397: dwrite d6/dc/d42/f5c [0,4194304] 0 2024-04-22T23:45:42.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/398: write d6/dc/f62 [914057,37878] 0 2024-04-22T23:45:42.399 INFO:tasks.workunit.client.0.smithi031.stdout:3/346: creat d4/d6/d11/d25/d44/f71 x:0 0 0 2024-04-22T23:45:42.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/346: getdents d0/d1/d2/d36/d11/d1c/d54 0 2024-04-22T23:45:42.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/347: fdatasync d0/d13/d18/d1e/d25/f38 0 2024-04-22T23:45:42.408 INFO:tasks.workunit.client.0.smithi031.stdout:9/433: creat d2/d15/f86 x:0 0 0 2024-04-22T23:45:42.412 INFO:tasks.workunit.client.0.smithi031.stdout:3/347: mkdir d4/d49/d72 0 2024-04-22T23:45:42.413 INFO:tasks.workunit.client.0.smithi031.stdout:7/387: link d9/d15/d3a/d41/c46 d9/d14/d22/d28/d7a/c87 0 2024-04-22T23:45:42.414 INFO:tasks.workunit.client.0.smithi031.stdout:5/379: sync 2024-04-22T23:45:42.415 INFO:tasks.workunit.client.0.smithi031.stdout:4/526: getdents db/d31/d40/d43 0 2024-04-22T23:45:42.417 INFO:tasks.workunit.client.0.smithi031.stdout:8/389: creat d5/d48/d61/d6b/d6c/f73 x:0 0 0 2024-04-22T23:45:42.417 INFO:tasks.workunit.client.0.smithi031.stdout:8/390: chown d5 0 1 2024-04-22T23:45:42.419 INFO:tasks.workunit.client.0.smithi031.stdout:8/391: read d5/da/d12/d37/d34/d6f/f4b [3200970,7455] 0 2024-04-22T23:45:42.420 INFO:tasks.workunit.client.0.smithi031.stdout:0/348: symlink d0/d13/d18/d6c/l7d 0 2024-04-22T23:45:42.421 INFO:tasks.workunit.client.0.smithi031.stdout:5/380: dwrite db/d16/f65 [0,4194304] 0 2024-04-22T23:45:42.424 INFO:tasks.workunit.client.0.smithi031.stdout:5/381: dread - db/d16/d1d/d24/d58/d5d/f7e zero size 2024-04-22T23:45:42.425 INFO:tasks.workunit.client.0.smithi031.stdout:3/348: mknod d4/d6/da/c73 0 2024-04-22T23:45:42.425 INFO:tasks.workunit.client.0.smithi031.stdout:5/382: chown db/d16/c19 218774554 1 2024-04-22T23:45:42.426 INFO:tasks.workunit.client.0.smithi031.stdout:7/388: symlink d9/d15/d6d/l88 0 2024-04-22T23:45:42.427 INFO:tasks.workunit.client.0.smithi031.stdout:5/383: write db/d16/d1d/f20 [3887688,1484] 0 2024-04-22T23:45:42.427 INFO:tasks.workunit.client.0.smithi031.stdout:7/389: chown d9/f4b 364 1 2024-04-22T23:45:42.429 INFO:tasks.workunit.client.0.smithi031.stdout:1/360: creat d1/db/d16/f83 x:0 0 0 2024-04-22T23:45:42.429 INFO:tasks.workunit.client.0.smithi031.stdout:5/384: rename db to db/d16/d1d/d24/d32/d41/d80 22 2024-04-22T23:45:42.430 INFO:tasks.workunit.client.0.smithi031.stdout:2/399: link d6/dc/d17/c5e d6/dc/d17/d69/d78/c80 0 2024-04-22T23:45:42.431 INFO:tasks.workunit.client.0.smithi031.stdout:9/434: symlink d2/dd/d75/d4a/l87 0 2024-04-22T23:45:42.432 INFO:tasks.workunit.client.0.smithi031.stdout:0/349: creat d0/d1/d2/d36/d11/d1a/d56/f7e x:0 0 0 2024-04-22T23:45:42.437 INFO:tasks.workunit.client.0.smithi031.stdout:7/390: creat d9/d15/d6d/d65/f89 x:0 0 0 2024-04-22T23:45:42.437 INFO:tasks.workunit.client.0.smithi031.stdout:1/361: rename d1/d2e to d1/d3f/d54/d84 0 2024-04-22T23:45:42.440 INFO:tasks.workunit.client.0.smithi031.stdout:4/527: dwrite db/d14/fad [0,4194304] 0 2024-04-22T23:45:42.442 INFO:tasks.workunit.client.0.smithi031.stdout:1/362: chown d1/d21/d4b/d6f 38 1 2024-04-22T23:45:42.444 INFO:tasks.workunit.client.0.smithi031.stdout:7/391: mknod d9/d14/d22/d28/c8a 0 2024-04-22T23:45:42.446 INFO:tasks.workunit.client.0.smithi031.stdout:6/460: getdents d3/d1f/d60 0 2024-04-22T23:45:42.453 INFO:tasks.workunit.client.0.smithi031.stdout:2/400: dwrite d6/dc/f5f [0,4194304] 0 2024-04-22T23:45:42.454 INFO:tasks.workunit.client.0.smithi031.stdout:5/385: dwrite db/d16/d2a/f7f [0,4194304] 0 2024-04-22T23:45:42.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/392: dread - d9/d15/d6d/d65/f86 zero size 2024-04-22T23:45:42.456 INFO:tasks.workunit.client.0.smithi031.stdout:0/350: dwrite d0/d13/d18/d1e/f6f [0,4194304] 0 2024-04-22T23:45:42.460 INFO:tasks.workunit.client.0.smithi031.stdout:6/461: dread - d3/d2a/d3c/f73 zero size 2024-04-22T23:45:42.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:41 smithi044 ceph-mon[34463]: pgmap v27: 65 pgs: 65 active+clean; 662 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.8 MiB/s rd, 81 MiB/s wr, 383 op/s 2024-04-22T23:45:42.461 INFO:tasks.workunit.client.0.smithi031.stdout:2/401: readlink d6/dc/d17/l2d 0 2024-04-22T23:45:42.463 INFO:tasks.workunit.client.0.smithi031.stdout:9/435: dwrite d2/dc/f43 [0,4194304] 0 2024-04-22T23:45:42.469 INFO:tasks.workunit.client.0.smithi031.stdout:1/363: dread d1/db/d16/f25 [0,4194304] 0 2024-04-22T23:45:42.469 INFO:tasks.workunit.client.0.smithi031.stdout:7/393: symlink d9/d14/d1f/l8b 0 2024-04-22T23:45:42.473 INFO:tasks.workunit.client.0.smithi031.stdout:2/402: mknod d6/d45/c81 0 2024-04-22T23:45:42.474 INFO:tasks.workunit.client.0.smithi031.stdout:0/351: dwrite d0/d1/d2/d36/d11/d1a/d56/f7e [0,4194304] 0 2024-04-22T23:45:42.475 INFO:tasks.workunit.client.0.smithi031.stdout:9/436: rmdir d2/dd 39 2024-04-22T23:45:42.475 INFO:tasks.workunit.client.0.smithi031.stdout:1/364: readlink d1/d21/d29/d28/l78 0 2024-04-22T23:45:42.478 INFO:tasks.workunit.client.0.smithi031.stdout:4/528: truncate db/d14/d1d/f76 3099508 0 2024-04-22T23:45:42.480 INFO:tasks.workunit.client.0.smithi031.stdout:0/352: creat d0/f7f x:0 0 0 2024-04-22T23:45:42.484 INFO:tasks.workunit.client.0.smithi031.stdout:7/394: rename d9/d14/d22/d5b/f5d to d9/d14/d1f/d7f/f8c 0 2024-04-22T23:45:42.492 INFO:tasks.workunit.client.0.smithi031.stdout:4/529: dwrite db/d14/d2d/d49/d59/d6a/d63/d86/f8c [0,4194304] 0 2024-04-22T23:45:42.495 INFO:tasks.workunit.client.0.smithi031.stdout:3/349: dread d4/d6/d11/f37 [0,4194304] 0 2024-04-22T23:45:42.497 INFO:tasks.workunit.client.0.smithi031.stdout:4/530: write db/d14/d2d/d49/d59/d6a/d63/fa5 [422636,116304] 0 2024-04-22T23:45:42.498 INFO:tasks.workunit.client.0.smithi031.stdout:9/437: dwrite d2/dd/d75/d4a/f49 [0,4194304] 0 2024-04-22T23:45:42.498 INFO:tasks.workunit.client.0.smithi031.stdout:6/462: getdents d3/d2a/d3c/d40/d7a 0 2024-04-22T23:45:42.501 INFO:tasks.workunit.client.0.smithi031.stdout:4/531: dread db/d14/d1d/d7b/d7f/d2a/d30/f42 [0,4194304] 0 2024-04-22T23:45:42.502 INFO:tasks.workunit.client.0.smithi031.stdout:0/353: dread d0/d1/d2/d36/d11/d1a/d3c/f40 [0,4194304] 0 2024-04-22T23:45:42.505 INFO:tasks.workunit.client.0.smithi031.stdout:7/395: mknod d9/d14/d22/d5c/d5f/c8d 0 2024-04-22T23:45:42.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/386: dread db/d16/d1d/d24/f2e [0,4194304] 0 2024-04-22T23:45:42.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/438: dread d2/f6e [0,4194304] 0 2024-04-22T23:45:42.507 INFO:tasks.workunit.client.0.smithi031.stdout:1/365: getdents d1/db/d16/d2f 0 2024-04-22T23:45:42.510 INFO:tasks.workunit.client.0.smithi031.stdout:3/350: rename d4/d6/d11/l21 to d4/d6/d4c/d4d/d5f/l74 0 2024-04-22T23:45:42.511 INFO:tasks.workunit.client.0.smithi031.stdout:2/403: truncate d6/dc/d42/d28/f41 1100 0 2024-04-22T23:45:42.513 INFO:tasks.workunit.client.0.smithi031.stdout:4/532: mkdir db/d14/d2d/d49/d59/d8a/dbe 0 2024-04-22T23:45:42.517 INFO:tasks.workunit.client.0.smithi031.stdout:9/439: rename d2/dd/d75/d4a/f27 to d2/dc/d21/f88 0 2024-04-22T23:45:42.517 INFO:tasks.workunit.client.0.smithi031.stdout:3/351: rmdir d4/d6/d4c 39 2024-04-22T23:45:42.518 INFO:tasks.workunit.client.0.smithi031.stdout:2/404: stat d6/dc/d42/c27 0 2024-04-22T23:45:42.519 INFO:tasks.workunit.client.0.smithi031.stdout:3/352: read - d4/d6/d11/d25/d44/f71 zero size 2024-04-22T23:45:42.519 INFO:tasks.workunit.client.0.smithi031.stdout:1/366: unlink f0 0 2024-04-22T23:45:42.522 INFO:tasks.workunit.client.0.smithi031.stdout:4/533: mkdir db/d14/d1d/d7b/d7f/dbf 0 2024-04-22T23:45:42.523 INFO:tasks.workunit.client.0.smithi031.stdout:7/396: link d9/d15/d3a/d41/f83 d9/d15/d6d/f8e 0 2024-04-22T23:45:42.524 INFO:tasks.workunit.client.0.smithi031.stdout:7/397: readlink d9/d14/l1a 0 2024-04-22T23:45:42.525 INFO:tasks.workunit.client.0.smithi031.stdout:2/405: symlink d6/d45/d50/l82 0 2024-04-22T23:45:42.526 INFO:tasks.workunit.client.0.smithi031.stdout:3/353: creat d4/d54/d56/f75 x:0 0 0 2024-04-22T23:45:42.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/367: mknod d1/d21/d4e/c85 0 2024-04-22T23:45:42.529 INFO:tasks.workunit.client.0.smithi031.stdout:0/354: link d0/d13/d18/d1e/d25/d4c/l5d d0/d13/l80 0 2024-04-22T23:45:42.529 INFO:tasks.workunit.client.0.smithi031.stdout:2/406: chown d6/dc/d42/d24/d32/d79 7997535 1 2024-04-22T23:45:42.533 INFO:tasks.workunit.client.0.smithi031.stdout:1/368: write d1/d21/d29/f1b [1082778,51513] 0 2024-04-22T23:45:42.534 INFO:tasks.workunit.client.0.smithi031.stdout:3/354: getdents d4/d49/d72 0 2024-04-22T23:45:42.535 INFO:tasks.workunit.client.0.smithi031.stdout:0/355: dread - d0/d1/d2/f7a zero size 2024-04-22T23:45:42.536 INFO:tasks.workunit.client.0.smithi031.stdout:5/387: dread db/d16/d1d/d24/d27/f2d [0,4194304] 0 2024-04-22T23:45:42.538 INFO:tasks.workunit.client.0.smithi031.stdout:4/534: dwrite db/d14/d2d/f35 [0,4194304] 0 2024-04-22T23:45:42.541 INFO:tasks.workunit.client.0.smithi031.stdout:7/398: dwrite d9/d15/d6d/d65/f89 [0,4194304] 0 2024-04-22T23:45:42.541 INFO:tasks.workunit.client.0.smithi031.stdout:0/356: read - d0/f61 zero size 2024-04-22T23:45:42.541 INFO:tasks.workunit.client.0.smithi031.stdout:9/440: dread d2/dc/d3e/f4f [4194304,4194304] 0 2024-04-22T23:45:42.548 INFO:tasks.workunit.client.0.smithi031.stdout:7/399: creat d9/d15/d3a/d41/f8f x:0 0 0 2024-04-22T23:45:42.555 INFO:tasks.workunit.client.0.smithi031.stdout:3/355: dread d4/d6/d11/f37 [0,4194304] 0 2024-04-22T23:45:42.556 INFO:tasks.workunit.client.0.smithi031.stdout:7/400: dread d9/f37 [0,4194304] 0 2024-04-22T23:45:42.558 INFO:tasks.workunit.client.0.smithi031.stdout:9/441: fdatasync d2/dc/f4b 0 2024-04-22T23:45:42.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/388: truncate db/d16/f2b 837434 0 2024-04-22T23:45:42.562 INFO:tasks.workunit.client.0.smithi031.stdout:4/535: dwrite db/d14/d2d/d49/d59/d6a/d63/fa5 [0,4194304] 0 2024-04-22T23:45:42.563 INFO:tasks.workunit.client.0.smithi031.stdout:2/407: dread d6/f1c [0,4194304] 0 2024-04-22T23:45:42.564 INFO:tasks.workunit.client.0.smithi031.stdout:3/356: readlink d4/d6/d22/l24 0 2024-04-22T23:45:42.565 INFO:tasks.workunit.client.0.smithi031.stdout:0/357: dwrite d0/d13/d18/d1e/f6a [0,4194304] 0 2024-04-22T23:45:42.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/401: creat d9/d15/d6d/d65/f90 x:0 0 0 2024-04-22T23:45:42.567 INFO:tasks.workunit.client.0.smithi031.stdout:1/369: link d1/de/d1e/d34/l7a d1/d3f/d54/l86 0 2024-04-22T23:45:42.569 INFO:tasks.workunit.client.0.smithi031.stdout:2/408: rmdir d6/dc/d48 39 2024-04-22T23:45:42.575 INFO:tasks.workunit.client.0.smithi031.stdout:9/442: dwrite d2/dc/f1e [0,4194304] 0 2024-04-22T23:45:42.576 INFO:tasks.workunit.client.0.smithi031.stdout:3/357: truncate d4/d10/d2e/d45/f60 25779 0 2024-04-22T23:45:42.580 INFO:tasks.workunit.client.0.smithi031.stdout:1/370: dwrite d1/de/d1e/f64 [0,4194304] 0 2024-04-22T23:45:42.582 INFO:tasks.workunit.client.0.smithi031.stdout:2/409: unlink d6/dc/d17/d69/l74 0 2024-04-22T23:45:42.585 INFO:tasks.workunit.client.0.smithi031.stdout:5/389: link db/d16/d1d/f20 db/f81 0 2024-04-22T23:45:42.588 INFO:tasks.workunit.client.0.smithi031.stdout:2/410: read d6/dc/d42/d24/d32/f3a [37509,34955] 0 2024-04-22T23:45:42.588 INFO:tasks.workunit.client.0.smithi031.stdout:1/371: stat d1/db/d16/l17 0 2024-04-22T23:45:42.590 INFO:tasks.workunit.client.0.smithi031.stdout:5/390: chown db/d16/l48 94097880 1 2024-04-22T23:45:42.592 INFO:tasks.workunit.client.0.smithi031.stdout:9/443: dread d2/dc/d3e/f56 [0,4194304] 0 2024-04-22T23:45:42.593 INFO:tasks.workunit.client.0.smithi031.stdout:3/358: dwrite d4/d54/d59/f6e [0,4194304] 0 2024-04-22T23:45:42.594 INFO:tasks.workunit.client.0.smithi031.stdout:2/411: mkdir d6/dc/d49/d83 0 2024-04-22T23:45:42.596 INFO:tasks.workunit.client.0.smithi031.stdout:7/402: getdents d9/d14/d1f/d45 0 2024-04-22T23:45:42.598 INFO:tasks.workunit.client.0.smithi031.stdout:5/391: fdatasync db/d16/d1d/d24/f2e 0 2024-04-22T23:45:42.600 INFO:tasks.workunit.client.0.smithi031.stdout:3/359: write d4/d6/d4c/d4d/d5f/f63 [1697790,80589] 0 2024-04-22T23:45:42.601 INFO:tasks.workunit.client.0.smithi031.stdout:2/412: symlink d6/dc/d49/l84 0 2024-04-22T23:45:42.602 INFO:tasks.workunit.client.0.smithi031.stdout:2/413: stat d6/d45/d50/l82 0 2024-04-22T23:45:42.602 INFO:tasks.workunit.client.0.smithi031.stdout:2/414: chown d6/dc/d42 220470129 1 2024-04-22T23:45:42.604 INFO:tasks.workunit.client.0.smithi031.stdout:9/444: rename d2/f35 to d2/dd/d75/d64/d5d/f89 0 2024-04-22T23:45:42.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/360: symlink d4/d10/d2e/l76 0 2024-04-22T23:45:42.612 INFO:tasks.workunit.client.0.smithi031.stdout:5/392: mkdir db/d16/d2a/d36/d74/d82 0 2024-04-22T23:45:42.613 INFO:tasks.workunit.client.0.smithi031.stdout:2/415: mkdir d6/dc/d49/d83/d85 0 2024-04-22T23:45:42.614 INFO:tasks.workunit.client.0.smithi031.stdout:5/393: truncate db/d16/d2a/d59/f75 973307 0 2024-04-22T23:45:42.615 INFO:tasks.workunit.client.0.smithi031.stdout:2/416: mkdir d6/d45/d68/d6c/d86 0 2024-04-22T23:45:42.615 INFO:tasks.workunit.client.0.smithi031.stdout:4/536: dwrite db/d14/d2d/f37 [0,4194304] 0 2024-04-22T23:45:42.617 INFO:tasks.workunit.client.0.smithi031.stdout:2/417: chown d6/dc/d17/c3e 348084802 1 2024-04-22T23:45:42.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/394: dread - db/d16/d1d/d24/d58/d5d/f7e zero size 2024-04-22T23:45:42.618 INFO:tasks.workunit.client.0.smithi031.stdout:5/395: chown db/f25 24 1 2024-04-22T23:45:42.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/537: mknod db/d31/d40/d62/cc0 0 2024-04-22T23:45:42.620 INFO:tasks.workunit.client.0.smithi031.stdout:0/358: write d0/d1/d28/d2b/f64 [4256711,66315] 0 2024-04-22T23:45:42.621 INFO:tasks.workunit.client.0.smithi031.stdout:2/418: symlink d6/dc/d17/d69/l87 0 2024-04-22T23:45:42.622 INFO:tasks.workunit.client.0.smithi031.stdout:0/359: readlink d0/d13/d18/d1e/d25/d4c/l77 0 2024-04-22T23:45:42.628 INFO:tasks.workunit.client.0.smithi031.stdout:2/419: dread d6/dc/f1a [0,4194304] 0 2024-04-22T23:45:42.628 INFO:tasks.workunit.client.0.smithi031.stdout:5/396: link db/d16/d1d/d24/d27/f57 db/d16/d1d/d24/d32/f83 0 2024-04-22T23:45:42.631 INFO:tasks.workunit.client.0.smithi031.stdout:5/397: fsync db/d16/d2a/d59/f75 0 2024-04-22T23:45:42.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/360: dwrite d0/f7f [0,4194304] 0 2024-04-22T23:45:42.634 INFO:tasks.workunit.client.0.smithi031.stdout:0/361: fsync d0/d13/d18/d1e/f6f 0 2024-04-22T23:45:42.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/362: fsync d0/d13/d18/f48 0 2024-04-22T23:45:42.636 INFO:tasks.workunit.client.0.smithi031.stdout:2/420: read d6/dc/d57/f4d [2011120,95593] 0 2024-04-22T23:45:42.637 INFO:tasks.workunit.client.0.smithi031.stdout:1/372: dwrite d1/d3f/d54/d84/f3e [0,4194304] 0 2024-04-22T23:45:42.641 INFO:tasks.workunit.client.0.smithi031.stdout:2/421: write d6/dc/d57/d52/f6a [527041,68176] 0 2024-04-22T23:45:42.642 INFO:tasks.workunit.client.0.smithi031.stdout:4/538: getdents db/d14/d2d/d49/d59/d8a 0 2024-04-22T23:45:42.643 INFO:tasks.workunit.client.0.smithi031.stdout:2/422: chown d6/dc/d49 79 1 2024-04-22T23:45:42.643 INFO:tasks.workunit.client.0.smithi031.stdout:7/403: truncate d9/d15/f17 4258671 0 2024-04-22T23:45:42.643 INFO:tasks.workunit.client.0.smithi031.stdout:4/539: fdatasync db/d14/d1d/d7b/d7f/d2a/d30/f60 0 2024-04-22T23:45:42.643 INFO:tasks.workunit.client.0.smithi031.stdout:5/398: mknod db/d16/d1d/d24/d27/c84 0 2024-04-22T23:45:42.655 INFO:tasks.workunit.client.0.smithi031.stdout:5/399: rmdir db/d16/d2a/d36 39 2024-04-22T23:45:42.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/373: mknod d1/de/d1e/d34/d66/c87 0 2024-04-22T23:45:42.661 INFO:tasks.workunit.client.0.smithi031.stdout:2/423: fsync d6/dc/d42/f2c 0 2024-04-22T23:45:42.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/404: unlink d9/d14/d22/c74 0 2024-04-22T23:45:42.664 INFO:tasks.workunit.client.0.smithi031.stdout:4/540: mkdir db/d14/d1d/d7b/dc1 0 2024-04-22T23:45:42.667 INFO:tasks.workunit.client.0.smithi031.stdout:5/400: creat db/d16/d1d/d24/d58/f85 x:0 0 0 2024-04-22T23:45:42.668 INFO:tasks.workunit.client.0.smithi031.stdout:0/363: creat d0/d13/f81 x:0 0 0 2024-04-22T23:45:42.669 INFO:tasks.workunit.client.0.smithi031.stdout:0/364: chown d0/d1/d2 0 1 2024-04-22T23:45:42.669 INFO:tasks.workunit.client.0.smithi031.stdout:5/401: read db/d16/d1d/d24/d27/f57 [4034775,43245] 0 2024-04-22T23:45:42.670 INFO:tasks.workunit.client.0.smithi031.stdout:1/374: mkdir d1/db/d88 0 2024-04-22T23:45:42.671 INFO:tasks.workunit.client.0.smithi031.stdout:5/402: chown db/d16/d1d 950415 1 2024-04-22T23:45:42.672 INFO:tasks.workunit.client.0.smithi031.stdout:4/541: dwrite db/d14/d1d/d7b/d7f/d2a/d47/fa3 [0,4194304] 0 2024-04-22T23:45:42.672 INFO:tasks.workunit.client.0.smithi031.stdout:1/375: write d1/de/f11 [1079054,109256] 0 2024-04-22T23:45:42.674 INFO:tasks.workunit.client.0.smithi031.stdout:7/405: rmdir d9/d14/d22/d5b 39 2024-04-22T23:45:42.686 INFO:tasks.workunit.client.0.smithi031.stdout:1/376: dread d1/de/d1e/d34/d41/f4c [0,4194304] 0 2024-04-22T23:45:42.687 INFO:tasks.workunit.client.0.smithi031.stdout:7/406: unlink d9/d14/f20 0 2024-04-22T23:45:42.687 INFO:tasks.workunit.client.0.smithi031.stdout:1/377: chown d1/d21/d29/d28/f2a 1 1 2024-04-22T23:45:42.688 INFO:tasks.workunit.client.0.smithi031.stdout:2/424: rename d6/f40 to d6/dc/d42/f88 0 2024-04-22T23:45:42.691 INFO:tasks.workunit.client.0.smithi031.stdout:0/365: link d0/d13/f67 d0/d13/d18/d1e/d25/f82 0 2024-04-22T23:45:42.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/407: dwrite d9/d14/f75 [0,4194304] 0 2024-04-22T23:45:42.693 INFO:tasks.workunit.client.0.smithi031.stdout:7/408: chown d9/d15/d6d 13 1 2024-04-22T23:45:42.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/542: mknod db/cc2 0 2024-04-22T23:45:42.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/366: truncate d0/d1/d2/d36/d11/d1a/d3c/f40 2468354 0 2024-04-22T23:45:42.696 INFO:tasks.workunit.client.0.smithi031.stdout:1/378: symlink d1/de/d1e/d34/d4d/l89 0 2024-04-22T23:45:42.697 INFO:tasks.workunit.client.0.smithi031.stdout:0/367: chown d0/d1/d2/d36/d11/f43 22288 1 2024-04-22T23:45:42.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/403: mkdir db/d16/d2a/d36/d74/d82/d86 0 2024-04-22T23:45:42.701 INFO:tasks.workunit.client.0.smithi031.stdout:4/543: write db/d14/d2d/d49/d59/d83/d9c/f88 [117726,48873] 0 2024-04-22T23:45:42.704 INFO:tasks.workunit.client.0.smithi031.stdout:0/368: chown d0/d1/d2/d36/d11/d1c/d54/f69 11761787 1 2024-04-22T23:45:42.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/404: chown db/d16/d2a/f2c 66 1 2024-04-22T23:45:42.705 INFO:tasks.workunit.client.0.smithi031.stdout:1/379: write d1/db/d16/f25 [2426806,16513] 0 2024-04-22T23:45:42.707 INFO:tasks.workunit.client.0.smithi031.stdout:7/409: dwrite d9/d15/d6d/d65/f86 [0,4194304] 0 2024-04-22T23:45:42.709 INFO:tasks.workunit.client.0.smithi031.stdout:0/369: write d0/d1/d2/f23 [401198,98032] 0 2024-04-22T23:45:42.711 INFO:tasks.workunit.client.0.smithi031.stdout:1/380: chown d1/de/c56 0 1 2024-04-22T23:45:42.711 INFO:tasks.workunit.client.0.smithi031.stdout:2/425: dwrite d6/dc/d42/f67 [0,4194304] 0 2024-04-22T23:45:42.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/426: read - d6/d45/f77 zero size 2024-04-22T23:45:42.716 INFO:tasks.workunit.client.0.smithi031.stdout:7/410: mknod d9/d14/d1f/d7f/c91 0 2024-04-22T23:45:42.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/370: mknod d0/d1/d3f/c83 0 2024-04-22T23:45:42.720 INFO:tasks.workunit.client.0.smithi031.stdout:1/381: fsync d1/de/d1e/d34/d41/f42 0 2024-04-22T23:45:42.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/411: symlink d9/d14/l92 0 2024-04-22T23:45:42.724 INFO:tasks.workunit.client.0.smithi031.stdout:7/412: write d9/d14/d22/f78 [796343,17779] 0 2024-04-22T23:45:42.725 INFO:tasks.workunit.client.0.smithi031.stdout:2/427: rename d6/d45/c7a to d6/dc/d17/c89 0 2024-04-22T23:45:42.727 INFO:tasks.workunit.client.0.smithi031.stdout:0/371: symlink d0/d1/d2/d36/l84 0 2024-04-22T23:45:42.730 INFO:tasks.workunit.client.0.smithi031.stdout:5/405: dwrite db/d16/d1d/d24/d32/f39 [4194304,4194304] 0 2024-04-22T23:45:42.733 INFO:tasks.workunit.client.0.smithi031.stdout:5/406: write db/d16/d1d/d24/d58/f85 [594370,4540] 0 2024-04-22T23:45:42.733 INFO:tasks.workunit.client.0.smithi031.stdout:7/413: rmdir d9/d15 39 2024-04-22T23:45:42.735 INFO:tasks.workunit.client.0.smithi031.stdout:6/463: sync 2024-04-22T23:45:42.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/392: sync 2024-04-22T23:45:42.736 INFO:tasks.workunit.client.0.smithi031.stdout:1/382: link d1/de/d1e/d34/d41/f62 d1/d47/d7d/f8a 0 2024-04-22T23:45:42.738 INFO:tasks.workunit.client.0.smithi031.stdout:4/544: dread db/d14/f3a [8388608,4194304] 0 2024-04-22T23:45:42.738 INFO:tasks.workunit.client.0.smithi031.stdout:5/407: creat db/d16/d2a/d36/d74/f87 x:0 0 0 2024-04-22T23:45:42.740 INFO:tasks.workunit.client.0.smithi031.stdout:5/408: chown db/d16/l45 142837244 1 2024-04-22T23:45:42.740 INFO:tasks.workunit.client.0.smithi031.stdout:4/545: write db/d14/f6e [4284696,80462] 0 2024-04-22T23:45:42.742 INFO:tasks.workunit.client.0.smithi031.stdout:2/428: mkdir d6/dc/d48/d8a 0 2024-04-22T23:45:42.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/429: fdatasync d6/dc/f5f 0 2024-04-22T23:45:42.744 INFO:tasks.workunit.client.0.smithi031.stdout:7/414: mknod d9/d14/d22/d5c/d5f/c93 0 2024-04-22T23:45:42.747 INFO:tasks.workunit.client.0.smithi031.stdout:6/464: unlink d3/d35/f5d 0 2024-04-22T23:45:42.750 INFO:tasks.workunit.client.0.smithi031.stdout:8/393: truncate d5/fb 7612550 0 2024-04-22T23:45:42.753 INFO:tasks.workunit.client.0.smithi031.stdout:5/409: creat db/d16/d2a/d59/f88 x:0 0 0 2024-04-22T23:45:42.754 INFO:tasks.workunit.client.0.smithi031.stdout:4/546: creat db/d14/d2d/d49/d59/d6a/d41/fc3 x:0 0 0 2024-04-22T23:45:42.755 INFO:tasks.workunit.client.0.smithi031.stdout:0/372: truncate d0/d13/d18/d1e/f30 1206702 0 2024-04-22T23:45:42.756 INFO:tasks.workunit.client.0.smithi031.stdout:7/415: dread f2 [4194304,4194304] 0 2024-04-22T23:45:42.761 INFO:tasks.workunit.client.0.smithi031.stdout:6/465: mknod d3/d35/d86/ca2 0 2024-04-22T23:45:42.761 INFO:tasks.workunit.client.0.smithi031.stdout:4/547: dwrite db/d14/d1d/d7b/d7f/d2a/da1/fbc [0,4194304] 0 2024-04-22T23:45:42.762 INFO:tasks.workunit.client.0.smithi031.stdout:4/548: chown db/d14/d2d/f37 389261754 1 2024-04-22T23:45:42.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/373: rename d0/d1/d2/f52 to d0/d13/d18/d1e/d25/f85 0 2024-04-22T23:45:42.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/374: readlink d0/d1/d28/d2b/l7b 0 2024-04-22T23:45:42.770 INFO:tasks.workunit.client.0.smithi031.stdout:7/416: rmdir d9/d14/d1f/d45 39 2024-04-22T23:45:42.775 INFO:tasks.workunit.client.0.smithi031.stdout:4/549: dread - db/f5f zero size 2024-04-22T23:45:42.776 INFO:tasks.workunit.client.0.smithi031.stdout:4/550: fsync db/d14/d1d/d7b/d7f/fae 0 2024-04-22T23:45:42.778 INFO:tasks.workunit.client.0.smithi031.stdout:2/430: creat d6/f8b x:0 0 0 2024-04-22T23:45:42.780 INFO:tasks.workunit.client.0.smithi031.stdout:0/375: creat d0/f86 x:0 0 0 2024-04-22T23:45:42.781 INFO:tasks.workunit.client.0.smithi031.stdout:7/417: readlink d9/d14/d1f/l85 0 2024-04-22T23:45:42.781 INFO:tasks.workunit.client.0.smithi031.stdout:0/376: stat d0/d1/d2/d36/d11/d1a/l4a 0 2024-04-22T23:45:42.782 INFO:tasks.workunit.client.0.smithi031.stdout:1/383: getdents d1/db/d16 0 2024-04-22T23:45:42.782 INFO:tasks.workunit.client.0.smithi031.stdout:7/418: fdatasync d9/d14/d22/d28/f36 0 2024-04-22T23:45:42.784 INFO:tasks.workunit.client.0.smithi031.stdout:6/466: mkdir d3/d1f/da3 0 2024-04-22T23:45:42.785 INFO:tasks.workunit.client.0.smithi031.stdout:4/551: readlink db/lac 0 2024-04-22T23:45:42.785 INFO:tasks.workunit.client.0.smithi031.stdout:4/552: chown db/d31/d40/d9b 3699 1 2024-04-22T23:45:42.786 INFO:tasks.workunit.client.0.smithi031.stdout:4/553: chown db/d14/d2d/d49/d59/d6a/d41/f7d 1462 1 2024-04-22T23:45:42.786 INFO:tasks.workunit.client.0.smithi031.stdout:8/394: write d5/fb [2976255,97912] 0 2024-04-22T23:45:42.787 INFO:tasks.workunit.client.0.smithi031.stdout:2/431: symlink d6/d45/l8c 0 2024-04-22T23:45:42.789 INFO:tasks.workunit.client.0.smithi031.stdout:4/554: chown db/d14/d2d/d49/d59/d6a/d63/d86/f8c 4020 1 2024-04-22T23:45:42.791 INFO:tasks.workunit.client.0.smithi031.stdout:1/384: creat d1/db/d16/d1f/f8b x:0 0 0 2024-04-22T23:45:42.793 INFO:tasks.workunit.client.0.smithi031.stdout:0/377: rename d0/d1/d28/d2b/c5e to d0/d1/d2/d36/d11/d1c/c87 0 2024-04-22T23:45:42.798 INFO:tasks.workunit.client.0.smithi031.stdout:6/467: mkdir d3/da4 0 2024-04-22T23:45:42.798 INFO:tasks.workunit.client.0.smithi031.stdout:2/432: dwrite d6/d45/f73 [0,4194304] 0 2024-04-22T23:45:42.800 INFO:tasks.workunit.client.0.smithi031.stdout:7/419: dwrite d9/d15/d3a/f51 [0,4194304] 0 2024-04-22T23:45:42.809 INFO:tasks.workunit.client.0.smithi031.stdout:4/555: rename db/d31/c87 to db/d14/d1d/d7b/d7f/d2a/d47/d80/cc4 0 2024-04-22T23:45:42.811 INFO:tasks.workunit.client.0.smithi031.stdout:8/395: truncate d5/da/d12/d37/d49/f4f 2930085 0 2024-04-22T23:45:42.815 INFO:tasks.workunit.client.0.smithi031.stdout:1/385: dwrite d1/de/d1e/d34/d41/f42 [0,4194304] 0 2024-04-22T23:45:42.816 INFO:tasks.workunit.client.0.smithi031.stdout:4/556: dread db/d14/d1d/d7b/d7f/d2a/d47/fa3 [0,4194304] 0 2024-04-22T23:45:42.817 INFO:tasks.workunit.client.0.smithi031.stdout:1/386: fsync d1/de/f11 0 2024-04-22T23:45:42.818 INFO:tasks.workunit.client.0.smithi031.stdout:1/387: chown d1/de/d1e/c6a 554838285 1 2024-04-22T23:45:42.818 INFO:tasks.workunit.client.0.smithi031.stdout:2/433: write d6/d45/d68/d6c/f7e [588084,43356] 0 2024-04-22T23:45:42.819 INFO:tasks.workunit.client.0.smithi031.stdout:1/388: fsync d1/db/d16/d2f/f3b 0 2024-04-22T23:45:42.822 INFO:tasks.workunit.client.0.smithi031.stdout:2/434: dread d6/f1c [0,4194304] 0 2024-04-22T23:45:42.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/468: mkdir d3/da5 0 2024-04-22T23:45:42.828 INFO:tasks.workunit.client.0.smithi031.stdout:0/378: link d0/d1/d2/d36/d11/f14 d0/d1/d2/d36/d11/d1c/f88 0 2024-04-22T23:45:42.831 INFO:tasks.workunit.client.0.smithi031.stdout:6/469: dwrite d3/f74 [0,4194304] 0 2024-04-22T23:45:42.833 INFO:tasks.workunit.client.0.smithi031.stdout:0/379: dread d0/f7f [0,4194304] 0 2024-04-22T23:45:42.835 INFO:tasks.workunit.client.0.smithi031.stdout:2/435: mknod d6/dc/d17/d69/c8d 0 2024-04-22T23:45:42.838 INFO:tasks.workunit.client.0.smithi031.stdout:8/396: rename d5/da/df/f43 to d5/da/d12/d37/d34/d6f/d5b/f74 0 2024-04-22T23:45:42.839 INFO:tasks.workunit.client.0.smithi031.stdout:8/397: chown d5/da/d12/d1d 227643 1 2024-04-22T23:45:42.840 INFO:tasks.workunit.client.0.smithi031.stdout:0/380: dwrite d0/d1/d2/d36/d11/d1a/d3c/f59 [0,4194304] 0 2024-04-22T23:45:42.850 INFO:tasks.workunit.client.0.smithi031.stdout:7/420: creat d9/f94 x:0 0 0 2024-04-22T23:45:42.854 INFO:tasks.workunit.client.0.smithi031.stdout:6/470: symlink d3/d2a/d3c/d40/d7a/la6 0 2024-04-22T23:45:42.860 INFO:tasks.workunit.client.0.smithi031.stdout:0/381: fsync d0/d13/d18/d1e/d25/f82 0 2024-04-22T23:45:42.860 INFO:tasks.workunit.client.0.smithi031.stdout:6/471: dwrite d3/d2a/d3c/d65/f82 [0,4194304] 0 2024-04-22T23:45:42.869 INFO:tasks.workunit.client.0.smithi031.stdout:4/557: rmdir db/d14/d2d/d49/d59/d8a/dbe 0 2024-04-22T23:45:42.871 INFO:tasks.workunit.client.0.smithi031.stdout:7/421: fsync d9/d15/d3a/d41/f83 0 2024-04-22T23:45:42.872 INFO:tasks.workunit.client.0.smithi031.stdout:7/422: read f3 [3102932,105159] 0 2024-04-22T23:45:42.873 INFO:tasks.workunit.client.0.smithi031.stdout:1/389: link d1/d35/l48 d1/db/d16/d2f/l8c 0 2024-04-22T23:45:42.879 INFO:tasks.workunit.client.0.smithi031.stdout:6/472: symlink d3/d2a/la7 0 2024-04-22T23:45:42.881 INFO:tasks.workunit.client.0.smithi031.stdout:0/382: symlink d0/d13/d18/l89 0 2024-04-22T23:45:42.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/558: creat db/d14/d2d/d49/d59/d6a/d63/d86/fc5 x:0 0 0 2024-04-22T23:45:42.886 INFO:tasks.workunit.client.0.smithi031.stdout:4/559: write db/d14/d1d/d7b/d7f/fae [809431,61286] 0 2024-04-22T23:45:42.892 INFO:tasks.workunit.client.0.smithi031.stdout:7/423: creat d9/d14/d22/d28/d7a/f95 x:0 0 0 2024-04-22T23:45:42.892 INFO:tasks.workunit.client.0.smithi031.stdout:7/424: chown d9/d14/d1f/d47 0 1 2024-04-22T23:45:42.896 INFO:tasks.workunit.client.0.smithi031.stdout:6/473: symlink d3/d77/d8d/la8 0 2024-04-22T23:45:42.899 INFO:tasks.workunit.client.0.smithi031.stdout:0/383: mknod d0/c8a 0 2024-04-22T23:45:42.900 INFO:tasks.workunit.client.0.smithi031.stdout:0/384: fsync d0/d1/d2/d36/d11/d1a/d3c/f59 0 2024-04-22T23:45:42.901 INFO:tasks.workunit.client.0.smithi031.stdout:1/390: truncate d1/db/f19 204774 0 2024-04-22T23:45:42.903 INFO:tasks.workunit.client.0.smithi031.stdout:2/436: getdents d6/dc/d49/d83 0 2024-04-22T23:45:42.903 INFO:tasks.workunit.client.0.smithi031.stdout:2/437: chown d6/f16 146263948 1 2024-04-22T23:45:42.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/438: write d6/f19 [4324864,53769] 0 2024-04-22T23:45:42.905 INFO:tasks.workunit.client.0.smithi031.stdout:4/560: mkdir db/d14/d2d/d49/d59/d83/d9c/d72/dc6 0 2024-04-22T23:45:42.909 INFO:tasks.workunit.client.0.smithi031.stdout:0/385: creat d0/d1/d3f/f8b x:0 0 0 2024-04-22T23:45:42.909 INFO:tasks.workunit.client.0.smithi031.stdout:6/474: dwrite d3/d1f/f64 [0,4194304] 0 2024-04-22T23:45:42.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/386: chown d0/d1/d28/d2b/l4b 17532591 1 2024-04-22T23:45:42.910 INFO:tasks.workunit.client.0.smithi031.stdout:1/391: mkdir d1/d35/d8d 0 2024-04-22T23:45:42.911 INFO:tasks.workunit.client.0.smithi031.stdout:1/392: chown d1/d21/c79 411 1 2024-04-22T23:45:42.913 INFO:tasks.workunit.client.0.smithi031.stdout:7/425: symlink d9/d15/d3f/l96 0 2024-04-22T23:45:42.913 INFO:tasks.workunit.client.0.smithi031.stdout:6/475: read d3/d2a/d3c/d40/f4e [315689,61021] 0 2024-04-22T23:45:42.918 INFO:tasks.workunit.client.0.smithi031.stdout:4/561: mknod db/d14/d1d/d7b/d7f/d2a/d47/d80/cc7 0 2024-04-22T23:45:42.918 INFO:tasks.workunit.client.0.smithi031.stdout:4/562: write db/d14/d1d/d7b/d7f/d2a/fb1 [659464,2127] 0 2024-04-22T23:45:42.922 INFO:tasks.workunit.client.0.smithi031.stdout:0/387: creat d0/d13/d18/d1e/f8c x:0 0 0 2024-04-22T23:45:42.925 INFO:tasks.workunit.client.0.smithi031.stdout:6/476: symlink d3/d2a/la9 0 2024-04-22T23:45:42.927 INFO:tasks.workunit.client.0.smithi031.stdout:2/439: mknod d6/dc/d49/d83/d85/c8e 0 2024-04-22T23:45:42.928 INFO:tasks.workunit.client.0.smithi031.stdout:0/388: dwrite d0/d1/d2/d36/f1d [0,4194304] 0 2024-04-22T23:45:42.932 INFO:tasks.workunit.client.0.smithi031.stdout:4/563: mknod db/d14/d2d/d49/d59/d6a/d63/d86/cc8 0 2024-04-22T23:45:42.933 INFO:tasks.workunit.client.0.smithi031.stdout:1/393: dread d1/db/d16/d2f/f3b [0,4194304] 0 2024-04-22T23:45:42.934 INFO:tasks.workunit.client.0.smithi031.stdout:1/394: write d1/d21/d29/f1b [2239886,90935] 0 2024-04-22T23:45:42.938 INFO:tasks.workunit.client.0.smithi031.stdout:6/477: creat d3/d2a/d3c/d40/d5b/faa x:0 0 0 2024-04-22T23:45:42.940 INFO:tasks.workunit.client.0.smithi031.stdout:1/395: dwrite d1/f46 [0,4194304] 0 2024-04-22T23:45:42.941 INFO:tasks.workunit.client.0.smithi031.stdout:1/396: read d1/f1d [710283,45318] 0 2024-04-22T23:45:42.942 INFO:tasks.workunit.client.0.smithi031.stdout:1/397: chown d1/db/d16/d2f/d81/c6e 51004 1 2024-04-22T23:45:42.944 INFO:tasks.workunit.client.0.smithi031.stdout:0/389: rename d0/d1/d28/d2b/f51 to d0/d1/d2/d36/f8d 0 2024-04-22T23:45:42.947 INFO:tasks.workunit.client.0.smithi031.stdout:0/390: truncate d0/d1/d2/d36/d11/d1c/d54/f72 825811 0 2024-04-22T23:45:42.947 INFO:tasks.workunit.client.0.smithi031.stdout:4/564: mkdir db/d14/d2d/d49/d59/d6a/d63/dc9 0 2024-04-22T23:45:42.948 INFO:tasks.workunit.client.0.smithi031.stdout:0/391: chown d0/d1/d2/d36/d11/d1a/d3c/f59 218840 1 2024-04-22T23:45:42.950 INFO:tasks.workunit.client.0.smithi031.stdout:0/392: chown d0/d13/d18/d1e/f8c 156377866 1 2024-04-22T23:45:42.953 INFO:tasks.workunit.client.0.smithi031.stdout:6/478: dread - d3/d1f/d60/f6a zero size 2024-04-22T23:45:42.955 INFO:tasks.workunit.client.0.smithi031.stdout:2/440: creat d6/dc/d42/d24/f8f x:0 0 0 2024-04-22T23:45:42.959 INFO:tasks.workunit.client.0.smithi031.stdout:4/565: creat db/d14/d2d/d49/d59/d6a/d63/d86/fca x:0 0 0 2024-04-22T23:45:42.960 INFO:tasks.workunit.client.0.smithi031.stdout:0/393: write d0/d1/d2/d36/f2f [189319,98231] 0 2024-04-22T23:45:42.962 INFO:tasks.workunit.client.0.smithi031.stdout:6/479: creat d3/d2a/d85/fab x:0 0 0 2024-04-22T23:45:42.970 INFO:tasks.workunit.client.0.smithi031.stdout:4/566: getdents db/d14/d1d/d7b/d7f/dbf 0 2024-04-22T23:45:42.973 INFO:tasks.workunit.client.0.smithi031.stdout:0/394: dread d0/d1/f6b [0,4194304] 0 2024-04-22T23:45:42.986 INFO:tasks.workunit.client.0.smithi031.stdout:4/567: creat db/fcb x:0 0 0 2024-04-22T23:45:42.988 INFO:tasks.workunit.client.0.smithi031.stdout:2/441: getdents d6/dc/d42/d24 0 2024-04-22T23:45:42.989 INFO:tasks.workunit.client.0.smithi031.stdout:2/442: chown d6/dc/d17/c1d 26676892 1 2024-04-22T23:45:42.989 INFO:tasks.workunit.client.0.smithi031.stdout:2/443: dread - d6/f60 zero size 2024-04-22T23:45:42.990 INFO:tasks.workunit.client.0.smithi031.stdout:4/568: symlink db/d14/d2d/d49/d59/d6a/d41/lcc 0 2024-04-22T23:45:42.992 INFO:tasks.workunit.client.0.smithi031.stdout:4/569: mkdir db/d14/d2d/d49/dcd 0 2024-04-22T23:45:42.995 INFO:tasks.workunit.client.0.smithi031.stdout:3/361: sync 2024-04-22T23:45:42.995 INFO:tasks.workunit.client.0.smithi031.stdout:9/445: sync 2024-04-22T23:45:42.995 INFO:tasks.workunit.client.0.smithi031.stdout:6/480: truncate d3/f15 7037085 0 2024-04-22T23:45:42.996 INFO:tasks.workunit.client.0.smithi031.stdout:0/395: truncate d0/d1/d28/d2b/f64 1880070 0 2024-04-22T23:45:42.996 INFO:tasks.workunit.client.0.smithi031.stdout:2/444: dwrite d6/dc/f5f [0,4194304] 0 2024-04-22T23:45:42.998 INFO:tasks.workunit.client.0.smithi031.stdout:9/446: read d2/dc/f43 [3503733,117381] 0 2024-04-22T23:45:43.002 INFO:tasks.workunit.client.0.smithi031.stdout:1/398: dread d1/de/d1e/d34/d41/f58 [0,4194304] 0 2024-04-22T23:45:43.002 INFO:tasks.workunit.client.0.smithi031.stdout:9/447: write d2/dd/d75/d64/d5d/f57 [3432206,91567] 0 2024-04-22T23:45:43.002 INFO:tasks.workunit.client.0.smithi031.stdout:4/570: dwrite db/d31/d40/fb2 [0,4194304] 0 2024-04-22T23:45:43.012 INFO:tasks.workunit.client.0.smithi031.stdout:2/445: unlink d6/dc/d17/c5e 0 2024-04-22T23:45:43.017 INFO:tasks.workunit.client.0.smithi031.stdout:1/399: mknod d1/d21/d29/c8e 0 2024-04-22T23:45:43.017 INFO:tasks.workunit.client.0.smithi031.stdout:1/400: chown d1/d21 113 1 2024-04-22T23:45:43.021 INFO:tasks.workunit.client.0.smithi031.stdout:9/448: symlink d2/dc/d21/l8a 0 2024-04-22T23:45:43.023 INFO:tasks.workunit.client.0.smithi031.stdout:1/401: dwrite d1/d35/f7b [0,4194304] 0 2024-04-22T23:45:43.024 INFO:tasks.workunit.client.0.smithi031.stdout:4/571: creat db/d14/d1d/d7b/d7f/d2a/d30/fce x:0 0 0 2024-04-22T23:45:43.027 INFO:tasks.workunit.client.0.smithi031.stdout:7/426: sync 2024-04-22T23:45:43.027 INFO:tasks.workunit.client.0.smithi031.stdout:5/410: sync 2024-04-22T23:45:43.027 INFO:tasks.workunit.client.0.smithi031.stdout:8/398: sync 2024-04-22T23:45:43.027 INFO:tasks.workunit.client.0.smithi031.stdout:4/572: write db/d14/d2d/d49/d59/d83/d9c/f88 [952622,31743] 0 2024-04-22T23:45:43.027 INFO:tasks.workunit.client.0.smithi031.stdout:6/481: dread d3/f32 [0,4194304] 0 2024-04-22T23:45:43.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/446: mkdir d6/dc/d17/d69/d90 0 2024-04-22T23:45:43.030 INFO:tasks.workunit.client.0.smithi031.stdout:6/482: fdatasync d3/d2a/f70 0 2024-04-22T23:45:43.030 INFO:tasks.workunit.client.0.smithi031.stdout:2/447: stat d6/f1c 0 2024-04-22T23:45:43.031 INFO:tasks.workunit.client.0.smithi031.stdout:5/411: fdatasync db/d16/d1d/d24/d32/f79 0 2024-04-22T23:45:43.033 INFO:tasks.workunit.client.0.smithi031.stdout:3/362: dwrite d4/d49/f6f [0,4194304] 0 2024-04-22T23:45:43.034 INFO:tasks.workunit.client.0.smithi031.stdout:5/412: stat db/f1e 0 2024-04-22T23:45:43.034 INFO:tasks.workunit.client.0.smithi031.stdout:1/402: creat d1/d3f/d54/f8f x:0 0 0 2024-04-22T23:45:43.039 INFO:tasks.workunit.client.0.smithi031.stdout:8/399: dwrite d5/fb [0,4194304] 0 2024-04-22T23:45:43.042 INFO:tasks.workunit.client.0.smithi031.stdout:3/363: dread d4/d54/d59/f6e [0,4194304] 0 2024-04-22T23:45:43.043 INFO:tasks.workunit.client.0.smithi031.stdout:0/396: truncate d0/d13/f5b 735110 0 2024-04-22T23:45:43.044 INFO:tasks.workunit.client.0.smithi031.stdout:3/364: readlink d4/d6/d11/d25/d44/l6b 0 2024-04-22T23:45:43.046 INFO:tasks.workunit.client.0.smithi031.stdout:4/573: creat db/d14/d1d/d7b/d7f/d2a/d47/d80/fcf x:0 0 0 2024-04-22T23:45:43.047 INFO:tasks.workunit.client.0.smithi031.stdout:2/448: rmdir d6/dc/d48/d4b/d6b 39 2024-04-22T23:45:43.049 INFO:tasks.workunit.client.0.smithi031.stdout:8/400: dwrite d5/da/d12/d2f/f51 [0,4194304] 0 2024-04-22T23:45:43.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/401: read - d5/da/d12/d2e/d4d/f62 zero size 2024-04-22T23:45:43.056 INFO:tasks.workunit.client.0.smithi031.stdout:4/574: dwrite db/d14/d2d/d49/d59/d6a/d41/f4b [0,4194304] 0 2024-04-22T23:45:43.059 INFO:tasks.workunit.client.0.smithi031.stdout:7/427: symlink d9/d14/d22/l97 0 2024-04-22T23:45:43.064 INFO:tasks.workunit.client.0.smithi031.stdout:9/449: rename d2/dc/d21 to d2/dc/d8b 0 2024-04-22T23:45:43.066 INFO:tasks.workunit.client.0.smithi031.stdout:0/397: mkdir d0/d13/d18/d1e/d25/d8e 0 2024-04-22T23:45:43.071 INFO:tasks.workunit.client.0.smithi031.stdout:3/365: symlink d4/d10/d2e/l77 0 2024-04-22T23:45:43.072 INFO:tasks.workunit.client.0.smithi031.stdout:6/483: rename d3/d1f/l54 to d3/d2a/d3c/d40/d4d/lac 0 2024-04-22T23:45:43.076 INFO:tasks.workunit.client.0.smithi031.stdout:2/449: truncate d6/dc/d42/d24/d32/f3a 325948 0 2024-04-22T23:45:43.079 INFO:tasks.workunit.client.0.smithi031.stdout:8/402: creat d5/d48/d61/d6b/d6c/f75 x:0 0 0 2024-04-22T23:45:43.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/575: rmdir db/d31/d40/d9b 39 2024-04-22T23:45:43.083 INFO:tasks.workunit.client.0.smithi031.stdout:4/576: fsync db/d14/d1d/d7b/d7f/d2a/fb1 0 2024-04-22T23:45:43.083 INFO:tasks.workunit.client.0.smithi031.stdout:1/403: write d1/d47/d7d/f8a [355062,1269] 0 2024-04-22T23:45:43.084 INFO:tasks.workunit.client.0.smithi031.stdout:7/428: dread d9/f37 [0,4194304] 0 2024-04-22T23:45:43.085 INFO:tasks.workunit.client.0.smithi031.stdout:9/450: chown d2/c37 4365 1 2024-04-22T23:45:43.087 INFO:tasks.workunit.client.0.smithi031.stdout:1/404: dread d1/d35/f7b [0,4194304] 0 2024-04-22T23:45:43.098 INFO:tasks.workunit.client.0.smithi031.stdout:7/429: creat d9/d14/d22/d5c/d5f/f98 x:0 0 0 2024-04-22T23:45:43.099 INFO:tasks.workunit.client.0.smithi031.stdout:9/451: symlink d2/dc/d6d/l8c 0 2024-04-22T23:45:43.099 INFO:tasks.workunit.client.0.smithi031.stdout:4/577: dwrite db/d31/d4f/f78 [0,4194304] 0 2024-04-22T23:45:43.102 INFO:tasks.workunit.client.0.smithi031.stdout:0/398: write d0/d1/d2/d36/d11/d1c/f88 [1004754,116509] 0 2024-04-22T23:45:43.102 INFO:tasks.workunit.client.0.smithi031.stdout:4/578: write db/d14/d1d/d7b/d7f/d2a/d30/fce [597250,43157] 0 2024-04-22T23:45:43.106 INFO:tasks.workunit.client.0.smithi031.stdout:6/484: rename d3/l1b to d3/d35/lad 0 2024-04-22T23:45:43.107 INFO:tasks.workunit.client.0.smithi031.stdout:2/450: creat d6/dc/d17/d69/d90/f91 x:0 0 0 2024-04-22T23:45:43.109 INFO:tasks.workunit.client.0.smithi031.stdout:0/399: write d0/d13/d18/d6c/f74 [390647,122832] 0 2024-04-22T23:45:43.109 INFO:tasks.workunit.client.0.smithi031.stdout:3/366: truncate d4/d6/da/ff 756164 0 2024-04-22T23:45:43.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/485: write d3/f78 [5315,111550] 0 2024-04-22T23:45:43.110 INFO:tasks.workunit.client.0.smithi031.stdout:1/405: dread d1/f6 [0,4194304] 0 2024-04-22T23:45:43.113 INFO:tasks.workunit.client.0.smithi031.stdout:9/452: dwrite d2/f3c [4194304,4194304] 0 2024-04-22T23:45:43.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/367: creat d4/d54/f78 x:0 0 0 2024-04-22T23:45:43.116 INFO:tasks.workunit.client.0.smithi031.stdout:1/406: readlink d1/de/d1e/d34/d4d/l89 0 2024-04-22T23:45:43.116 INFO:tasks.workunit.client.0.smithi031.stdout:0/400: fsync d0/d1/d2/d36/d11/d1a/d3c/f4e 0 2024-04-22T23:45:43.118 INFO:tasks.workunit.client.0.smithi031.stdout:6/486: chown d3/d1f/d4c/l51 435773 1 2024-04-22T23:45:43.119 INFO:tasks.workunit.client.0.smithi031.stdout:2/451: write d6/dc/d42/f43 [935302,112196] 0 2024-04-22T23:45:43.120 INFO:tasks.workunit.client.0.smithi031.stdout:4/579: dread db/d14/d2d/d49/d59/d6a/d63/fa5 [0,4194304] 0 2024-04-22T23:45:43.120 INFO:tasks.workunit.client.0.smithi031.stdout:8/403: rename d5/da/d12/d37/f3c to d5/f76 0 2024-04-22T23:45:43.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/453: creat d2/d15/d5a/f8d x:0 0 0 2024-04-22T23:45:43.123 INFO:tasks.workunit.client.0.smithi031.stdout:6/487: read d3/d2a/d3c/f71 [3407693,67248] 0 2024-04-22T23:45:43.124 INFO:tasks.workunit.client.0.smithi031.stdout:3/368: creat d4/d6/d4c/f79 x:0 0 0 2024-04-22T23:45:43.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/407: rename d1/de/d1e/d34/d4d to d1/d3f/d90 0 2024-04-22T23:45:43.132 INFO:tasks.workunit.client.0.smithi031.stdout:7/430: dwrite d9/d14/d22/f2d [0,4194304] 0 2024-04-22T23:45:43.132 INFO:tasks.workunit.client.0.smithi031.stdout:8/404: creat d5/d48/f77 x:0 0 0 2024-04-22T23:45:43.134 INFO:tasks.workunit.client.0.smithi031.stdout:0/401: mknod d0/d1/d2/d36/d11/d1a/d3c/c8f 0 2024-04-22T23:45:43.137 INFO:tasks.workunit.client.0.smithi031.stdout:0/402: read d0/d1/d2/d36/d11/d1a/d3c/f40 [1030555,129555] 0 2024-04-22T23:45:43.138 INFO:tasks.workunit.client.0.smithi031.stdout:6/488: write d3/d1f/f2f [5515748,94151] 0 2024-04-22T23:45:43.141 INFO:tasks.workunit.client.0.smithi031.stdout:6/489: dread - d3/d77/d8d/fa0 zero size 2024-04-22T23:45:43.142 INFO:tasks.workunit.client.0.smithi031.stdout:6/490: chown d3/d1f/f2f 37 1 2024-04-22T23:45:43.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/454: dwrite d2/dc/f50 [0,4194304] 0 2024-04-22T23:45:43.146 INFO:tasks.workunit.client.0.smithi031.stdout:7/431: rmdir d9/d15/d3a 39 2024-04-22T23:45:43.147 INFO:tasks.workunit.client.0.smithi031.stdout:8/405: read - d5/da/df/d33/f5c zero size 2024-04-22T23:45:43.148 INFO:tasks.workunit.client.0.smithi031.stdout:2/452: dread d6/ff [0,4194304] 0 2024-04-22T23:45:43.149 INFO:tasks.workunit.client.0.smithi031.stdout:2/453: chown d6/dc/d48 32340 1 2024-04-22T23:45:43.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/406: dwrite d5/da/f59 [0,4194304] 0 2024-04-22T23:45:43.156 INFO:tasks.workunit.client.0.smithi031.stdout:0/403: rmdir d0/d1/d2 39 2024-04-22T23:45:43.157 INFO:tasks.workunit.client.0.smithi031.stdout:3/369: write d4/d6/d11/f37 [80665,72950] 0 2024-04-22T23:45:43.158 INFO:tasks.workunit.client.0.smithi031.stdout:3/370: write d4/d6/da/f13 [2041807,99614] 0 2024-04-22T23:45:43.162 INFO:tasks.workunit.client.0.smithi031.stdout:6/491: fsync d3/d1f/f28 0 2024-04-22T23:45:43.164 INFO:tasks.workunit.client.0.smithi031.stdout:6/492: read - d3/d2a/d85/f9d zero size 2024-04-22T23:45:43.164 INFO:tasks.workunit.client.0.smithi031.stdout:4/580: dread db/d31/d40/d9b/fa6 [0,4194304] 0 2024-04-22T23:45:43.164 INFO:tasks.workunit.client.0.smithi031.stdout:6/493: chown d3/f78 22 1 2024-04-22T23:45:43.171 INFO:tasks.workunit.client.0.smithi031.stdout:4/581: dwrite db/d14/d1d/d7b/d7f/d2a/d48/f5d [0,4194304] 0 2024-04-22T23:45:43.171 INFO:tasks.workunit.client.0.smithi031.stdout:0/404: creat d0/d1/d28/d2b/f90 x:0 0 0 2024-04-22T23:45:43.175 INFO:tasks.workunit.client.0.smithi031.stdout:9/455: unlink d2/d15/l1f 0 2024-04-22T23:45:43.176 INFO:tasks.workunit.client.0.smithi031.stdout:5/413: sync 2024-04-22T23:45:43.178 INFO:tasks.workunit.client.0.smithi031.stdout:6/494: mkdir d3/d35/d86/dae 0 2024-04-22T23:45:43.180 INFO:tasks.workunit.client.0.smithi031.stdout:2/454: mkdir d6/dc/d48/d8a/d92 0 2024-04-22T23:45:43.181 INFO:tasks.workunit.client.0.smithi031.stdout:2/455: chown d6/f8b 977698022 1 2024-04-22T23:45:43.182 INFO:tasks.workunit.client.0.smithi031.stdout:2/456: fdatasync d6/d45/d68/d6c/f76 0 2024-04-22T23:45:43.184 INFO:tasks.workunit.client.0.smithi031.stdout:0/405: mknod d0/c91 0 2024-04-22T23:45:43.186 INFO:tasks.workunit.client.0.smithi031.stdout:3/371: mknod d4/d6/c7a 0 2024-04-22T23:45:43.186 INFO:tasks.workunit.client.0.smithi031.stdout:6/495: dwrite d3/d1f/d60/f62 [0,4194304] 0 2024-04-22T23:45:43.187 INFO:tasks.workunit.client.0.smithi031.stdout:1/408: getdents d1/d47 0 2024-04-22T23:45:43.188 INFO:tasks.workunit.client.0.smithi031.stdout:9/456: mkdir d2/dd/d75/d64/d5d/d59/d8e 0 2024-04-22T23:45:43.189 INFO:tasks.workunit.client.0.smithi031.stdout:1/409: chown d1/d35/d36 1228 1 2024-04-22T23:45:43.189 INFO:tasks.workunit.client.0.smithi031.stdout:6/496: readlink d3/d2a/d3c/d40/d4d/l5a 0 2024-04-22T23:45:43.193 INFO:tasks.workunit.client.0.smithi031.stdout:6/497: write d3/f8 [321453,32383] 0 2024-04-22T23:45:43.193 INFO:tasks.workunit.client.0.smithi031.stdout:9/457: chown d2/dc/f4b 15460 1 2024-04-22T23:45:43.196 INFO:tasks.workunit.client.0.smithi031.stdout:6/498: write d3/d1f/d4c/f56 [1087421,5752] 0 2024-04-22T23:45:43.196 INFO:tasks.workunit.client.0.smithi031.stdout:9/458: write d2/dc/f50 [973033,3173] 0 2024-04-22T23:45:43.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/407: write d5/f9 [3391577,30609] 0 2024-04-22T23:45:43.196 INFO:tasks.workunit.client.0.smithi031.stdout:4/582: dread db/d14/d1d/d7b/d7f/d2a/d30/f5e [0,4194304] 0 2024-04-22T23:45:43.198 INFO:tasks.workunit.client.0.smithi031.stdout:6/499: chown d3/d1f/f64 256977053 1 2024-04-22T23:45:43.198 INFO:tasks.workunit.client.0.smithi031.stdout:3/372: symlink d4/d10/d2e/l7b 0 2024-04-22T23:45:43.199 INFO:tasks.workunit.client.0.smithi031.stdout:5/414: mkdir db/d16/d2a/d7c/d89 0 2024-04-22T23:45:43.200 INFO:tasks.workunit.client.0.smithi031.stdout:9/459: read d2/dc/d8b/d3d/d40/f5c [44053,58741] 0 2024-04-22T23:45:43.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/500: chown d3/d2a/d3c/d40/d7a 3104179 1 2024-04-22T23:45:43.202 INFO:tasks.workunit.client.0.smithi031.stdout:1/410: mknod d1/db/d16/d1f/c91 0 2024-04-22T23:45:43.204 INFO:tasks.workunit.client.0.smithi031.stdout:1/411: truncate d1/d21/d29/f1b 5947101 0 2024-04-22T23:45:43.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/412: write d1/de/d1e/d34/d41/f42 [2641659,37856] 0 2024-04-22T23:45:43.211 INFO:tasks.workunit.client.0.smithi031.stdout:7/432: dread d9/d15/d3a/d41/f5a [0,4194304] 0 2024-04-22T23:45:43.212 INFO:tasks.workunit.client.0.smithi031.stdout:0/406: dwrite d0/d13/d18/d1e/f6a [4194304,4194304] 0 2024-04-22T23:45:43.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/413: dread d1/f6 [0,4194304] 0 2024-04-22T23:45:43.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/460: write d2/dd/d75/d80/f82 [4360015,104255] 0 2024-04-22T23:45:43.222 INFO:tasks.workunit.client.0.smithi031.stdout:4/583: link db/d14/d2d/d49/d59/d6a/d63/fa5 db/d14/d1d/fd0 0 2024-04-22T23:45:43.227 INFO:tasks.workunit.client.0.smithi031.stdout:5/415: dread f5 [0,4194304] 0 2024-04-22T23:45:43.228 INFO:tasks.workunit.client.0.smithi031.stdout:3/373: link d4/d6/d11/f19 d4/d6/d11/d3e/f7c 0 2024-04-22T23:45:43.229 INFO:tasks.workunit.client.0.smithi031.stdout:1/414: creat d1/de/d1e/d34/d66/f92 x:0 0 0 2024-04-22T23:45:43.239 INFO:tasks.workunit.client.0.smithi031.stdout:7/433: write f2 [205810,100228] 0 2024-04-22T23:45:43.240 INFO:tasks.workunit.client.0.smithi031.stdout:0/407: creat d0/d1/d2/d36/d11/f92 x:0 0 0 2024-04-22T23:45:43.241 INFO:tasks.workunit.client.0.smithi031.stdout:5/416: dread db/f23 [0,4194304] 0 2024-04-22T23:45:43.241 INFO:tasks.workunit.client.0.smithi031.stdout:7/434: dread - d9/d15/d6d/d65/f90 zero size 2024-04-22T23:45:43.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/415: rename d1/db/d16/l3c to d1/d47/d7d/l93 0 2024-04-22T23:45:43.244 INFO:tasks.workunit.client.0.smithi031.stdout:5/417: dread db/fd [0,4194304] 0 2024-04-22T23:45:43.245 INFO:tasks.workunit.client.0.smithi031.stdout:1/416: write d1/de/d1e/d34/d41/f62 [505472,6803] 0 2024-04-22T23:45:43.247 INFO:tasks.workunit.client.0.smithi031.stdout:8/408: truncate d5/da/d12/d37/d34/f46 338911 0 2024-04-22T23:45:43.248 INFO:tasks.workunit.client.0.smithi031.stdout:2/457: link d6/dc/d42/d24/d32/l33 d6/dc/d17/d69/d78/l93 0 2024-04-22T23:45:43.248 INFO:tasks.workunit.client.0.smithi031.stdout:2/458: chown d6/dc/d48/c5d 520472 1 2024-04-22T23:45:43.249 INFO:tasks.workunit.client.0.smithi031.stdout:9/461: write d2/dc/f12 [720614,127085] 0 2024-04-22T23:45:43.250 INFO:tasks.workunit.client.0.smithi031.stdout:4/584: unlink db/cd 0 2024-04-22T23:45:43.253 INFO:tasks.workunit.client.0.smithi031.stdout:0/408: rmdir d0/d13/d18 39 2024-04-22T23:45:43.255 INFO:tasks.workunit.client.0.smithi031.stdout:9/462: dwrite d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:43.259 INFO:tasks.workunit.client.0.smithi031.stdout:6/501: link d3/d2a/d3c/d40/c49 d3/d1f/caf 0 2024-04-22T23:45:43.260 INFO:tasks.workunit.client.0.smithi031.stdout:6/502: fsync d3/d1f/f64 0 2024-04-22T23:45:43.268 INFO:tasks.workunit.client.0.smithi031.stdout:2/459: chown d6/dc/d42/f2c 218544 1 2024-04-22T23:45:43.269 INFO:tasks.workunit.client.0.smithi031.stdout:2/460: stat d6/d45/d68/d6c 0 2024-04-22T23:45:43.270 INFO:tasks.workunit.client.0.smithi031.stdout:0/409: creat d0/d1/d3f/f93 x:0 0 0 2024-04-22T23:45:43.271 INFO:tasks.workunit.client.0.smithi031.stdout:3/374: truncate d4/d10/d2e/f4f 96385 0 2024-04-22T23:45:43.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/375: chown d4/d6/d11/d3e/f7c 427413034 1 2024-04-22T23:45:43.275 INFO:tasks.workunit.client.0.smithi031.stdout:3/376: dread d4/d6/d11/d3e/f43 [0,4194304] 0 2024-04-22T23:45:43.279 INFO:tasks.workunit.client.0.smithi031.stdout:6/503: mknod d3/d2a/cb0 0 2024-04-22T23:45:43.280 INFO:tasks.workunit.client.0.smithi031.stdout:6/504: write d3/d2a/d3c/d40/d5b/faa [696578,12996] 0 2024-04-22T23:45:43.282 INFO:tasks.workunit.client.0.smithi031.stdout:6/505: write d3/d1f/d4c/d8e/f93 [993002,108310] 0 2024-04-22T23:45:43.285 INFO:tasks.workunit.client.0.smithi031.stdout:5/418: write db/d16/f2b [543201,68457] 0 2024-04-22T23:45:43.287 INFO:tasks.workunit.client.0.smithi031.stdout:5/419: fdatasync db/d16/f54 0 2024-04-22T23:45:43.287 INFO:tasks.workunit.client.0.smithi031.stdout:1/417: dwrite d1/db/d16/f83 [0,4194304] 0 2024-04-22T23:45:43.289 INFO:tasks.workunit.client.0.smithi031.stdout:7/435: creat d9/d15/f99 x:0 0 0 2024-04-22T23:45:43.292 INFO:tasks.workunit.client.0.smithi031.stdout:0/410: write d0/d13/d18/d1e/d25/f38 [355685,67475] 0 2024-04-22T23:45:43.294 INFO:tasks.workunit.client.0.smithi031.stdout:9/463: chown d2/dd/d75/d64/d5d/d59/d8e 3832 1 2024-04-22T23:45:43.298 INFO:tasks.workunit.client.0.smithi031.stdout:1/418: dwrite d1/d21/d4e/f71 [0,4194304] 0 2024-04-22T23:45:43.298 INFO:tasks.workunit.client.0.smithi031.stdout:6/506: mkdir d3/d35/db1 0 2024-04-22T23:45:43.301 INFO:tasks.workunit.client.0.smithi031.stdout:0/411: write d0/d1/d2/d36/d11/d1c/d54/f72 [1324641,98786] 0 2024-04-22T23:45:43.301 INFO:tasks.workunit.client.0.smithi031.stdout:2/461: mkdir d6/dc/d42/d28/d94 0 2024-04-22T23:45:43.303 INFO:tasks.workunit.client.0.smithi031.stdout:4/585: rename db/d31/l91 to db/ld1 0 2024-04-22T23:45:43.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/436: dwrite d9/d14/d22/f52 [0,4194304] 0 2024-04-22T23:45:43.307 INFO:tasks.workunit.client.0.smithi031.stdout:5/420: dread db/f3f [0,4194304] 0 2024-04-22T23:45:43.308 INFO:tasks.workunit.client.0.smithi031.stdout:7/437: dread - d9/d15/d3a/d41/f83 zero size 2024-04-22T23:45:43.311 INFO:tasks.workunit.client.0.smithi031.stdout:1/419: dread d1/db/d16/f83 [0,4194304] 0 2024-04-22T23:45:43.311 INFO:tasks.workunit.client.0.smithi031.stdout:4/586: truncate db/d14/d2d/d49/d59/d6a/d63/d86/fca 357856 0 2024-04-22T23:45:43.313 INFO:tasks.workunit.client.0.smithi031.stdout:2/462: dread d6/dc/d57/d52/f6a [0,4194304] 0 2024-04-22T23:45:43.313 INFO:tasks.workunit.client.0.smithi031.stdout:3/377: link d4/d6/d11/f1c d4/d6/d4c/d4d/f7d 0 2024-04-22T23:45:43.321 INFO:tasks.workunit.client.0.smithi031.stdout:6/507: creat d3/d1f/d4c/d8e/fb2 x:0 0 0 2024-04-22T23:45:43.329 INFO:tasks.workunit.client.0.smithi031.stdout:4/587: rmdir db/d14/d1d/d7b/d7f 39 2024-04-22T23:45:43.330 INFO:tasks.workunit.client.0.smithi031.stdout:2/463: creat d6/d45/f95 x:0 0 0 2024-04-22T23:45:43.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/464: chown d6/dc/d42/f43 505451529 1 2024-04-22T23:45:43.333 INFO:tasks.workunit.client.0.smithi031.stdout:3/378: rename d4/d54/d56/f65 to d4/d6/d11/d25/d44/f7e 0 2024-04-22T23:45:43.334 INFO:tasks.workunit.client.0.smithi031.stdout:8/409: dwrite d5/da/d12/d37/d34/f46 [0,4194304] 0 2024-04-22T23:45:43.369 INFO:tasks.workunit.client.0.smithi031.stdout:7/438: write d9/f3e [502101,68961] 0 2024-04-22T23:45:43.369 INFO:tasks.workunit.client.0.smithi031.stdout:2/465: fsync d6/d45/f95 0 2024-04-22T23:45:43.369 INFO:tasks.workunit.client.0.smithi031.stdout:8/410: read d5/da/d12/d2e/d4d/f52 [3342640,10068] 0 2024-04-22T23:45:43.369 INFO:tasks.workunit.client.0.smithi031.stdout:6/508: write d3/f1a [913861,128956] 0 2024-04-22T23:45:43.371 INFO:tasks.workunit.client.0.smithi031.stdout:9/464: truncate d2/dc/f10 1501125 0 2024-04-22T23:45:43.371 INFO:tasks.workunit.client.0.smithi031.stdout:1/420: mkdir d1/de/d1e/d94 0 2024-04-22T23:45:43.372 INFO:tasks.workunit.client.0.smithi031.stdout:3/379: rename d4/d6/d4c/d4d to d4/d6/d11/d25/d44/d7f 0 2024-04-22T23:45:43.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/439: write d9/d14/d22/d28/d7a/f7d [1306006,86948] 0 2024-04-22T23:45:43.384 INFO:tasks.workunit.client.0.smithi031.stdout:5/421: link db/d16/c3d db/d16/d1d/d24/d27/c8a 0 2024-04-22T23:45:43.384 INFO:tasks.workunit.client.0.smithi031.stdout:1/421: chown d1/d21/d29/d28/c49 719 1 2024-04-22T23:45:43.385 INFO:tasks.workunit.client.0.smithi031.stdout:9/465: unlink d2/dc/f4b 0 2024-04-22T23:45:43.386 INFO:tasks.workunit.client.0.smithi031.stdout:5/422: chown f5 1 1 2024-04-22T23:45:43.386 INFO:tasks.workunit.client.0.smithi031.stdout:8/411: creat d5/da/d12/d37/f78 x:0 0 0 2024-04-22T23:45:43.387 INFO:tasks.workunit.client.0.smithi031.stdout:0/412: getdents d0/d1/d2/d36 0 2024-04-22T23:45:43.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/440: creat d9/d14/d22/d5c/f9a x:0 0 0 2024-04-22T23:45:43.392 INFO:tasks.workunit.client.0.smithi031.stdout:3/380: dwrite d4/d10/d2e/d45/f6c [0,4194304] 0 2024-04-22T23:45:43.393 INFO:tasks.workunit.client.0.smithi031.stdout:6/509: link d3/c13 d3/da5/cb3 0 2024-04-22T23:45:43.394 INFO:tasks.workunit.client.0.smithi031.stdout:9/466: rmdir d2/dc/d8b/d3d 39 2024-04-22T23:45:43.396 INFO:tasks.workunit.client.0.smithi031.stdout:0/413: symlink d0/d1/d3f/l94 0 2024-04-22T23:45:43.401 INFO:tasks.workunit.client.0.smithi031.stdout:2/466: write d6/dc/d49/f70 [5031694,26817] 0 2024-04-22T23:45:43.402 INFO:tasks.workunit.client.0.smithi031.stdout:6/510: dwrite d3/d35/d75/d7d/f87 [0,4194304] 0 2024-04-22T23:45:43.403 INFO:tasks.workunit.client.0.smithi031.stdout:6/511: read - d3/d2a/d3c/f52 zero size 2024-04-22T23:45:43.408 INFO:tasks.workunit.client.0.smithi031.stdout:7/441: dread d9/d15/d6d/d65/f89 [0,4194304] 0 2024-04-22T23:45:43.410 INFO:tasks.workunit.client.0.smithi031.stdout:1/422: creat d1/de/d1e/f95 x:0 0 0 2024-04-22T23:45:43.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/423: creat db/d16/f8b x:0 0 0 2024-04-22T23:45:43.416 INFO:tasks.workunit.client.0.smithi031.stdout:9/467: creat d2/dc/d8b/d3d/f8f x:0 0 0 2024-04-22T23:45:43.421 INFO:tasks.workunit.client.0.smithi031.stdout:1/423: creat d1/d35/f96 x:0 0 0 2024-04-22T23:45:43.425 INFO:tasks.workunit.client.0.smithi031.stdout:3/381: creat d4/d6/f80 x:0 0 0 2024-04-22T23:45:43.427 INFO:tasks.workunit.client.0.smithi031.stdout:6/512: unlink d3/d1f/c4b 0 2024-04-22T23:45:43.429 INFO:tasks.workunit.client.0.smithi031.stdout:9/468: mknod d2/dd/d75/d80/c90 0 2024-04-22T23:45:43.429 INFO:tasks.workunit.client.0.smithi031.stdout:6/513: dread - d3/d2a/d3c/f98 zero size 2024-04-22T23:45:43.430 INFO:tasks.workunit.client.0.smithi031.stdout:0/414: rmdir d0/d1/d2/d36/d11/d1c/d5f 0 2024-04-22T23:45:43.431 INFO:tasks.workunit.client.0.smithi031.stdout:7/442: truncate d9/d15/f17 2275961 0 2024-04-22T23:45:43.432 INFO:tasks.workunit.client.0.smithi031.stdout:3/382: dwrite d4/d54/d56/f75 [0,4194304] 0 2024-04-22T23:45:43.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/467: symlink d6/dc/d17/d69/d78/l96 0 2024-04-22T23:45:43.435 INFO:tasks.workunit.client.0.smithi031.stdout:1/424: fsync d1/db/d16/d1f/f26 0 2024-04-22T23:45:43.439 INFO:tasks.workunit.client.0.smithi031.stdout:6/514: unlink d3/d2a/cb0 0 2024-04-22T23:45:43.443 INFO:tasks.workunit.client.0.smithi031.stdout:7/443: mknod d9/d14/d22/d28/c9b 0 2024-04-22T23:45:43.443 INFO:tasks.workunit.client.0.smithi031.stdout:3/383: symlink d4/d6/d11/d25/d44/l81 0 2024-04-22T23:45:43.445 INFO:tasks.workunit.client.0.smithi031.stdout:6/515: dwrite d3/d2a/d3c/d65/f82 [0,4194304] 0 2024-04-22T23:45:43.446 INFO:tasks.workunit.client.0.smithi031.stdout:6/516: chown d3 262347 1 2024-04-22T23:45:43.447 INFO:tasks.workunit.client.0.smithi031.stdout:6/517: chown d3/d2a/f70 425234 1 2024-04-22T23:45:43.449 INFO:tasks.workunit.client.0.smithi031.stdout:5/424: rename db/d16/c3d to db/d16/d1d/c8c 0 2024-04-22T23:45:43.452 INFO:tasks.workunit.client.0.smithi031.stdout:1/425: mknod d1/d47/d7d/c97 0 2024-04-22T23:45:43.457 INFO:tasks.workunit.client.0.smithi031.stdout:3/384: creat d4/d54/d6d/f82 x:0 0 0 2024-04-22T23:45:43.459 INFO:tasks.workunit.client.0.smithi031.stdout:6/518: creat d3/d2a/d3c/d65/fb4 x:0 0 0 2024-04-22T23:45:43.464 INFO:tasks.workunit.client.0.smithi031.stdout:1/426: creat d1/d21/d29/f98 x:0 0 0 2024-04-22T23:45:43.468 INFO:tasks.workunit.client.0.smithi031.stdout:7/444: symlink d9/d14/d1f/d45/l9c 0 2024-04-22T23:45:43.469 INFO:tasks.workunit.client.0.smithi031.stdout:2/468: rmdir d6/dc/d42/d28/d7c 0 2024-04-22T23:45:43.470 INFO:tasks.workunit.client.0.smithi031.stdout:2/469: dread - d6/dc/d48/d4b/f7d zero size 2024-04-22T23:45:43.470 INFO:tasks.workunit.client.0.smithi031.stdout:6/519: mkdir d3/d77/db5 0 2024-04-22T23:45:43.472 INFO:tasks.workunit.client.0.smithi031.stdout:5/425: mkdir db/d8d 0 2024-04-22T23:45:43.474 INFO:tasks.workunit.client.0.smithi031.stdout:5/426: truncate db/d16/d1d/d24/d58/d5d/f7e 187475 0 2024-04-22T23:45:43.474 INFO:tasks.workunit.client.0.smithi031.stdout:5/427: chown db/d16/d2a/d59/f88 7769436 1 2024-04-22T23:45:43.475 INFO:tasks.workunit.client.0.smithi031.stdout:5/428: fdatasync db/f25 0 2024-04-22T23:45:43.480 INFO:tasks.workunit.client.0.smithi031.stdout:9/469: getdents d2/dc/d8b 0 2024-04-22T23:45:43.480 INFO:tasks.workunit.client.0.smithi031.stdout:3/385: mkdir d4/d83 0 2024-04-22T23:45:43.481 INFO:tasks.workunit.client.0.smithi031.stdout:3/386: stat d4/d6/d11/d25/d44/d7f/f7d 0 2024-04-22T23:45:43.481 INFO:tasks.workunit.client.0.smithi031.stdout:3/387: stat d4/d49/f5e 0 2024-04-22T23:45:43.485 INFO:tasks.workunit.client.0.smithi031.stdout:2/470: creat d6/dc/d17/d69/f97 x:0 0 0 2024-04-22T23:45:43.490 INFO:tasks.workunit.client.0.smithi031.stdout:5/429: creat db/d16/d1d/d24/d58/d78/f8e x:0 0 0 2024-04-22T23:45:43.497 INFO:tasks.workunit.client.0.smithi031.stdout:5/430: dwrite db/d16/d2a/d36/f6f [0,4194304] 0 2024-04-22T23:45:43.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/388: unlink d4/d49/f5e 0 2024-04-22T23:45:43.499 INFO:tasks.workunit.client.0.smithi031.stdout:5/431: read db/f25 [2946358,99882] 0 2024-04-22T23:45:43.503 INFO:tasks.workunit.client.0.smithi031.stdout:2/471: fsync d6/dc/d17/f25 0 2024-04-22T23:45:43.506 INFO:tasks.workunit.client.0.smithi031.stdout:1/427: rename d1/d47 to d1/d99 0 2024-04-22T23:45:43.507 INFO:tasks.workunit.client.0.smithi031.stdout:9/470: creat d2/dd/d75/d64/d5d/d59/d8e/f91 x:0 0 0 2024-04-22T23:45:43.513 INFO:tasks.workunit.client.0.smithi031.stdout:6/520: write d3/d2a/d3c/f73 [793098,98809] 0 2024-04-22T23:45:43.515 INFO:tasks.workunit.client.0.smithi031.stdout:3/389: mknod d4/d6/d22/c84 0 2024-04-22T23:45:43.516 INFO:tasks.workunit.client.0.smithi031.stdout:3/390: write d4/d6/f23 [961038,109763] 0 2024-04-22T23:45:43.517 INFO:tasks.workunit.client.0.smithi031.stdout:3/391: write d4/d6/d11/d3e/f5a [611050,78544] 0 2024-04-22T23:45:43.518 INFO:tasks.workunit.client.0.smithi031.stdout:2/472: chown d6/dc/d42/d24/d32/f3a 7979272 1 2024-04-22T23:45:43.520 INFO:tasks.workunit.client.0.smithi031.stdout:1/428: write d1/d99/d63/f6d [5164344,111207] 0 2024-04-22T23:45:43.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/429: fsync d1/de/d1e/f95 0 2024-04-22T23:45:43.521 INFO:tasks.workunit.client.0.smithi031.stdout:9/471: symlink d2/dd/d65/l92 0 2024-04-22T23:45:43.522 INFO:tasks.workunit.client.0.smithi031.stdout:1/430: write d1/db/d16/f40 [1578865,22314] 0 2024-04-22T23:45:43.523 INFO:tasks.workunit.client.0.smithi031.stdout:7/445: getdents d9/d14/d1f 0 2024-04-22T23:45:43.524 INFO:tasks.workunit.client.0.smithi031.stdout:7/446: dread - d9/d14/d22/d5c/f9a zero size 2024-04-22T23:45:43.525 INFO:tasks.workunit.client.0.smithi031.stdout:7/447: dread - d9/d14/d22/d5c/d5f/f98 zero size 2024-04-22T23:45:43.526 INFO:tasks.workunit.client.0.smithi031.stdout:3/392: fdatasync d4/f5 0 2024-04-22T23:45:43.527 INFO:tasks.workunit.client.0.smithi031.stdout:2/473: symlink d6/d45/d68/l98 0 2024-04-22T23:45:43.531 INFO:tasks.workunit.client.0.smithi031.stdout:6/521: dwrite d3/d2a/d3c/f98 [0,4194304] 0 2024-04-22T23:45:43.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/448: mknod d9/d14/d22/d28/c9d 0 2024-04-22T23:45:43.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/393: mkdir d4/d54/d85 0 2024-04-22T23:45:43.541 INFO:tasks.workunit.client.0.smithi031.stdout:2/474: write d6/dc/f1a [2596690,86491] 0 2024-04-22T23:45:43.542 INFO:tasks.workunit.client.0.smithi031.stdout:9/472: symlink d2/dc/d8b/d3d/d40/l93 0 2024-04-22T23:45:43.543 INFO:tasks.workunit.client.0.smithi031.stdout:1/431: rename d1/d99/d7d/l93 to d1/d21/d29/l9a 0 2024-04-22T23:45:43.544 INFO:tasks.workunit.client.0.smithi031.stdout:1/432: readlink d1/d3f/d90/l89 0 2024-04-22T23:45:43.545 INFO:tasks.workunit.client.0.smithi031.stdout:7/449: creat d9/d14/d22/d5c/f9e x:0 0 0 2024-04-22T23:45:43.546 INFO:tasks.workunit.client.0.smithi031.stdout:3/394: symlink d4/d6/d4c/l86 0 2024-04-22T23:45:43.547 INFO:tasks.workunit.client.0.smithi031.stdout:2/475: fsync d6/dc/d42/f5c 0 2024-04-22T23:45:43.550 INFO:tasks.workunit.client.0.smithi031.stdout:1/433: chown d1/db/d16/d2f/c43 0 1 2024-04-22T23:45:43.551 INFO:tasks.workunit.client.0.smithi031.stdout:6/522: mkdir d3/d2a/d3c/db6 0 2024-04-22T23:45:43.552 INFO:tasks.workunit.client.0.smithi031.stdout:1/434: write d1/db/d16/f40 [2173316,70255] 0 2024-04-22T23:45:43.553 INFO:tasks.workunit.client.0.smithi031.stdout:2/476: dwrite d6/dc/d42/f5c [0,4194304] 0 2024-04-22T23:45:43.555 INFO:tasks.workunit.client.0.smithi031.stdout:1/435: fdatasync d1/de/d1e/d34/d41/f62 0 2024-04-22T23:45:43.556 INFO:tasks.workunit.client.0.smithi031.stdout:9/473: symlink d2/dc/l94 0 2024-04-22T23:45:43.559 INFO:tasks.workunit.client.0.smithi031.stdout:7/450: symlink d9/l9f 0 2024-04-22T23:45:43.563 INFO:tasks.workunit.client.0.smithi031.stdout:1/436: creat d1/d21/f9b x:0 0 0 2024-04-22T23:45:43.564 INFO:tasks.workunit.client.0.smithi031.stdout:2/477: mkdir d6/dc/d49/d83/d85/d99 0 2024-04-22T23:45:43.565 INFO:tasks.workunit.client.0.smithi031.stdout:2/478: chown d6/dc/f62 1 1 2024-04-22T23:45:43.567 INFO:tasks.workunit.client.0.smithi031.stdout:6/523: creat d3/d35/fb7 x:0 0 0 2024-04-22T23:45:43.568 INFO:tasks.workunit.client.0.smithi031.stdout:9/474: rename d2/d15/d5a/l5b to d2/dd/d75/d64/d5d/d59/l95 0 2024-04-22T23:45:43.569 INFO:tasks.workunit.client.0.smithi031.stdout:3/395: dread d4/d6/da/f3d [0,4194304] 0 2024-04-22T23:45:43.569 INFO:tasks.workunit.client.0.smithi031.stdout:7/451: unlink d9/d14/l1a 0 2024-04-22T23:45:43.570 INFO:tasks.workunit.client.0.smithi031.stdout:3/396: chown d4/d6/d11/d25/c33 14727 1 2024-04-22T23:45:43.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/437: chown d1/d35/l48 2 1 2024-04-22T23:45:43.571 INFO:tasks.workunit.client.0.smithi031.stdout:6/524: symlink d3/d35/d86/lb8 0 2024-04-22T23:45:43.573 INFO:tasks.workunit.client.0.smithi031.stdout:9/475: fsync d2/d15/f7c 0 2024-04-22T23:45:43.574 INFO:tasks.workunit.client.0.smithi031.stdout:3/397: creat d4/d54/d56/f87 x:0 0 0 2024-04-22T23:45:43.578 INFO:tasks.workunit.client.0.smithi031.stdout:2/479: rename d6/d45/d68 to d6/dc/d9a 0 2024-04-22T23:45:43.580 INFO:tasks.workunit.client.0.smithi031.stdout:1/438: dwrite d1/d99/d7d/f8a [0,4194304] 0 2024-04-22T23:45:43.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/525: creat d3/da4/fb9 x:0 0 0 2024-04-22T23:45:43.589 INFO:tasks.workunit.client.0.smithi031.stdout:1/439: creat d1/d99/d63/f9c x:0 0 0 2024-04-22T23:45:43.595 INFO:tasks.workunit.client.0.smithi031.stdout:2/480: creat d6/dc/d9a/d6c/d86/f9b x:0 0 0 2024-04-22T23:45:43.597 INFO:tasks.workunit.client.0.smithi031.stdout:1/440: unlink d1/d21/c39 0 2024-04-22T23:45:43.600 INFO:tasks.workunit.client.0.smithi031.stdout:9/476: getdents d2/dd/d75 0 2024-04-22T23:45:43.602 INFO:tasks.workunit.client.0.smithi031.stdout:7/452: rename d9/d15/d3a/c48 to d9/ca0 0 2024-04-22T23:45:43.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/526: chown d3/d1f/l27 155743822 1 2024-04-22T23:45:43.605 INFO:tasks.workunit.client.0.smithi031.stdout:2/481: mkdir d6/dc/d42/d24/d9c 0 2024-04-22T23:45:43.606 INFO:tasks.workunit.client.0.smithi031.stdout:3/398: getdents d4/d10 0 2024-04-22T23:45:43.609 INFO:tasks.workunit.client.0.smithi031.stdout:9/477: mknod d2/d15/d5a/c96 0 2024-04-22T23:45:43.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/441: rename d1/de/d1e/d34/l5f to d1/d35/d36/l9d 0 2024-04-22T23:45:43.612 INFO:tasks.workunit.client.0.smithi031.stdout:1/442: chown d1/d21/d29/d28/c2b 286 1 2024-04-22T23:45:43.612 INFO:tasks.workunit.client.0.smithi031.stdout:7/453: symlink d9/d14/d22/d5c/la1 0 2024-04-22T23:45:43.614 INFO:tasks.workunit.client.0.smithi031.stdout:2/482: symlink d6/d45/l9d 0 2024-04-22T23:45:43.615 INFO:tasks.workunit.client.0.smithi031.stdout:3/399: mkdir d4/d54/d88 0 2024-04-22T23:45:43.617 INFO:tasks.workunit.client.0.smithi031.stdout:9/478: write d2/dc/f25 [244882,87320] 0 2024-04-22T23:45:43.617 INFO:tasks.workunit.client.0.smithi031.stdout:2/483: mkdir d6/dc/d9a/d9e 0 2024-04-22T23:45:43.619 INFO:tasks.workunit.client.0.smithi031.stdout:1/443: dread d1/f46 [0,4194304] 0 2024-04-22T23:45:43.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/484: readlink d6/dc/d9a/l98 0 2024-04-22T23:45:43.620 INFO:tasks.workunit.client.0.smithi031.stdout:2/485: chown d6/dc/d9a/l98 150795485 1 2024-04-22T23:45:43.620 INFO:tasks.workunit.client.0.smithi031.stdout:6/527: link d3/da4/fb9 d3/fba 0 2024-04-22T23:45:43.623 INFO:tasks.workunit.client.0.smithi031.stdout:3/400: rename d4/d10 to d4/d54/d2c/d89 0 2024-04-22T23:45:43.625 INFO:tasks.workunit.client.0.smithi031.stdout:1/444: creat d1/d35/d36/f9e x:0 0 0 2024-04-22T23:45:43.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/445: dwrite d1/db/d16/f40 [0,4194304] 0 2024-04-22T23:45:43.635 INFO:tasks.workunit.client.0.smithi031.stdout:3/401: dread d4/d54/d2c/d89/d2e/f4f [0,4194304] 0 2024-04-22T23:45:43.642 INFO:tasks.workunit.client.0.smithi031.stdout:6/528: mkdir d3/d35/d86/dae/dbb 0 2024-04-22T23:45:43.644 INFO:tasks.workunit.client.0.smithi031.stdout:8/412: sync 2024-04-22T23:45:43.644 INFO:tasks.workunit.client.0.smithi031.stdout:4/588: sync 2024-04-22T23:45:43.644 INFO:tasks.workunit.client.0.smithi031.stdout:9/479: rename d2/dc/ff to d2/dd/d75/f97 0 2024-04-22T23:45:43.646 INFO:tasks.workunit.client.0.smithi031.stdout:8/413: truncate d5/d48/f77 723221 0 2024-04-22T23:45:43.646 INFO:tasks.workunit.client.0.smithi031.stdout:7/454: getdents d9/d15/d3f 0 2024-04-22T23:45:43.649 INFO:tasks.workunit.client.0.smithi031.stdout:3/402: symlink d4/d54/d6d/l8a 0 2024-04-22T23:45:43.650 INFO:tasks.workunit.client.0.smithi031.stdout:3/403: chown d4/d6/f7 980139 1 2024-04-22T23:45:43.650 INFO:tasks.workunit.client.0.smithi031.stdout:8/414: chown d5/da/df/f13 272264 1 2024-04-22T23:45:43.651 INFO:tasks.workunit.client.0.smithi031.stdout:2/486: rmdir d6/dc/d49/d83/d85/d99 0 2024-04-22T23:45:43.655 INFO:tasks.workunit.client.0.smithi031.stdout:7/455: dwrite d9/d14/d22/d28/f36 [4194304,4194304] 0 2024-04-22T23:45:43.656 INFO:tasks.workunit.client.0.smithi031.stdout:9/480: truncate d2/dc/d8b/d3d/d40/f5c 601728 0 2024-04-22T23:45:43.657 INFO:tasks.workunit.client.0.smithi031.stdout:9/481: dread - d2/dc/f83 zero size 2024-04-22T23:45:43.663 INFO:tasks.workunit.client.0.smithi031.stdout:4/589: mkdir db/d14/d2d/d49/d59/d83/d9c/d72/dc6/dd2 0 2024-04-22T23:45:43.668 INFO:tasks.workunit.client.0.smithi031.stdout:3/404: creat d4/d83/f8b x:0 0 0 2024-04-22T23:45:43.669 INFO:tasks.workunit.client.0.smithi031.stdout:1/446: dwrite d1/de/d1e/d34/d41/f4c [0,4194304] 0 2024-04-22T23:45:43.669 INFO:tasks.workunit.client.0.smithi031.stdout:6/529: link d3/d2a/d3c/d40/d5b/c9b d3/d2a/d85/cbc 0 2024-04-22T23:45:43.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/415: creat d5/da/df/f79 x:0 0 0 2024-04-22T23:45:43.672 INFO:tasks.workunit.client.0.smithi031.stdout:7/456: rename d9/d14/d22/d28/f64 to d9/d14/d1f/d45/fa2 0 2024-04-22T23:45:43.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/487: creat d6/dc/f9f x:0 0 0 2024-04-22T23:45:43.675 INFO:tasks.workunit.client.0.smithi031.stdout:8/416: dread d5/da/d12/d37/d34/f4c [0,4194304] 0 2024-04-22T23:45:43.676 INFO:tasks.workunit.client.0.smithi031.stdout:3/405: mkdir d4/d6/d4c/d8c 0 2024-04-22T23:45:43.677 INFO:tasks.workunit.client.0.smithi031.stdout:3/406: chown d4/d6/c31 444959 1 2024-04-22T23:45:43.678 INFO:tasks.workunit.client.0.smithi031.stdout:6/530: fsync d3/d2a/d3c/f71 0 2024-04-22T23:45:43.680 INFO:tasks.workunit.client.0.smithi031.stdout:3/407: chown d4/d6/d11/d25/d44/d7f/f69 77474 1 2024-04-22T23:45:43.681 INFO:tasks.workunit.client.0.smithi031.stdout:1/447: rename d1/db/d88 to d1/d35/d36/d9f 0 2024-04-22T23:45:43.683 INFO:tasks.workunit.client.0.smithi031.stdout:9/482: symlink d2/dd/d75/d64/d5d/l98 0 2024-04-22T23:45:43.684 INFO:tasks.workunit.client.0.smithi031.stdout:9/483: stat d2/dc/d3e/f56 0 2024-04-22T23:45:43.685 INFO:tasks.workunit.client.0.smithi031.stdout:2/488: readlink d6/l10 0 2024-04-22T23:45:43.686 INFO:tasks.workunit.client.0.smithi031.stdout:2/489: write d6/f22 [34135,127847] 0 2024-04-22T23:45:43.686 INFO:tasks.workunit.client.0.smithi031.stdout:8/417: creat d5/d48/d61/d6b/d6c/f7a x:0 0 0 2024-04-22T23:45:43.694 INFO:tasks.workunit.client.0.smithi031.stdout:3/408: rename d4/d6/d4c/d4a to d4/d54/d2c/d89/d2e/d8d 0 2024-04-22T23:45:43.694 INFO:tasks.workunit.client.0.smithi031.stdout:5/432: sync 2024-04-22T23:45:43.694 INFO:tasks.workunit.client.0.smithi031.stdout:0/415: sync 2024-04-22T23:45:43.695 INFO:tasks.workunit.client.0.smithi031.stdout:8/418: dwrite d5/da/d12/d37/d34/f5e [0,4194304] 0 2024-04-22T23:45:43.696 INFO:tasks.workunit.client.0.smithi031.stdout:5/433: stat db/d16/d1d/d24/d27/f57 0 2024-04-22T23:45:43.698 INFO:tasks.workunit.client.0.smithi031.stdout:9/484: creat d2/dc/d3e/f99 x:0 0 0 2024-04-22T23:45:43.701 INFO:tasks.workunit.client.0.smithi031.stdout:4/590: dread db/d14/d2d/d49/d59/d83/d9c/f55 [4194304,4194304] 0 2024-04-22T23:45:43.701 INFO:tasks.workunit.client.0.smithi031.stdout:4/591: chown db/cc2 63417254 1 2024-04-22T23:45:43.702 INFO:tasks.workunit.client.0.smithi031.stdout:6/531: unlink d3/d1f/l42 0 2024-04-22T23:45:43.707 INFO:tasks.workunit.client.0.smithi031.stdout:4/592: dread db/d14/d2d/db7/fb8 [0,4194304] 0 2024-04-22T23:45:43.707 INFO:tasks.workunit.client.0.smithi031.stdout:7/457: truncate d9/f3b 2962652 0 2024-04-22T23:45:43.709 INFO:tasks.workunit.client.0.smithi031.stdout:7/458: chown d9/d14/d22/d5c/d5f/f98 0 1 2024-04-22T23:45:43.710 INFO:tasks.workunit.client.0.smithi031.stdout:0/416: stat d0/d1/d2/l65 0 2024-04-22T23:45:43.711 INFO:tasks.workunit.client.0.smithi031.stdout:7/459: truncate d9/d14/d22/d28/f82 976094 0 2024-04-22T23:45:43.712 INFO:tasks.workunit.client.0.smithi031.stdout:7/460: read - d9/d15/d6d/f8e zero size 2024-04-22T23:45:43.714 INFO:tasks.workunit.client.0.smithi031.stdout:6/532: dread d3/d2a/d3c/d40/d4d/f58 [0,4194304] 0 2024-04-22T23:45:43.715 INFO:tasks.workunit.client.0.smithi031.stdout:1/448: truncate d1/d99/d7d/f8a 2687238 0 2024-04-22T23:45:43.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/449: dread - d1/d99/d63/f9c zero size 2024-04-22T23:45:43.717 INFO:tasks.workunit.client.0.smithi031.stdout:3/409: write d4/d6/d11/d3e/f7c [5230306,51211] 0 2024-04-22T23:45:43.718 INFO:tasks.workunit.client.0.smithi031.stdout:9/485: stat d2/d15/f54 0 2024-04-22T23:45:43.720 INFO:tasks.workunit.client.0.smithi031.stdout:2/490: truncate d6/f19 2189129 0 2024-04-22T23:45:43.721 INFO:tasks.workunit.client.0.smithi031.stdout:2/491: fdatasync d6/dc/d42/d28/f2a 0 2024-04-22T23:45:43.722 INFO:tasks.workunit.client.0.smithi031.stdout:4/593: write db/d14/d2d/fb6 [7431900,129202] 0 2024-04-22T23:45:43.723 INFO:tasks.workunit.client.0.smithi031.stdout:4/594: fdatasync db/d14/d2d/d49/d59/fa2 0 2024-04-22T23:45:43.724 INFO:tasks.workunit.client.0.smithi031.stdout:4/595: chown db/d14/d1d/d7b/dc1 0 1 2024-04-22T23:45:43.724 INFO:tasks.workunit.client.0.smithi031.stdout:4/596: fdatasync db/f5f 0 2024-04-22T23:45:43.740 INFO:tasks.workunit.client.0.smithi031.stdout:0/417: symlink d0/d13/d18/d1e/d25/d8e/l95 0 2024-04-22T23:45:43.741 INFO:tasks.workunit.client.0.smithi031.stdout:8/419: dwrite d5/da/d12/d1d/d38/f3f [0,4194304] 0 2024-04-22T23:45:43.744 INFO:tasks.workunit.client.0.smithi031.stdout:1/450: mkdir d1/db/d16/da0 0 2024-04-22T23:45:43.746 INFO:tasks.workunit.client.0.smithi031.stdout:9/486: mkdir d2/d9a 0 2024-04-22T23:45:43.746 INFO:tasks.workunit.client.0.smithi031.stdout:4/597: dwrite db/d14/d2d/d49/d59/d6a/f92 [8388608,4194304] 0 2024-04-22T23:45:43.752 INFO:tasks.workunit.client.0.smithi031.stdout:0/418: symlink d0/d13/d18/d6c/l96 0 2024-04-22T23:45:43.752 INFO:tasks.workunit.client.0.smithi031.stdout:0/419: dread - d0/d13/d18/d1e/f6e zero size 2024-04-22T23:45:43.753 INFO:tasks.workunit.client.0.smithi031.stdout:5/434: getdents db/d16/d2a/d36 0 2024-04-22T23:45:43.756 INFO:tasks.workunit.client.0.smithi031.stdout:8/420: symlink d5/da/d12/d1d/d38/l7b 0 2024-04-22T23:45:43.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/421: stat d5/da/f1e 0 2024-04-22T23:45:43.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/422: chown d5/d48/d61/d6b/d6c/f75 3065354 1 2024-04-22T23:45:43.758 INFO:tasks.workunit.client.0.smithi031.stdout:7/461: truncate d9/f1c 4025217 0 2024-04-22T23:45:43.759 INFO:tasks.workunit.client.0.smithi031.stdout:7/462: chown d9/ce 789684 1 2024-04-22T23:45:43.760 INFO:tasks.workunit.client.0.smithi031.stdout:7/463: write d9/d15/f99 [669491,82922] 0 2024-04-22T23:45:43.760 INFO:tasks.workunit.client.0.smithi031.stdout:9/487: rename d2/dd/f13 to d2/d15/f9b 0 2024-04-22T23:45:43.761 INFO:tasks.workunit.client.0.smithi031.stdout:2/492: creat d6/dc/d42/fa0 x:0 0 0 2024-04-22T23:45:43.762 INFO:tasks.workunit.client.0.smithi031.stdout:0/420: symlink d0/d13/d18/d1e/l97 0 2024-04-22T23:45:43.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/493: fdatasync d6/dc/d42/d28/f2a 0 2024-04-22T23:45:43.765 INFO:tasks.workunit.client.0.smithi031.stdout:4/598: creat db/d14/d2d/d49/dcd/fd3 x:0 0 0 2024-04-22T23:45:43.766 INFO:tasks.workunit.client.0.smithi031.stdout:4/599: fdatasync db/d31/d40/f68 0 2024-04-22T23:45:43.767 INFO:tasks.workunit.client.0.smithi031.stdout:4/600: chown db/d14/d2d/d49/d59/d8a 2939928 1 2024-04-22T23:45:43.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/494: creat d6/dc/d49/d83/d85/fa1 x:0 0 0 2024-04-22T23:45:43.777 INFO:tasks.workunit.client.0.smithi031.stdout:5/435: creat db/d16/d2a/f8f x:0 0 0 2024-04-22T23:45:43.779 INFO:tasks.workunit.client.0.smithi031.stdout:4/601: symlink db/d14/d2d/d49/d59/d6a/d63/d86/ld4 0 2024-04-22T23:45:43.781 INFO:tasks.workunit.client.0.smithi031.stdout:0/421: dread d0/d1/d28/f55 [0,4194304] 0 2024-04-22T23:45:43.781 INFO:tasks.workunit.client.0.smithi031.stdout:4/602: chown db/d31/d40/d62/fba 26226880 1 2024-04-22T23:45:43.782 INFO:tasks.workunit.client.0.smithi031.stdout:2/495: creat d6/dc/d57/fa2 x:0 0 0 2024-04-22T23:45:43.784 INFO:tasks.workunit.client.0.smithi031.stdout:9/488: rename d2/c29 to d2/dd/d75/d4a/c9c 0 2024-04-22T23:45:43.784 INFO:tasks.workunit.client.0.smithi031.stdout:5/436: dwrite db/d16/f54 [4194304,4194304] 0 2024-04-22T23:45:43.788 INFO:tasks.workunit.client.0.smithi031.stdout:2/496: rmdir d6/d45/d50 39 2024-04-22T23:45:43.789 INFO:tasks.workunit.client.0.smithi031.stdout:0/422: write d0/d1/d2/d36/d11/d1a/d3c/f40 [3042600,68770] 0 2024-04-22T23:45:43.791 INFO:tasks.workunit.client.0.smithi031.stdout:8/423: dwrite d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:43.792 INFO:tasks.workunit.client.0.smithi031.stdout:5/437: unlink db/d16/c19 0 2024-04-22T23:45:43.794 INFO:tasks.workunit.client.0.smithi031.stdout:0/423: dread - d0/d1/d2/d36/f8d zero size 2024-04-22T23:45:43.801 INFO:tasks.workunit.client.0.smithi031.stdout:5/438: dread db/d16/d1d/f1f [0,4194304] 0 2024-04-22T23:45:43.802 INFO:tasks.workunit.client.0.smithi031.stdout:1/451: dwrite d1/d99/d7d/f8a [0,4194304] 0 2024-04-22T23:45:43.804 INFO:tasks.workunit.client.0.smithi031.stdout:9/489: dwrite d2/dd/d75/d80/f82 [0,4194304] 0 2024-04-22T23:45:43.804 INFO:tasks.workunit.client.0.smithi031.stdout:2/497: rename d6/dc/d42/f61 to d6/dc/d9a/d9e/fa3 0 2024-04-22T23:45:43.805 INFO:tasks.workunit.client.0.smithi031.stdout:1/452: chown d1/de/d1e 8888 1 2024-04-22T23:45:43.808 INFO:tasks.workunit.client.0.smithi031.stdout:8/424: dwrite d5/da/d12/d2e/d4d/d56/f5a [4194304,4194304] 0 2024-04-22T23:45:43.808 INFO:tasks.workunit.client.0.smithi031.stdout:1/453: dread d1/db/f19 [0,4194304] 0 2024-04-22T23:45:43.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/454: chown d1/d21/d29/d28/c2b 748025253 1 2024-04-22T23:45:43.813 INFO:tasks.workunit.client.0.smithi031.stdout:2/498: mkdir d6/dc/d9a/d9e/da4 0 2024-04-22T23:45:43.814 INFO:tasks.workunit.client.0.smithi031.stdout:5/439: dwrite db/d16/d1d/d24/d58/d78/f8e [0,4194304] 0 2024-04-22T23:45:43.817 INFO:tasks.workunit.client.0.smithi031.stdout:4/603: link db/d14/d1d/d7b/d7f/l9d db/d14/d2d/db7/ld5 0 2024-04-22T23:45:43.820 INFO:tasks.workunit.client.0.smithi031.stdout:4/604: truncate db/d14/d2d/d49/d59/d6a/d63/d86/fca 1074243 0 2024-04-22T23:45:43.821 INFO:tasks.workunit.client.0.smithi031.stdout:9/490: rename d2/dd/d75/d4a/f32 to d2/dd/d75/d64/d5d/d59/d8e/f9d 0 2024-04-22T23:45:43.825 INFO:tasks.workunit.client.0.smithi031.stdout:0/424: link d0/d1/d28/d2b/l7b d0/d1/d2/l98 0 2024-04-22T23:45:43.827 INFO:tasks.workunit.client.0.smithi031.stdout:1/455: mkdir d1/d3f/da1 0 2024-04-22T23:45:43.830 INFO:tasks.workunit.client.0.smithi031.stdout:4/605: creat db/d14/d2d/d49/d59/fd6 x:0 0 0 2024-04-22T23:45:43.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/606: write db/d14/d1d/d7b/d7f/d2a/d47/f7a [1353694,128434] 0 2024-04-22T23:45:43.886 INFO:tasks.workunit.client.0.smithi031.stdout:5/440: rename db/d16/d1d/d24/d58/d78 to db/d16/d2a/d59/d90 0 2024-04-22T23:45:43.891 INFO:tasks.workunit.client.0.smithi031.stdout:1/456: creat d1/de/d1e/d34/fa2 x:0 0 0 2024-04-22T23:45:43.891 INFO:tasks.workunit.client.0.smithi031.stdout:4/607: creat db/d14/d1d/fd7 x:0 0 0 2024-04-22T23:45:43.893 INFO:tasks.workunit.client.0.smithi031.stdout:5/441: write db/f42 [4717201,115703] 0 2024-04-22T23:45:43.897 INFO:tasks.workunit.client.0.smithi031.stdout:0/425: rename d0/d1/d28 to d0/d1/d2/d36/d11/d1a/d3c/d99 0 2024-04-22T23:45:43.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/457: creat d1/d21/d69/fa3 x:0 0 0 2024-04-22T23:45:43.903 INFO:tasks.workunit.client.0.smithi031.stdout:4/608: mkdir db/d14/d1d/dd8 0 2024-04-22T23:45:43.904 INFO:tasks.workunit.client.0.smithi031.stdout:9/491: link d2/dc/d8b/d3d/d40/l93 d2/dc/d6d/l9e 0 2024-04-22T23:45:43.905 INFO:tasks.workunit.client.0.smithi031.stdout:1/458: dread d1/f46 [0,4194304] 0 2024-04-22T23:45:43.906 INFO:tasks.workunit.client.0.smithi031.stdout:0/426: fdatasync d0/d1/d2/d36/d11/f43 0 2024-04-22T23:45:43.909 INFO:tasks.workunit.client.0.smithi031.stdout:1/459: read d1/d21/d29/f1a [317378,51608] 0 2024-04-22T23:45:43.910 INFO:tasks.workunit.client.0.smithi031.stdout:5/442: dread db/d16/f18 [0,4194304] 0 2024-04-22T23:45:43.911 INFO:tasks.workunit.client.0.smithi031.stdout:5/443: dread - db/d16/d2a/f8f zero size 2024-04-22T23:45:43.916 INFO:tasks.workunit.client.0.smithi031.stdout:0/427: mknod d0/c9a 0 2024-04-22T23:45:43.916 INFO:tasks.workunit.client.0.smithi031.stdout:0/428: fdatasync d0/d13/d18/f48 0 2024-04-22T23:45:43.918 INFO:tasks.workunit.client.0.smithi031.stdout:1/460: rename d1/db/d16/f40 to d1/db/d16/da0/fa4 0 2024-04-22T23:45:43.921 INFO:tasks.workunit.client.0.smithi031.stdout:4/609: link db/d31/d40/d43/f51 db/d14/d2d/d49/d59/d83/d9c/d72/dc6/dd2/fd9 0 2024-04-22T23:45:43.926 INFO:tasks.workunit.client.0.smithi031.stdout:0/429: mknod d0/d13/d18/d1e/d25/d8e/c9b 0 2024-04-22T23:45:43.926 INFO:tasks.workunit.client.0.smithi031.stdout:0/430: dread - d0/d13/f81 zero size 2024-04-22T23:45:43.927 INFO:tasks.workunit.client.0.smithi031.stdout:1/461: creat d1/d21/d29/d28/fa5 x:0 0 0 2024-04-22T23:45:43.928 INFO:tasks.workunit.client.0.smithi031.stdout:1/462: fsync d1/de/f11 0 2024-04-22T23:45:43.932 INFO:tasks.workunit.client.0.smithi031.stdout:5/444: link db/d16/d2a/f7f db/d16/d2a/d36/d74/d82/d86/f91 0 2024-04-22T23:45:43.933 INFO:tasks.workunit.client.0.smithi031.stdout:9/492: getdents d2/dd/d75/d64/d5d/d59 0 2024-04-22T23:45:43.934 INFO:tasks.workunit.client.0.smithi031.stdout:0/431: creat d0/d1/d2/d36/f9c x:0 0 0 2024-04-22T23:45:43.935 INFO:tasks.workunit.client.0.smithi031.stdout:0/432: dread - d0/d1/d2/d36/d11/f92 zero size 2024-04-22T23:45:43.936 INFO:tasks.workunit.client.0.smithi031.stdout:1/463: rmdir d1/d3f 39 2024-04-22T23:45:43.937 INFO:tasks.workunit.client.0.smithi031.stdout:1/464: write d1/de/d1e/d34/d41/f44 [1908092,69943] 0 2024-04-22T23:45:43.938 INFO:tasks.workunit.client.0.smithi031.stdout:5/445: creat db/d16/d1d/d24/d58/d5d/f92 x:0 0 0 2024-04-22T23:45:43.941 INFO:tasks.workunit.client.0.smithi031.stdout:0/433: symlink d0/d13/d18/d1e/d25/d4c/l9d 0 2024-04-22T23:45:43.941 INFO:tasks.workunit.client.0.smithi031.stdout:9/493: read d2/dd/d75/d64/f55 [3027668,39663] 0 2024-04-22T23:45:43.941 INFO:tasks.workunit.client.0.smithi031.stdout:0/434: stat d0/d13/d18/d1e/d25/c53 0 2024-04-22T23:45:43.945 INFO:tasks.workunit.client.0.smithi031.stdout:5/446: fdatasync f5 0 2024-04-22T23:45:43.947 INFO:tasks.workunit.client.0.smithi031.stdout:9/494: dwrite d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:43.950 INFO:tasks.workunit.client.0.smithi031.stdout:1/465: mkdir d1/d21/d4b/d6f/da6 0 2024-04-22T23:45:43.954 INFO:tasks.workunit.client.0.smithi031.stdout:9/495: creat d2/d15/d5a/f9f x:0 0 0 2024-04-22T23:45:43.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/435: dread d0/d1/d2/d36/d11/d1c/f88 [0,4194304] 0 2024-04-22T23:45:43.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/466: mknod d1/d35/ca7 0 2024-04-22T23:45:43.959 INFO:tasks.workunit.client.0.smithi031.stdout:9/496: chown d2/dc/f10 301 1 2024-04-22T23:45:43.961 INFO:tasks.workunit.client.0.smithi031.stdout:0/436: creat d0/d1/d2/d36/d11/f9e x:0 0 0 2024-04-22T23:45:43.962 INFO:tasks.workunit.client.0.smithi031.stdout:1/467: creat d1/d21/d29/d28/fa8 x:0 0 0 2024-04-22T23:45:43.968 INFO:tasks.workunit.client.0.smithi031.stdout:1/468: dwrite d1/de/d1e/f95 [0,4194304] 0 2024-04-22T23:45:43.970 INFO:tasks.workunit.client.0.smithi031.stdout:0/437: rename d0/d1/d2/d36/d11/d1c/c87 to d0/d13/d18/d1e/d25/d8e/c9f 0 2024-04-22T23:45:43.970 INFO:tasks.workunit.client.0.smithi031.stdout:0/438: chown d0/d1/f6b 188726 1 2024-04-22T23:45:43.972 INFO:tasks.workunit.client.0.smithi031.stdout:1/469: truncate d1/d21/d29/f1a 1438907 0 2024-04-22T23:45:43.982 INFO:tasks.workunit.client.0.smithi031.stdout:0/439: getdents d0/d1/d2 0 2024-04-22T23:45:43.984 INFO:tasks.workunit.client.0.smithi031.stdout:1/470: rename d1/d3f/d54/f8f to d1/db/d16/fa9 0 2024-04-22T23:45:43.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/471: write d1/db/d16/d1f/d52/f5c [1235516,106228] 0 2024-04-22T23:45:43.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/472: creat d1/db/d16/d1f/faa x:0 0 0 2024-04-22T23:45:43.999 INFO:tasks.workunit.client.0.smithi031.stdout:0/440: dread d0/d1/d2/d36/fd [0,4194304] 0 2024-04-22T23:45:44.002 INFO:tasks.workunit.client.0.smithi031.stdout:4/610: write db/d14/d2d/d49/d59/d6a/d63/fa5 [863502,95294] 0 2024-04-22T23:45:44.005 INFO:tasks.workunit.client.0.smithi031.stdout:0/441: dwrite d0/d1/d2/d36/d11/f9e [0,4194304] 0 2024-04-22T23:45:44.014 INFO:tasks.workunit.client.0.smithi031.stdout:4/611: dwrite db/d14/d2d/db7/fb8 [0,4194304] 0 2024-04-22T23:45:44.018 INFO:tasks.workunit.client.0.smithi031.stdout:9/497: write d2/f3c [1672816,14882] 0 2024-04-22T23:45:44.020 INFO:tasks.workunit.client.0.smithi031.stdout:5/447: dwrite db/d16/d1d/f20 [0,4194304] 0 2024-04-22T23:45:44.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/612: dwrite db/d14/d2d/d49/d59/d83/d9c/fa0 [4194304,4194304] 0 2024-04-22T23:45:44.029 INFO:tasks.workunit.client.0.smithi031.stdout:9/498: dwrite d2/dc/d8b/d3d/f51 [0,4194304] 0 2024-04-22T23:45:44.037 INFO:tasks.workunit.client.0.smithi031.stdout:6/533: sync 2024-04-22T23:45:44.037 INFO:tasks.workunit.client.0.smithi031.stdout:5/448: link db/fd db/d16/d1d/f93 0 2024-04-22T23:45:44.039 INFO:tasks.workunit.client.0.smithi031.stdout:9/499: mkdir d2/dd/d75/d4a/da0 0 2024-04-22T23:45:44.044 INFO:tasks.workunit.client.0.smithi031.stdout:1/473: dwrite d1/db/f19 [0,4194304] 0 2024-04-22T23:45:44.044 INFO:tasks.workunit.client.0.smithi031.stdout:9/500: rmdir d2/dd/d65 39 2024-04-22T23:45:44.047 INFO:tasks.workunit.client.0.smithi031.stdout:1/474: chown d1/d35/l48 1558702 1 2024-04-22T23:45:44.049 INFO:tasks.workunit.client.0.smithi031.stdout:6/534: chown d3/da5/cb3 1 1 2024-04-22T23:45:44.050 INFO:tasks.workunit.client.0.smithi031.stdout:5/449: dwrite db/d16/d1d/d24/d58/d5d/f92 [0,4194304] 0 2024-04-22T23:45:44.052 INFO:tasks.workunit.client.0.smithi031.stdout:9/501: mkdir d2/dc/d3e/da1 0 2024-04-22T23:45:44.053 INFO:tasks.workunit.client.0.smithi031.stdout:1/475: mknod d1/d21/cab 0 2024-04-22T23:45:44.054 INFO:tasks.workunit.client.0.smithi031.stdout:0/442: dwrite d0/d1/d2/d36/d11/d1a/d3c/d99/d2b/f64 [0,4194304] 0 2024-04-22T23:45:44.058 INFO:tasks.workunit.client.0.smithi031.stdout:9/502: read d2/dd/d75/d4a/f61 [3623229,19821] 0 2024-04-22T23:45:44.067 INFO:tasks.workunit.client.0.smithi031.stdout:0/443: dwrite d0/d13/d18/d1e/d25/f38 [0,4194304] 0 2024-04-22T23:45:44.068 INFO:tasks.workunit.client.0.smithi031.stdout:5/450: link db/d16/d1d/d24/d27/l3b db/d16/d1d/d24/d58/l94 0 2024-04-22T23:45:44.070 INFO:tasks.workunit.client.0.smithi031.stdout:5/451: read db/d16/d1d/d24/d27/f2d [4044578,22205] 0 2024-04-22T23:45:44.076 INFO:tasks.workunit.client.0.smithi031.stdout:6/535: dread d3/f45 [0,4194304] 0 2024-04-22T23:45:44.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/444: rmdir d0/d1/d3f 39 2024-04-22T23:45:44.078 INFO:tasks.workunit.client.0.smithi031.stdout:1/476: getdents d1/de/d1e/d34/d66 0 2024-04-22T23:45:44.079 INFO:tasks.workunit.client.0.smithi031.stdout:1/477: fsync d1/d21/d29/f1b 0 2024-04-22T23:45:44.080 INFO:tasks.workunit.client.0.smithi031.stdout:1/478: chown d1/de/d1e/d34/d41/f58 21945 1 2024-04-22T23:45:44.082 INFO:tasks.workunit.client.0.smithi031.stdout:1/479: write d1/de/d1e/d34/d41/f42 [1528605,100212] 0 2024-04-22T23:45:44.083 INFO:tasks.workunit.client.0.smithi031.stdout:1/480: chown d1/d21/d4b/d6f/da6 6711757 1 2024-04-22T23:45:44.084 INFO:tasks.workunit.client.0.smithi031.stdout:4/613: dread db/d14/fad [0,4194304] 0 2024-04-22T23:45:44.085 INFO:tasks.workunit.client.0.smithi031.stdout:6/536: creat d3/d35/fbd x:0 0 0 2024-04-22T23:45:44.086 INFO:tasks.workunit.client.0.smithi031.stdout:6/537: chown d3/d1f/d60 536507 1 2024-04-22T23:45:44.091 INFO:tasks.workunit.client.0.smithi031.stdout:0/445: dwrite d0/d1/d3f/f8b [0,4194304] 0 2024-04-22T23:45:44.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/614: mknod db/d14/d1d/d7b/d7f/d2a/da1/da7/cda 0 2024-04-22T23:45:44.101 INFO:tasks.workunit.client.0.smithi031.stdout:0/446: rename d0/d1/d2/d36/l84 to d0/d1/d2/d36/d11/d1c/la0 0 2024-04-22T23:45:44.102 INFO:tasks.workunit.client.0.smithi031.stdout:9/503: truncate d2/dc/d8b/d3d/d40/f44 3160688 0 2024-04-22T23:45:44.102 INFO:tasks.workunit.client.0.smithi031.stdout:0/447: readlink d0/d1/d2/d36/d11/d1a/d56/l71 0 2024-04-22T23:45:44.102 INFO:tasks.workunit.client.0.smithi031.stdout:1/481: link d1/d21/d29/d28/l78 d1/db/d16/d2f/d81/lac 0 2024-04-22T23:45:44.105 INFO:tasks.workunit.client.0.smithi031.stdout:4/615: mknod db/d14/d2d/d49/d59/d6a/d63/cdb 0 2024-04-22T23:45:44.106 INFO:tasks.workunit.client.0.smithi031.stdout:4/616: chown db/d14/d1d/d7b/d7f/d2a 5604 1 2024-04-22T23:45:44.109 INFO:tasks.workunit.client.0.smithi031.stdout:0/448: creat d0/d1/d3f/fa1 x:0 0 0 2024-04-22T23:45:44.110 INFO:tasks.workunit.client.0.smithi031.stdout:1/482: symlink d1/d3f/d54/lad 0 2024-04-22T23:45:44.112 INFO:tasks.workunit.client.0.smithi031.stdout:6/538: link d3/d2a/d3c/d40/d4d/lac d3/d2a/d3c/db6/lbe 0 2024-04-22T23:45:44.114 INFO:tasks.workunit.client.0.smithi031.stdout:9/504: dread d2/dd/d75/d4a/f49 [0,4194304] 0 2024-04-22T23:45:44.116 INFO:tasks.workunit.client.0.smithi031.stdout:0/449: creat d0/d1/d2/d36/d11/d1a/d56/fa2 x:0 0 0 2024-04-22T23:45:44.118 INFO:tasks.workunit.client.0.smithi031.stdout:1/483: creat d1/db/d16/d2f/fae x:0 0 0 2024-04-22T23:45:44.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/425: sync 2024-04-22T23:45:44.122 INFO:tasks.workunit.client.0.smithi031.stdout:3/410: sync 2024-04-22T23:45:44.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/499: sync 2024-04-22T23:45:44.122 INFO:tasks.workunit.client.0.smithi031.stdout:7/464: sync 2024-04-22T23:45:44.123 INFO:tasks.workunit.client.0.smithi031.stdout:0/450: creat d0/d1/d2/d36/d11/d1a/d3c/d99/d2b/fa3 x:0 0 0 2024-04-22T23:45:44.124 INFO:tasks.workunit.client.0.smithi031.stdout:1/484: rmdir d1/d99 39 2024-04-22T23:45:44.126 INFO:tasks.workunit.client.0.smithi031.stdout:6/539: creat d3/d2a/d85/d96/fbf x:0 0 0 2024-04-22T23:45:44.131 INFO:tasks.workunit.client.0.smithi031.stdout:8/426: write d5/da/df/d33/f65 [5515606,67775] 0 2024-04-22T23:45:44.134 INFO:tasks.workunit.client.0.smithi031.stdout:7/465: creat d9/d15/d3a/d41/fa3 x:0 0 0 2024-04-22T23:45:44.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/466: chown d9/d15/d6d/d65/f90 145 1 2024-04-22T23:45:44.135 INFO:tasks.workunit.client.0.smithi031.stdout:3/411: creat d4/d6/d11/d25/d44/f8e x:0 0 0 2024-04-22T23:45:44.136 INFO:tasks.workunit.client.0.smithi031.stdout:0/451: creat d0/d1/d2/d36/d11/d1a/d56/fa4 x:0 0 0 2024-04-22T23:45:44.137 INFO:tasks.workunit.client.0.smithi031.stdout:7/467: write d9/d14/d1f/d47/f69 [777793,29376] 0 2024-04-22T23:45:44.139 INFO:tasks.workunit.client.0.smithi031.stdout:1/485: creat d1/d35/d36/d9f/faf x:0 0 0 2024-04-22T23:45:44.139 INFO:tasks.workunit.client.0.smithi031.stdout:1/486: chown d1/db/d16/d2f/f75 119 1 2024-04-22T23:45:44.140 INFO:tasks.workunit.client.0.smithi031.stdout:9/505: rename d2/dd/d75/d4a/f34 to d2/dd/fa2 0 2024-04-22T23:45:44.144 INFO:tasks.workunit.client.0.smithi031.stdout:8/427: mkdir d5/da/d12/d2f/d7c 0 2024-04-22T23:45:44.145 INFO:tasks.workunit.client.0.smithi031.stdout:7/468: creat d9/d15/d3a/d41/fa4 x:0 0 0 2024-04-22T23:45:44.145 INFO:tasks.workunit.client.0.smithi031.stdout:4/617: truncate db/d14/d2d/d49/d59/d6a/d41/f4b 1423066 0 2024-04-22T23:45:44.146 INFO:tasks.workunit.client.0.smithi031.stdout:4/618: stat db/d14/l1c 0 2024-04-22T23:45:44.149 INFO:tasks.workunit.client.0.smithi031.stdout:1/487: mkdir d1/db/d16/d1f/d52/db0 0 2024-04-22T23:45:44.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/488: readlink d1/d3f/d54/lad 0 2024-04-22T23:45:44.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/489: truncate d1/f6 4568803 0 2024-04-22T23:45:44.151 INFO:tasks.workunit.client.0.smithi031.stdout:0/452: rename d0/d1/d2/c8 to d0/ca5 0 2024-04-22T23:45:44.154 INFO:tasks.workunit.client.0.smithi031.stdout:7/469: symlink d9/d14/d22/d28/d7a/la5 0 2024-04-22T23:45:44.156 INFO:tasks.workunit.client.0.smithi031.stdout:3/412: link d4/d6/d11/d25/d44/f8e d4/d6/d11/d25/d44/d7f/f8f 0 2024-04-22T23:45:44.158 INFO:tasks.workunit.client.0.smithi031.stdout:3/413: dread d4/d54/d2c/d89/d2e/d45/f60 [0,4194304] 0 2024-04-22T23:45:44.160 INFO:tasks.workunit.client.0.smithi031.stdout:9/506: rename d2/dd/d75/d4a/f49 to d2/d15/d5a/fa3 0 2024-04-22T23:45:44.161 INFO:tasks.workunit.client.0.smithi031.stdout:9/507: stat d2/dd/d75/d64/d5d/l98 0 2024-04-22T23:45:44.164 INFO:tasks.workunit.client.0.smithi031.stdout:0/453: fdatasync d0/d1/d2/d36/d11/d1a/d3c/d99/f55 0 2024-04-22T23:45:44.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/470: readlink d9/d15/d6d/d65/l67 0 2024-04-22T23:45:44.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/414: symlink d4/d54/d2c/d89/d2e/l90 0 2024-04-22T23:45:44.173 INFO:tasks.workunit.client.0.smithi031.stdout:2/500: getdents d6/dc/d48/d4b/d6b 0 2024-04-22T23:45:44.174 INFO:tasks.workunit.client.0.smithi031.stdout:3/415: truncate d4/d54/d2c/d89/d2e/f39 109993 0 2024-04-22T23:45:44.174 INFO:tasks.workunit.client.0.smithi031.stdout:2/501: truncate d6/d45/f73 5076924 0 2024-04-22T23:45:44.175 INFO:tasks.workunit.client.0.smithi031.stdout:3/416: write d4/d54/d56/f87 [347604,45458] 0 2024-04-22T23:45:44.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/428: creat d5/da/d12/d2e/f7d x:0 0 0 2024-04-22T23:45:44.178 INFO:tasks.workunit.client.0.smithi031.stdout:4/619: write db/d14/d1d/d7b/d7f/d2a/d47/fa3 [4095822,25686] 0 2024-04-22T23:45:44.179 INFO:tasks.workunit.client.0.smithi031.stdout:0/454: symlink d0/d13/d18/d6c/la6 0 2024-04-22T23:45:44.180 INFO:tasks.workunit.client.0.smithi031.stdout:7/471: rmdir d9/d14/d22/d28 39 2024-04-22T23:45:44.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/472: fsync d9/d14/f75 0 2024-04-22T23:45:44.184 INFO:tasks.workunit.client.0.smithi031.stdout:1/490: link d1/db/d16/d2f/c4f d1/db/d16/d2f/cb1 0 2024-04-22T23:45:44.193 INFO:tasks.workunit.client.0.smithi031.stdout:7/473: mkdir d9/d14/d1f/d45/da6 0 2024-04-22T23:45:44.197 INFO:tasks.workunit.client.0.smithi031.stdout:1/491: dread d1/db/d16/d2f/f50 [0,4194304] 0 2024-04-22T23:45:44.197 INFO:tasks.workunit.client.0.smithi031.stdout:9/508: dwrite d2/dc/f43 [0,4194304] 0 2024-04-22T23:45:44.199 INFO:tasks.workunit.client.0.smithi031.stdout:9/509: fsync d2/dd/d75/d64/d5d/d59/f78 0 2024-04-22T23:45:44.199 INFO:tasks.workunit.client.0.smithi031.stdout:3/417: write d4/d6/da/f29 [992207,85924] 0 2024-04-22T23:45:44.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/474: creat d9/d14/d22/d5c/fa7 x:0 0 0 2024-04-22T23:45:44.203 INFO:tasks.workunit.client.0.smithi031.stdout:7/475: write d9/d14/d1f/d45/f76 [483294,13414] 0 2024-04-22T23:45:44.204 INFO:tasks.workunit.client.0.smithi031.stdout:8/429: rename d5/da/d12/d2f to d5/da/d12/d2e/d7e 0 2024-04-22T23:45:44.205 INFO:tasks.workunit.client.0.smithi031.stdout:1/492: dwrite d1/d21/d29/d28/fa8 [0,4194304] 0 2024-04-22T23:45:44.208 INFO:tasks.workunit.client.0.smithi031.stdout:2/502: getdents d6 0 2024-04-22T23:45:44.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:43 smithi044 ceph-mon[34463]: pgmap v28: 65 pgs: 65 active+clean; 662 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s rd, 79 MiB/s wr, 257 op/s 2024-04-22T23:45:44.210 INFO:tasks.workunit.client.0.smithi031.stdout:4/620: getdents db/d14/d2d/d49/d59 0 2024-04-22T23:45:44.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/621: write db/d14/d1d/d7b/d7f/d2a/da1/fbc [417727,45166] 0 2024-04-22T23:45:44.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/622: fdatasync db/d14/d1d/d7b/d7f/fae 0 2024-04-22T23:45:44.217 INFO:tasks.workunit.client.0.smithi031.stdout:0/455: rename d0/d1/d2/d36/d11/d1a/d3c/d99 to d0/d13/d18/d1e/d25/d8e/da7 0 2024-04-22T23:45:44.217 INFO:tasks.workunit.client.0.smithi031.stdout:0/456: chown d0/d1/d2 21 1 2024-04-22T23:45:44.221 INFO:tasks.workunit.client.0.smithi031.stdout:1/493: symlink d1/db/d16/d2f/lb2 0 2024-04-22T23:45:44.233 INFO:tasks.workunit.client.0.smithi031.stdout:7/476: creat d9/d14/d22/d28/d7a/fa8 x:0 0 0 2024-04-22T23:45:44.234 INFO:tasks.workunit.client.0.smithi031.stdout:8/430: dwrite d5/d48/d61/d6b/d6c/f70 [0,4194304] 0 2024-04-22T23:45:44.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/418: truncate d4/d6/da/f3d 558074 0 2024-04-22T23:45:44.237 INFO:tasks.workunit.client.0.smithi031.stdout:0/457: rename d0/d1/d2/d36/fd to d0/d13/d18/d1e/d25/d8e/fa8 0 2024-04-22T23:45:44.239 INFO:tasks.workunit.client.0.smithi031.stdout:2/503: rmdir d6/d45/d50 39 2024-04-22T23:45:44.240 INFO:tasks.workunit.client.0.smithi031.stdout:3/419: write d4/d54/d2c/d89/d2e/d45/f6c [640399,10133] 0 2024-04-22T23:45:44.242 INFO:tasks.workunit.client.0.smithi031.stdout:9/510: dwrite d2/dc/d8b/d3d/d40/f5c [0,4194304] 0 2024-04-22T23:45:44.243 INFO:tasks.workunit.client.0.smithi031.stdout:7/477: rmdir d9/d14/d22/d5c/d5f 39 2024-04-22T23:45:44.246 INFO:tasks.workunit.client.0.smithi031.stdout:8/431: dread d5/da/d12/d37/d49/f54 [0,4194304] 0 2024-04-22T23:45:44.251 INFO:tasks.workunit.client.0.smithi031.stdout:8/432: chown d5/f9 6119 1 2024-04-22T23:45:44.251 INFO:tasks.workunit.client.0.smithi031.stdout:0/458: dwrite d0/f3d [0,4194304] 0 2024-04-22T23:45:44.252 INFO:tasks.workunit.client.0.smithi031.stdout:9/511: dread d2/dc/f17 [0,4194304] 0 2024-04-22T23:45:44.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/494: mkdir d1/db3 0 2024-04-22T23:45:44.254 INFO:tasks.workunit.client.0.smithi031.stdout:0/459: dread - d0/d13/d18/d1e/d25/f82 zero size 2024-04-22T23:45:44.254 INFO:tasks.workunit.client.0.smithi031.stdout:8/433: truncate d5/da/df/f79 1005586 0 2024-04-22T23:45:44.256 INFO:tasks.workunit.client.0.smithi031.stdout:4/623: truncate db/d14/d2d/d49/d59/d6a/d63/fa5 3156928 0 2024-04-22T23:45:44.256 INFO:tasks.workunit.client.0.smithi031.stdout:3/420: creat d4/d54/d2c/d89/f91 x:0 0 0 2024-04-22T23:45:44.256 INFO:tasks.workunit.client.0.smithi031.stdout:1/495: truncate d1/db/d16/d2f/f75 828934 0 2024-04-22T23:45:44.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/624: stat db/d31/d40/f68 0 2024-04-22T23:45:44.259 INFO:tasks.workunit.client.0.smithi031.stdout:0/460: dread d0/d1/d2/d36/d11/f9e [0,4194304] 0 2024-04-22T23:45:44.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/504: mknod d6/dc/d48/d72/ca5 0 2024-04-22T23:45:44.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/478: dwrite f6 [0,4194304] 0 2024-04-22T23:45:44.266 INFO:tasks.workunit.client.0.smithi031.stdout:7/479: chown d9/d14/d22/d5c/f80 3180 1 2024-04-22T23:45:44.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/625: creat db/d14/d1d/fdc x:0 0 0 2024-04-22T23:45:44.280 INFO:tasks.workunit.client.0.smithi031.stdout:6/540: sync 2024-04-22T23:45:44.280 INFO:tasks.workunit.client.0.smithi031.stdout:5/452: sync 2024-04-22T23:45:44.281 INFO:tasks.workunit.client.0.smithi031.stdout:2/505: mkdir d6/dc/d49/da6 0 2024-04-22T23:45:44.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/541: chown d3/d2a/d3c/d65/l99 1 1 2024-04-22T23:45:44.282 INFO:tasks.workunit.client.0.smithi031.stdout:7/480: mkdir d9/d15/d6d/da9 0 2024-04-22T23:45:44.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/481: write d9/d15/d3a/d41/fa4 [361763,109527] 0 2024-04-22T23:45:44.287 INFO:tasks.workunit.client.0.smithi031.stdout:8/434: creat d5/da/df/f7f x:0 0 0 2024-04-22T23:45:44.289 INFO:tasks.workunit.client.0.smithi031.stdout:5/453: rename db/d16/d1d/f22 to db/d16/d2a/d36/d74/d82/d86/f95 0 2024-04-22T23:45:44.290 INFO:tasks.workunit.client.0.smithi031.stdout:9/512: truncate d2/d15/d5a/fa3 796058 0 2024-04-22T23:45:44.292 INFO:tasks.workunit.client.0.smithi031.stdout:3/421: creat d4/d54/d2c/d89/d2e/d45/f92 x:0 0 0 2024-04-22T23:45:44.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/542: dwrite d3/d1f/d60/f84 [0,4194304] 0 2024-04-22T23:45:44.293 INFO:tasks.workunit.client.0.smithi031.stdout:2/506: dwrite d6/dc/d48/d4b/d6b/f6f [0,4194304] 0 2024-04-22T23:45:44.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/496: truncate d1/de/d1e/d34/d41/f58 2674172 0 2024-04-22T23:45:44.299 INFO:tasks.workunit.client.0.smithi031.stdout:8/435: unlink d5/da/d12/d2e/d7e/l32 0 2024-04-22T23:45:44.301 INFO:tasks.workunit.client.0.smithi031.stdout:0/461: getdents d0/d1/d2/d17 0 2024-04-22T23:45:44.303 INFO:tasks.workunit.client.0.smithi031.stdout:5/454: mknod db/d16/d1d/d24/d32/c96 0 2024-04-22T23:45:44.304 INFO:tasks.workunit.client.0.smithi031.stdout:7/482: mkdir d9/d14/d1f/d45/da6/daa 0 2024-04-22T23:45:44.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/483: chown d9/d14/d22/d28/d7a/f7d 403007062 1 2024-04-22T23:45:44.312 INFO:tasks.workunit.client.0.smithi031.stdout:6/543: fsync d3/d2a/d3c/f5f 0 2024-04-22T23:45:44.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/436: unlink d5/da/d12/d2e/d4d/f62 0 2024-04-22T23:45:44.316 INFO:tasks.workunit.client.0.smithi031.stdout:0/462: symlink d0/d1/d2/d36/d11/d1c/d54/la9 0 2024-04-22T23:45:44.319 INFO:tasks.workunit.client.0.smithi031.stdout:7/484: symlink d9/d14/d1f/d47/lab 0 2024-04-22T23:45:44.320 INFO:tasks.workunit.client.0.smithi031.stdout:9/513: mkdir d2/dc/d8b/d3d/d40/da4 0 2024-04-22T23:45:44.322 INFO:tasks.workunit.client.0.smithi031.stdout:3/422: mkdir d4/d6/d4c/d8c/d93 0 2024-04-22T23:45:44.323 INFO:tasks.workunit.client.0.smithi031.stdout:2/507: mknod d6/d45/ca7 0 2024-04-22T23:45:44.324 INFO:tasks.workunit.client.0.smithi031.stdout:2/508: read - d6/dc/d42/d24/f8f zero size 2024-04-22T23:45:44.325 INFO:tasks.workunit.client.0.smithi031.stdout:6/544: write d3/f74 [2600591,110368] 0 2024-04-22T23:45:44.327 INFO:tasks.workunit.client.0.smithi031.stdout:8/437: mknod d5/da/d12/d2e/d30/c80 0 2024-04-22T23:45:44.328 INFO:tasks.workunit.client.0.smithi031.stdout:5/455: symlink db/d16/l97 0 2024-04-22T23:45:44.329 INFO:tasks.workunit.client.0.smithi031.stdout:5/456: chown db/d16/d1d/d24/d27/c31 241249174 1 2024-04-22T23:45:44.331 INFO:tasks.workunit.client.0.smithi031.stdout:5/457: dread - db/d16/d2a/d36/d74/f87 zero size 2024-04-22T23:45:44.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/509: dwrite d6/dc/d42/d24/f8f [0,4194304] 0 2024-04-22T23:45:44.332 INFO:tasks.workunit.client.0.smithi031.stdout:5/458: fsync db/d16/d1d/d24/d32/f73 0 2024-04-22T23:45:44.335 INFO:tasks.workunit.client.0.smithi031.stdout:9/514: creat d2/d15/d5a/fa5 x:0 0 0 2024-04-22T23:45:44.336 INFO:tasks.workunit.client.0.smithi031.stdout:3/423: read d4/d6/da/f29 [110362,69786] 0 2024-04-22T23:45:44.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/424: chown d4/d54/d2c/d89/d2e/f4f 469767 1 2024-04-22T23:45:44.338 INFO:tasks.workunit.client.0.smithi031.stdout:3/425: truncate d4/d6/f80 777051 0 2024-04-22T23:45:44.338 INFO:tasks.workunit.client.0.smithi031.stdout:3/426: stat d4/c3f 0 2024-04-22T23:45:44.339 INFO:tasks.workunit.client.0.smithi031.stdout:3/427: write d4/d83/f8b [105624,33556] 0 2024-04-22T23:45:44.341 INFO:tasks.workunit.client.0.smithi031.stdout:3/428: dread d4/d54/d2c/d89/d2e/f39 [0,4194304] 0 2024-04-22T23:45:44.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/626: getdents db/d14/d2d/d49/d59 0 2024-04-22T23:45:44.343 INFO:tasks.workunit.client.0.smithi031.stdout:4/627: readlink db/d14/d1d/d7b/d7f/l17 0 2024-04-22T23:45:44.344 INFO:tasks.workunit.client.0.smithi031.stdout:6/545: creat d3/d35/fc0 x:0 0 0 2024-04-22T23:45:44.347 INFO:tasks.workunit.client.0.smithi031.stdout:7/485: mkdir d9/d15/dac 0 2024-04-22T23:45:44.349 INFO:tasks.workunit.client.0.smithi031.stdout:7/486: chown d9/d14/d22/d5c/f9a 10 1 2024-04-22T23:45:44.349 INFO:tasks.workunit.client.0.smithi031.stdout:2/510: unlink d6/dc/d17/d69/l87 0 2024-04-22T23:45:44.350 INFO:tasks.workunit.client.0.smithi031.stdout:0/463: dwrite d0/f3d [4194304,4194304] 0 2024-04-22T23:45:44.351 INFO:tasks.workunit.client.0.smithi031.stdout:5/459: unlink db/d16/d2a/f30 0 2024-04-22T23:45:44.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/515: rename d2/dc/d8b/f63 to d2/dc/d6d/fa6 0 2024-04-22T23:45:44.353 INFO:tasks.workunit.client.0.smithi031.stdout:5/460: truncate db/d16/d2a/d59/d90/f7b 588825 0 2024-04-22T23:45:44.354 INFO:tasks.workunit.client.0.smithi031.stdout:9/516: write d2/dc/d3e/f4f [6195441,93212] 0 2024-04-22T23:45:44.355 INFO:tasks.workunit.client.0.smithi031.stdout:9/517: dread - d2/dd/d75/d64/d5d/d59/d8e/f91 zero size 2024-04-22T23:45:44.356 INFO:tasks.workunit.client.0.smithi031.stdout:4/628: creat db/d14/d1d/d7b/d7f/d2a/da1/fdd x:0 0 0 2024-04-22T23:45:44.365 INFO:tasks.workunit.client.0.smithi031.stdout:0/464: fsync d0/d1/d2/d36/d11/f43 0 2024-04-22T23:45:44.366 INFO:tasks.workunit.client.0.smithi031.stdout:0/465: write d0/d1/d2/d36/d11/d1a/d3c/f59 [191981,1662] 0 2024-04-22T23:45:44.367 INFO:tasks.workunit.client.0.smithi031.stdout:3/429: rename d4/f5 to d4/d49/f94 0 2024-04-22T23:45:44.373 INFO:tasks.workunit.client.0.smithi031.stdout:3/430: dwrite d4/d54/d2c/d89/d2e/d45/f6c [4194304,4194304] 0 2024-04-22T23:45:44.375 INFO:tasks.workunit.client.0.smithi031.stdout:3/431: chown d4/d54/d88 41263 1 2024-04-22T23:45:44.376 INFO:tasks.workunit.client.0.smithi031.stdout:4/629: mkdir db/d14/d2d/d49/d59/d83/d9c/d72/dde 0 2024-04-22T23:45:44.377 INFO:tasks.workunit.client.0.smithi031.stdout:3/432: write d4/d6/d11/f37 [2610963,108752] 0 2024-04-22T23:45:44.379 INFO:tasks.workunit.client.0.smithi031.stdout:9/518: dwrite d2/dc/f83 [0,4194304] 0 2024-04-22T23:45:44.379 INFO:tasks.workunit.client.0.smithi031.stdout:3/433: write d4/d6/d11/d25/d44/f50 [1794690,56685] 0 2024-04-22T23:45:44.381 INFO:tasks.workunit.client.0.smithi031.stdout:3/434: write d4/d6/da/f13 [1627309,82025] 0 2024-04-22T23:45:44.383 INFO:tasks.workunit.client.0.smithi031.stdout:7/487: creat d9/d14/d22/d5b/fad x:0 0 0 2024-04-22T23:45:44.389 INFO:tasks.workunit.client.0.smithi031.stdout:0/466: mknod d0/d1/d3f/caa 0 2024-04-22T23:45:44.390 INFO:tasks.workunit.client.0.smithi031.stdout:0/467: stat d0/d13/d18/f48 0 2024-04-22T23:45:44.390 INFO:tasks.workunit.client.0.smithi031.stdout:8/438: getdents d5/da/d12/d37/d4e 0 2024-04-22T23:45:44.391 INFO:tasks.workunit.client.0.smithi031.stdout:8/439: stat d5/d48 0 2024-04-22T23:45:44.392 INFO:tasks.workunit.client.0.smithi031.stdout:8/440: write d5/da/d12/f44 [634302,23615] 0 2024-04-22T23:45:44.395 INFO:tasks.workunit.client.0.smithi031.stdout:6/546: link d3/d2a/d3c/d65/c80 d3/d35/d86/cc1 0 2024-04-22T23:45:44.396 INFO:tasks.workunit.client.0.smithi031.stdout:9/519: readlink d2/dd/d75/d64/l79 0 2024-04-22T23:45:44.396 INFO:tasks.workunit.client.0.smithi031.stdout:8/441: read d5/da/f1e [248705,33708] 0 2024-04-22T23:45:44.400 INFO:tasks.workunit.client.0.smithi031.stdout:0/468: mknod d0/d1/d2/d17/cab 0 2024-04-22T23:45:44.401 INFO:tasks.workunit.client.0.smithi031.stdout:4/630: dwrite db/d14/fad [0,4194304] 0 2024-04-22T23:45:44.402 INFO:tasks.workunit.client.0.smithi031.stdout:6/547: stat d3/c12 0 2024-04-22T23:45:44.403 INFO:tasks.workunit.client.0.smithi031.stdout:6/548: chown d3/d1f/d4c 0 1 2024-04-22T23:45:44.403 INFO:tasks.workunit.client.0.smithi031.stdout:6/549: chown d3/d1f 2 1 2024-04-22T23:45:44.412 INFO:tasks.workunit.client.0.smithi031.stdout:7/488: fsync d9/f3b 0 2024-04-22T23:45:44.413 INFO:tasks.workunit.client.0.smithi031.stdout:2/511: write d6/dc/d42/d28/f37 [1548372,9920] 0 2024-04-22T23:45:44.418 INFO:tasks.workunit.client.0.smithi031.stdout:4/631: rmdir db/d14/d2d/db7 39 2024-04-22T23:45:44.421 INFO:tasks.workunit.client.0.smithi031.stdout:2/512: dwrite d6/dc/d57/fa2 [0,4194304] 0 2024-04-22T23:45:44.421 INFO:tasks.workunit.client.0.smithi031.stdout:0/469: mkdir d0/d13/d18/d1e/d25/d8e/dac 0 2024-04-22T23:45:44.421 INFO:tasks.workunit.client.0.smithi031.stdout:4/632: read - db/d14/d2d/d49/d59/d6a/d63/d86/fc5 zero size 2024-04-22T23:45:44.422 INFO:tasks.workunit.client.0.smithi031.stdout:2/513: chown d6/d45/c81 417743723 1 2024-04-22T23:45:44.423 INFO:tasks.workunit.client.0.smithi031.stdout:9/520: readlink d2/dc/d8b/l8a 0 2024-04-22T23:45:44.424 INFO:tasks.workunit.client.0.smithi031.stdout:4/633: chown db/d14/d2d/d49/dcd/fd3 3 1 2024-04-22T23:45:44.424 INFO:tasks.workunit.client.0.smithi031.stdout:5/461: dread db/d16/d1d/f6b [0,4194304] 0 2024-04-22T23:45:44.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/521: chown d2/dd/d75/d64/f55 6758 1 2024-04-22T23:45:44.427 INFO:tasks.workunit.client.0.smithi031.stdout:6/550: rename d3/d2a/d3c/d40/d4d/f69 to d3/d35/fc2 0 2024-04-22T23:45:44.428 INFO:tasks.workunit.client.0.smithi031.stdout:9/522: truncate d2/f3c 2213729 0 2024-04-22T23:45:44.430 INFO:tasks.workunit.client.0.smithi031.stdout:9/523: fdatasync d2/d15/d5a/fa5 0 2024-04-22T23:45:44.431 INFO:tasks.workunit.client.0.smithi031.stdout:0/470: dwrite d0/d1/d2/d36/d11/d1a/d3c/f59 [0,4194304] 0 2024-04-22T23:45:44.431 INFO:tasks.workunit.client.0.smithi031.stdout:1/497: sync 2024-04-22T23:45:44.432 INFO:tasks.workunit.client.0.smithi031.stdout:1/498: dread - d1/db/d16/fa9 zero size 2024-04-22T23:45:44.433 INFO:tasks.workunit.client.0.smithi031.stdout:3/435: truncate d4/d6/d11/f37 1273706 0 2024-04-22T23:45:44.435 INFO:tasks.workunit.client.0.smithi031.stdout:3/436: truncate d4/d83/f8b 230366 0 2024-04-22T23:45:44.436 INFO:tasks.workunit.client.0.smithi031.stdout:7/489: unlink d9/d14/f23 0 2024-04-22T23:45:44.442 INFO:tasks.workunit.client.0.smithi031.stdout:3/437: dwrite d4/d6/d11/d25/d44/d7f/f69 [0,4194304] 0 2024-04-22T23:45:44.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:43 smithi031 ceph-mon[26721]: pgmap v28: 65 pgs: 65 active+clean; 662 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s rd, 79 MiB/s wr, 257 op/s 2024-04-22T23:45:44.447 INFO:tasks.workunit.client.0.smithi031.stdout:2/514: dread d6/f16 [0,4194304] 0 2024-04-22T23:45:44.456 INFO:tasks.workunit.client.0.smithi031.stdout:4/634: mknod db/d14/d2d/d49/dcd/cdf 0 2024-04-22T23:45:44.457 INFO:tasks.workunit.client.0.smithi031.stdout:1/499: symlink d1/db/d16/d1f/d52/lb4 0 2024-04-22T23:45:44.457 INFO:tasks.workunit.client.0.smithi031.stdout:6/551: chown d3/d2a/d3c/db6/lbe 33096384 1 2024-04-22T23:45:44.457 INFO:tasks.workunit.client.0.smithi031.stdout:4/635: read - db/f5f zero size 2024-04-22T23:45:44.457 INFO:tasks.workunit.client.0.smithi031.stdout:0/471: write d0/d13/d18/d1e/d25/f85 [1047412,10718] 0 2024-04-22T23:45:44.458 INFO:tasks.workunit.client.0.smithi031.stdout:8/442: truncate d5/da/d12/d37/d34/f4c 3409997 0 2024-04-22T23:45:44.458 INFO:tasks.workunit.client.0.smithi031.stdout:2/515: chown d6/dc/d17/d69/d90/f91 514610 1 2024-04-22T23:45:44.460 INFO:tasks.workunit.client.0.smithi031.stdout:7/490: mkdir d9/d14/d1f/d45/da6/dae 0 2024-04-22T23:45:44.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/438: mknod d4/d6/d11/d25/d44/d7f/d5f/c95 0 2024-04-22T23:45:44.462 INFO:tasks.workunit.client.0.smithi031.stdout:5/462: chown db/d16/d1d/f6b 2559674 1 2024-04-22T23:45:44.463 INFO:tasks.workunit.client.0.smithi031.stdout:4/636: chown db/l29 10241197 1 2024-04-22T23:45:44.466 INFO:tasks.workunit.client.0.smithi031.stdout:0/472: mkdir d0/d13/d18/d6c/dad 0 2024-04-22T23:45:44.469 INFO:tasks.workunit.client.0.smithi031.stdout:8/443: creat d5/da/d12/d37/d4e/f81 x:0 0 0 2024-04-22T23:45:44.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/516: creat d6/dc/d17/d69/fa8 x:0 0 0 2024-04-22T23:45:44.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/517: fdatasync d6/d45/f77 0 2024-04-22T23:45:44.474 INFO:tasks.workunit.client.0.smithi031.stdout:3/439: dread d4/d54/d2c/d89/d2e/d45/f60 [0,4194304] 0 2024-04-22T23:45:44.481 INFO:tasks.workunit.client.0.smithi031.stdout:8/444: dread d5/da/f59 [0,4194304] 0 2024-04-22T23:45:44.482 INFO:tasks.workunit.client.0.smithi031.stdout:8/445: read d5/da/d12/d37/d34/f5e [3757312,6751] 0 2024-04-22T23:45:44.483 INFO:tasks.workunit.client.0.smithi031.stdout:6/552: link d3/d2a/d3c/f98 d3/d2a/d3c/db6/fc3 0 2024-04-22T23:45:44.484 INFO:tasks.workunit.client.0.smithi031.stdout:2/518: mkdir d6/dc/d49/d83/d85/da9 0 2024-04-22T23:45:44.484 INFO:tasks.workunit.client.0.smithi031.stdout:6/553: write d3/d1f/d60/f84 [4063325,96437] 0 2024-04-22T23:45:44.487 INFO:tasks.workunit.client.0.smithi031.stdout:7/491: rename d9/d15/f17 to d9/d15/d6d/d65/faf 0 2024-04-22T23:45:44.487 INFO:tasks.workunit.client.0.smithi031.stdout:2/519: dread d6/dc/f4e [0,4194304] 0 2024-04-22T23:45:44.491 INFO:tasks.workunit.client.0.smithi031.stdout:4/637: truncate db/d14/d1d/fd0 2379149 0 2024-04-22T23:45:44.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/492: dwrite d9/d15/d3a/d41/fa4 [0,4194304] 0 2024-04-22T23:45:44.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/446: symlink d5/da/d12/d37/d34/l82 0 2024-04-22T23:45:44.505 INFO:tasks.workunit.client.0.smithi031.stdout:5/463: truncate db/fd 3356476 0 2024-04-22T23:45:44.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/464: stat db/f23 0 2024-04-22T23:45:44.507 INFO:tasks.workunit.client.0.smithi031.stdout:6/554: mkdir d3/d35/d75/d7d/dc4 0 2024-04-22T23:45:44.511 INFO:tasks.workunit.client.0.smithi031.stdout:0/473: truncate d0/d1/d2/d36/d11/d1a/d3c/f59 1055964 0 2024-04-22T23:45:44.513 INFO:tasks.workunit.client.0.smithi031.stdout:1/500: getdents d1/d3f/d54 0 2024-04-22T23:45:44.514 INFO:tasks.workunit.client.0.smithi031.stdout:1/501: read - d1/db/d16/d1f/faa zero size 2024-04-22T23:45:44.517 INFO:tasks.workunit.client.0.smithi031.stdout:7/493: dread d9/fa [0,4194304] 0 2024-04-22T23:45:44.518 INFO:tasks.workunit.client.0.smithi031.stdout:0/474: dwrite d0/d1/d2/d36/d11/d1a/d3c/f4d [0,4194304] 0 2024-04-22T23:45:44.518 INFO:tasks.workunit.client.0.smithi031.stdout:7/494: write d9/f4b [835618,90750] 0 2024-04-22T23:45:44.520 INFO:tasks.workunit.client.0.smithi031.stdout:7/495: write d9/d15/d3a/d41/f8f [449006,119578] 0 2024-04-22T23:45:44.522 INFO:tasks.workunit.client.0.smithi031.stdout:7/496: write d9/d15/f99 [984643,5765] 0 2024-04-22T23:45:44.525 INFO:tasks.workunit.client.0.smithi031.stdout:7/497: write d9/d14/d22/d28/d7a/fa8 [758073,46473] 0 2024-04-22T23:45:44.528 INFO:tasks.workunit.client.0.smithi031.stdout:6/555: readlink d3/d1f/l1e 0 2024-04-22T23:45:44.529 INFO:tasks.workunit.client.0.smithi031.stdout:5/465: truncate db/d16/d1d/d24/f76 635800 0 2024-04-22T23:45:44.529 INFO:tasks.workunit.client.0.smithi031.stdout:0/475: dwrite d0/d13/d18/d1e/f6e [0,4194304] 0 2024-04-22T23:45:44.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/520: write d6/dc/d57/f4d [148914,68475] 0 2024-04-22T23:45:44.533 INFO:tasks.workunit.client.0.smithi031.stdout:2/521: write d6/f22 [8973093,71254] 0 2024-04-22T23:45:44.535 INFO:tasks.workunit.client.0.smithi031.stdout:2/522: truncate d6/d45/f77 81486 0 2024-04-22T23:45:44.536 INFO:tasks.workunit.client.0.smithi031.stdout:1/502: rename d1/f46 to d1/d21/fb5 0 2024-04-22T23:45:44.536 INFO:tasks.workunit.client.0.smithi031.stdout:2/523: truncate d6/dc/d57/f4d 4453213 0 2024-04-22T23:45:44.537 INFO:tasks.workunit.client.0.smithi031.stdout:2/524: write d6/dc/d42/d24/f39 [2764843,16952] 0 2024-04-22T23:45:44.540 INFO:tasks.workunit.client.0.smithi031.stdout:8/447: mkdir d5/da/d83 0 2024-04-22T23:45:44.548 INFO:tasks.workunit.client.0.smithi031.stdout:0/476: creat d0/d1/d2/d36/d11/d1a/d56/fae x:0 0 0 2024-04-22T23:45:44.549 INFO:tasks.workunit.client.0.smithi031.stdout:5/466: dread db/d16/d2a/d36/d74/d82/d86/f91 [0,4194304] 0 2024-04-22T23:45:44.592 INFO:tasks.workunit.client.0.smithi031.stdout:7/498: rename d9/d14/d22/d5b to d9/d14/d1f/d45/da6/daa/db0 0 2024-04-22T23:45:44.593 INFO:tasks.workunit.client.0.smithi031.stdout:8/448: creat d5/da/d12/d37/d34/d6f/d5b/f84 x:0 0 0 2024-04-22T23:45:44.593 INFO:tasks.workunit.client.0.smithi031.stdout:7/499: fdatasync d9/f94 0 2024-04-22T23:45:44.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/638: getdents db/d14/d1d/d7b/d7f/d2a/da1/da7 0 2024-04-22T23:45:44.596 INFO:tasks.workunit.client.0.smithi031.stdout:5/467: stat db/d16/c5c 0 2024-04-22T23:45:44.597 INFO:tasks.workunit.client.0.smithi031.stdout:7/500: creat d9/d14/d1f/d47/fb1 x:0 0 0 2024-04-22T23:45:44.599 INFO:tasks.workunit.client.0.smithi031.stdout:0/477: dwrite d0/d1/d2/d36/d11/d1a/d3c/f3e [0,4194304] 0 2024-04-22T23:45:44.600 INFO:tasks.workunit.client.0.smithi031.stdout:8/449: dread d5/da/f59 [0,4194304] 0 2024-04-22T23:45:44.601 INFO:tasks.workunit.client.0.smithi031.stdout:6/556: getdents d3/da5 0 2024-04-22T23:45:44.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/478: truncate d0/d1/d2/d36/f9c 777456 0 2024-04-22T23:45:44.603 INFO:tasks.workunit.client.0.smithi031.stdout:7/501: truncate d9/d14/d1f/d45/da6/daa/db0/fad 769457 0 2024-04-22T23:45:44.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/468: creat db/d16/d2a/d36/f98 x:0 0 0 2024-04-22T23:45:44.606 INFO:tasks.workunit.client.0.smithi031.stdout:8/450: dread d5/f9 [4194304,4194304] 0 2024-04-22T23:45:44.609 INFO:tasks.workunit.client.0.smithi031.stdout:6/557: fdatasync d3/d35/d86/f91 0 2024-04-22T23:45:44.610 INFO:tasks.workunit.client.0.smithi031.stdout:7/502: dwrite d9/d14/d22/d28/d7a/fa8 [0,4194304] 0 2024-04-22T23:45:44.613 INFO:tasks.workunit.client.0.smithi031.stdout:0/479: unlink d0/d1/d2/d36/d11/d1a/d3c/f4e 0 2024-04-22T23:45:44.619 INFO:tasks.workunit.client.0.smithi031.stdout:8/451: unlink d5/da/d12/c28 0 2024-04-22T23:45:44.620 INFO:tasks.workunit.client.0.smithi031.stdout:8/452: write d5/d48/d61/d6b/d6c/f73 [512767,61086] 0 2024-04-22T23:45:44.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/558: creat d3/d2a/d3c/d40/d5b/fc5 x:0 0 0 2024-04-22T23:45:44.625 INFO:tasks.workunit.client.0.smithi031.stdout:7/503: mknod d9/d14/d22/d28/cb2 0 2024-04-22T23:45:44.625 INFO:tasks.workunit.client.0.smithi031.stdout:7/504: chown d9/d15 0 1 2024-04-22T23:45:44.629 INFO:tasks.workunit.client.0.smithi031.stdout:2/525: write d6/dc/d42/d28/f41 [45209,10817] 0 2024-04-22T23:45:44.631 INFO:tasks.workunit.client.0.smithi031.stdout:7/505: unlink d9/d15/l43 0 2024-04-22T23:45:44.632 INFO:tasks.workunit.client.0.smithi031.stdout:7/506: dread - d9/d15/d3a/d41/f83 zero size 2024-04-22T23:45:44.633 INFO:tasks.workunit.client.0.smithi031.stdout:7/507: truncate d9/d14/d22/d28/d7a/f95 272213 0 2024-04-22T23:45:44.633 INFO:tasks.workunit.client.0.smithi031.stdout:7/508: stat d9/c55 0 2024-04-22T23:45:44.637 INFO:tasks.workunit.client.0.smithi031.stdout:4/639: write db/d14/d1d/fd0 [2929348,21345] 0 2024-04-22T23:45:44.639 INFO:tasks.workunit.client.0.smithi031.stdout:7/509: dwrite d9/f3e [0,4194304] 0 2024-04-22T23:45:44.639 INFO:tasks.workunit.client.0.smithi031.stdout:2/526: symlink d6/dc/d49/d83/d85/laa 0 2024-04-22T23:45:44.643 INFO:tasks.workunit.client.0.smithi031.stdout:5/469: getdents db/d16/d2a/d36 0 2024-04-22T23:45:44.644 INFO:tasks.workunit.client.0.smithi031.stdout:6/559: rmdir d3/d35/d86/dae/dbb 0 2024-04-22T23:45:44.645 INFO:tasks.workunit.client.0.smithi031.stdout:4/640: creat db/d14/d2d/d49/d59/d6a/d63/fe0 x:0 0 0 2024-04-22T23:45:44.648 INFO:tasks.workunit.client.0.smithi031.stdout:7/510: dwrite d9/d14/d22/f78 [0,4194304] 0 2024-04-22T23:45:44.649 INFO:tasks.workunit.client.0.smithi031.stdout:7/511: readlink d9/d14/d1f/l4d 0 2024-04-22T23:45:44.652 INFO:tasks.workunit.client.0.smithi031.stdout:2/527: mkdir d6/dc/d42/d28/d94/dab 0 2024-04-22T23:45:44.655 INFO:tasks.workunit.client.0.smithi031.stdout:4/641: mknod db/d14/d2d/d49/d59/d6a/d63/ce1 0 2024-04-22T23:45:44.655 INFO:tasks.workunit.client.0.smithi031.stdout:0/480: write d0/d13/f5b [1393737,115016] 0 2024-04-22T23:45:44.657 INFO:tasks.workunit.client.0.smithi031.stdout:7/512: dread d9/d15/d3a/d41/f8f [0,4194304] 0 2024-04-22T23:45:44.659 INFO:tasks.workunit.client.0.smithi031.stdout:5/470: dwrite db/d16/d2a/d59/d90/f7b [0,4194304] 0 2024-04-22T23:45:44.659 INFO:tasks.workunit.client.0.smithi031.stdout:0/481: chown d0/d13/d18/d1e/d25/d8e/da7/d2b/fa3 234741284 1 2024-04-22T23:45:44.660 INFO:tasks.workunit.client.0.smithi031.stdout:2/528: mknod d6/dc/d48/d8a/cac 0 2024-04-22T23:45:44.666 INFO:tasks.workunit.client.0.smithi031.stdout:5/471: creat db/d16/d1d/d24/d27/f99 x:0 0 0 2024-04-22T23:45:44.670 INFO:tasks.workunit.client.0.smithi031.stdout:7/513: truncate d9/f1c 1138856 0 2024-04-22T23:45:44.670 INFO:tasks.workunit.client.0.smithi031.stdout:6/560: dwrite d3/d2a/d3c/f5f [0,4194304] 0 2024-04-22T23:45:44.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/529: creat d6/dc/d49/d83/d85/da9/fad x:0 0 0 2024-04-22T23:45:44.675 INFO:tasks.workunit.client.0.smithi031.stdout:4/642: rename db/d14/c18 to db/d31/ce2 0 2024-04-22T23:45:44.677 INFO:tasks.workunit.client.0.smithi031.stdout:7/514: symlink d9/d14/d1f/d45/da6/daa/lb3 0 2024-04-22T23:45:44.677 INFO:tasks.workunit.client.0.smithi031.stdout:4/643: chown db/d14/d1d/d7b/d7f/d2a/d47/lbd 977534208 1 2024-04-22T23:45:44.681 INFO:tasks.workunit.client.0.smithi031.stdout:5/472: rename db/d16/d2a/d59/f5f to db/d16/d2a/d36/d74/d82/d86/f9a 0 2024-04-22T23:45:44.682 INFO:tasks.workunit.client.0.smithi031.stdout:7/515: symlink d9/d14/d1f/d45/da6/daa/lb4 0 2024-04-22T23:45:44.685 INFO:tasks.workunit.client.0.smithi031.stdout:4/644: mknod db/d14/d2d/d49/d59/d6a/d63/d86/ce3 0 2024-04-22T23:45:44.689 INFO:tasks.workunit.client.0.smithi031.stdout:6/561: rename d3/d1f/f28 to d3/d35/fc6 0 2024-04-22T23:45:44.694 INFO:tasks.workunit.client.0.smithi031.stdout:6/562: dwrite d3/d2a/d3c/d40/d7a/f9a [0,4194304] 0 2024-04-22T23:45:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/473: mkdir db/d8d/d9b 0 2024-04-22T23:45:44.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/474: chown db/d16/d2a/d7c/d89 214932 1 2024-04-22T23:45:44.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/475: dread db/d16/d1d/d24/d58/f85 [0,4194304] 0 2024-04-22T23:45:44.704 INFO:tasks.workunit.client.0.smithi031.stdout:2/530: getdents d6/d45 0 2024-04-22T23:45:44.706 INFO:tasks.workunit.client.0.smithi031.stdout:6/563: symlink d3/d35/d86/lc7 0 2024-04-22T23:45:44.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/564: mknod d3/d1f/d60/cc8 0 2024-04-22T23:45:44.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/565: dread - d3/d2a/d85/f9d zero size 2024-04-22T23:45:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:4/645: link db/d14/d2d/d49/d59/d6a/d63/d86/f8c db/d14/d1d/d7b/fe4 0 2024-04-22T23:45:44.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/516: link d9/d14/d22/c30 d9/d15/d6d/cb5 0 2024-04-22T23:45:44.716 INFO:tasks.workunit.client.0.smithi031.stdout:5/476: mknod db/c9c 0 2024-04-22T23:45:44.717 INFO:tasks.workunit.client.0.smithi031.stdout:2/531: symlink d6/dc/d9a/d9e/da4/lae 0 2024-04-22T23:45:44.718 INFO:tasks.workunit.client.0.smithi031.stdout:5/477: write db/d16/f8b [531069,77317] 0 2024-04-22T23:45:44.718 INFO:tasks.workunit.client.0.smithi031.stdout:4/646: dread db/d14/d1d/d7b/d7f/d2a/d30/f5e [0,4194304] 0 2024-04-22T23:45:44.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/532: dread - d6/dc/d49/d83/d85/da9/fad zero size 2024-04-22T23:45:44.720 INFO:tasks.workunit.client.0.smithi031.stdout:7/517: creat d9/d14/d1f/d45/da6/daa/db0/fb6 x:0 0 0 2024-04-22T23:45:44.721 INFO:tasks.workunit.client.0.smithi031.stdout:2/533: write f3 [7624233,103314] 0 2024-04-22T23:45:44.727 INFO:tasks.workunit.client.0.smithi031.stdout:5/478: creat db/d16/d2a/d36/d74/d82/d86/f9d x:0 0 0 2024-04-22T23:45:44.727 INFO:tasks.workunit.client.0.smithi031.stdout:2/534: symlink d6/dc/d42/d24/d32/laf 0 2024-04-22T23:45:44.728 INFO:tasks.workunit.client.0.smithi031.stdout:6/566: getdents d3/d35/d75 0 2024-04-22T23:45:44.728 INFO:tasks.workunit.client.0.smithi031.stdout:2/535: read - d6/dc/d17/d69/fa8 zero size 2024-04-22T23:45:44.729 INFO:tasks.workunit.client.0.smithi031.stdout:7/518: rename d9/d15/d3a/d41/fa4 to d9/d15/d6d/d65/fb7 0 2024-04-22T23:45:44.732 INFO:tasks.workunit.client.0.smithi031.stdout:4/647: unlink db/d14/d2d/d49/d59/d6a/d41/f4b 0 2024-04-22T23:45:44.733 INFO:tasks.workunit.client.0.smithi031.stdout:5/479: mkdir db/d16/d1d/d24/d27/d9e 0 2024-04-22T23:45:44.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/536: dwrite d6/dc/d42/f5c [4194304,4194304] 0 2024-04-22T23:45:44.740 INFO:tasks.workunit.client.0.smithi031.stdout:4/648: write db/d14/d2d/d49/d59/d6a/d63/d86/fc5 [327107,126356] 0 2024-04-22T23:45:44.743 INFO:tasks.workunit.client.0.smithi031.stdout:7/519: dwrite d9/d15/d6d/d65/f6b [4194304,4194304] 0 2024-04-22T23:45:44.744 INFO:tasks.workunit.client.0.smithi031.stdout:5/480: creat db/d16/d2a/d36/d74/d82/f9f x:0 0 0 2024-04-22T23:45:44.745 INFO:tasks.workunit.client.0.smithi031.stdout:6/567: link d3/d35/c3a d3/d2a/d3c/d40/d4d/cc9 0 2024-04-22T23:45:44.750 INFO:tasks.workunit.client.0.smithi031.stdout:7/520: unlink d9/d14/d1f/d45/l84 0 2024-04-22T23:45:44.753 INFO:tasks.workunit.client.0.smithi031.stdout:6/568: unlink d3/d1f/d4c/f56 0 2024-04-22T23:45:44.756 INFO:tasks.workunit.client.0.smithi031.stdout:5/481: dwrite db/d16/d1d/d24/d58/d5d/f77 [0,4194304] 0 2024-04-22T23:45:44.757 INFO:tasks.workunit.client.0.smithi031.stdout:4/649: creat db/d14/d1d/d7b/d7f/d2a/d48/fe5 x:0 0 0 2024-04-22T23:45:44.758 INFO:tasks.workunit.client.0.smithi031.stdout:6/569: read d3/d35/d86/f91 [2487729,19728] 0 2024-04-22T23:45:44.759 INFO:tasks.workunit.client.0.smithi031.stdout:7/521: rmdir d9/d14/d1f/d45/da6/daa 39 2024-04-22T23:45:44.763 INFO:tasks.workunit.client.0.smithi031.stdout:2/537: rename d6/dc/d42/d28/f41 to d6/dc/d48/d4b/fb0 0 2024-04-22T23:45:44.769 INFO:tasks.workunit.client.0.smithi031.stdout:5/482: dread db/d16/d1d/f93 [0,4194304] 0 2024-04-22T23:45:44.770 INFO:tasks.workunit.client.0.smithi031.stdout:5/483: chown db/d16/d2a/d59 0 1 2024-04-22T23:45:44.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/538: unlink d6/dc/d17/c3e 0 2024-04-22T23:45:44.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/440: sync 2024-04-22T23:45:44.772 INFO:tasks.workunit.client.0.smithi031.stdout:1/503: sync 2024-04-22T23:45:44.773 INFO:tasks.workunit.client.0.smithi031.stdout:9/524: sync 2024-04-22T23:45:44.774 INFO:tasks.workunit.client.0.smithi031.stdout:6/570: mknod d3/d35/db1/cca 0 2024-04-22T23:45:44.775 INFO:tasks.workunit.client.0.smithi031.stdout:4/650: creat db/d14/d2d/d49/d59/d6a/d63/dc9/fe6 x:0 0 0 2024-04-22T23:45:44.776 INFO:tasks.workunit.client.0.smithi031.stdout:7/522: symlink d9/d14/d1f/d45/da6/lb8 0 2024-04-22T23:45:44.777 INFO:tasks.workunit.client.0.smithi031.stdout:4/651: write db/d14/d2d/d49/d59/d6a/d41/fc3 [132778,101655] 0 2024-04-22T23:45:44.780 INFO:tasks.workunit.client.0.smithi031.stdout:2/539: rename d6/dc/d49/d83/d85/laa to d6/dc/d42/d28/d94/lb1 0 2024-04-22T23:45:44.782 INFO:tasks.workunit.client.0.smithi031.stdout:1/504: mknod d1/d21/d29/d28/cb6 0 2024-04-22T23:45:44.784 INFO:tasks.workunit.client.0.smithi031.stdout:6/571: mkdir d3/d2a/d85/d96/dcb 0 2024-04-22T23:45:44.784 INFO:tasks.workunit.client.0.smithi031.stdout:4/652: dread db/d14/d1d/d7b/d7f/fae [0,4194304] 0 2024-04-22T23:45:44.785 INFO:tasks.workunit.client.0.smithi031.stdout:1/505: chown d1/db/d16/d2f/lb2 2 1 2024-04-22T23:45:44.786 INFO:tasks.workunit.client.0.smithi031.stdout:5/484: dwrite db/d16/d1d/f20 [0,4194304] 0 2024-04-22T23:45:44.787 INFO:tasks.workunit.client.0.smithi031.stdout:4/653: write db/d31/f33 [3461205,106883] 0 2024-04-22T23:45:44.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/540: unlink d6/d45/f77 0 2024-04-22T23:45:44.790 INFO:tasks.workunit.client.0.smithi031.stdout:3/441: mkdir d4/d54/d2c/d89/d2e/d45/d96 0 2024-04-22T23:45:44.791 INFO:tasks.workunit.client.0.smithi031.stdout:5/485: write db/d16/d2a/d59/f75 [1636544,97431] 0 2024-04-22T23:45:44.793 INFO:tasks.workunit.client.0.smithi031.stdout:6/572: write d3/f34 [2203573,18147] 0 2024-04-22T23:45:44.795 INFO:tasks.workunit.client.0.smithi031.stdout:2/541: readlink d6/dc/d42/d24/l66 0 2024-04-22T23:45:44.795 INFO:tasks.workunit.client.0.smithi031.stdout:1/506: creat d1/d21/d69/fb7 x:0 0 0 2024-04-22T23:45:44.796 INFO:tasks.workunit.client.0.smithi031.stdout:6/573: write d3/d1f/d4c/d8e/fb2 [91659,8391] 0 2024-04-22T23:45:44.800 INFO:tasks.workunit.client.0.smithi031.stdout:9/525: creat d2/dd/d75/d64/fa7 x:0 0 0 2024-04-22T23:45:44.800 INFO:tasks.workunit.client.0.smithi031.stdout:1/507: dread - d1/de/d1e/d34/d66/f92 zero size 2024-04-22T23:45:44.800 INFO:tasks.workunit.client.0.smithi031.stdout:1/508: dread - d1/db/d16/d2f/fae zero size 2024-04-22T23:45:44.803 INFO:tasks.workunit.client.0.smithi031.stdout:7/523: rename d9/d14/d22/f52 to d9/d14/d22/fb9 0 2024-04-22T23:45:44.804 INFO:tasks.workunit.client.0.smithi031.stdout:5/486: dwrite db/d16/d2a/d36/f38 [0,4194304] 0 2024-04-22T23:45:44.808 INFO:tasks.workunit.client.0.smithi031.stdout:6/574: dread - d3/d35/fc2 zero size 2024-04-22T23:45:44.813 INFO:tasks.workunit.client.0.smithi031.stdout:4/654: dread db/d31/d40/fb2 [0,4194304] 0 2024-04-22T23:45:44.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/487: creat db/d16/d1d/d24/d58/d5d/fa0 x:0 0 0 2024-04-22T23:45:44.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/575: creat d3/da4/fcc x:0 0 0 2024-04-22T23:45:44.827 INFO:tasks.workunit.client.0.smithi031.stdout:5/488: dwrite db/d16/d1d/d24/f76 [0,4194304] 0 2024-04-22T23:45:44.830 INFO:tasks.workunit.client.0.smithi031.stdout:3/442: write d4/d6/f16 [1562751,28548] 0 2024-04-22T23:45:44.831 INFO:tasks.workunit.client.0.smithi031.stdout:3/443: chown d4/d6/d11/d3e/c41 15749232 1 2024-04-22T23:45:44.836 INFO:tasks.workunit.client.0.smithi031.stdout:7/524: write d9/d15/d3a/d41/f83 [462618,81076] 0 2024-04-22T23:45:44.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/655: mknod db/d14/d1d/d7b/ce7 0 2024-04-22T23:45:44.839 INFO:tasks.workunit.client.0.smithi031.stdout:4/656: truncate db/d14/d2d/d49/f53 348492 0 2024-04-22T23:45:44.843 INFO:tasks.workunit.client.0.smithi031.stdout:6/576: symlink d3/d2a/d3c/d40/d5b/lcd 0 2024-04-22T23:45:44.845 INFO:tasks.workunit.client.0.smithi031.stdout:3/444: dread d4/d6/d11/f1c [0,4194304] 0 2024-04-22T23:45:44.847 INFO:tasks.workunit.client.0.smithi031.stdout:5/489: rename db/d16/d2a/d59/f72 to db/d16/d1d/d24/d27/d9e/fa1 0 2024-04-22T23:45:44.848 INFO:tasks.workunit.client.0.smithi031.stdout:6/577: dread d3/d2a/d3c/f5f [0,4194304] 0 2024-04-22T23:45:44.849 INFO:tasks.workunit.client.0.smithi031.stdout:5/490: fsync db/d16/d1d/d24/d58/d5d/f92 0 2024-04-22T23:45:44.851 INFO:tasks.workunit.client.0.smithi031.stdout:1/509: dwrite d1/d21/d29/f1a [0,4194304] 0 2024-04-22T23:45:44.851 INFO:tasks.workunit.client.0.smithi031.stdout:2/542: getdents d6/dc/d48/d4b 0 2024-04-22T23:45:44.851 INFO:tasks.workunit.client.0.smithi031.stdout:5/491: chown db/d16/d1d/d24/d58/f51 227 1 2024-04-22T23:45:44.852 INFO:tasks.workunit.client.0.smithi031.stdout:1/510: dread - d1/d21/d29/f98 zero size 2024-04-22T23:45:44.855 INFO:tasks.workunit.client.0.smithi031.stdout:4/657: read db/fc [1824047,81521] 0 2024-04-22T23:45:44.859 INFO:tasks.workunit.client.0.smithi031.stdout:5/492: dwrite db/d16/d1d/d24/d58/d5d/f7e [0,4194304] 0 2024-04-22T23:45:44.867 INFO:tasks.workunit.client.0.smithi031.stdout:4/658: dwrite db/d14/d2d/d49/d59/d6a/d63/dc9/fe6 [0,4194304] 0 2024-04-22T23:45:44.871 INFO:tasks.workunit.client.0.smithi031.stdout:2/543: mkdir d6/dc/d9a/d6c/db2 0 2024-04-22T23:45:44.871 INFO:tasks.workunit.client.0.smithi031.stdout:5/493: dwrite db/d16/d1d/d24/d32/f83 [4194304,4194304] 0 2024-04-22T23:45:44.872 INFO:tasks.workunit.client.0.smithi031.stdout:3/445: dwrite d4/d54/d59/f6e [0,4194304] 0 2024-04-22T23:45:44.876 INFO:tasks.workunit.client.0.smithi031.stdout:7/525: symlink d9/d14/d1f/d45/da6/daa/lba 0 2024-04-22T23:45:44.879 INFO:tasks.workunit.client.0.smithi031.stdout:1/511: fdatasync d1/db/d16/f25 0 2024-04-22T23:45:44.882 INFO:tasks.workunit.client.0.smithi031.stdout:3/446: dwrite d4/d54/d2c/d89/d2e/d45/f92 [0,4194304] 0 2024-04-22T23:45:44.890 INFO:tasks.workunit.client.0.smithi031.stdout:4/659: unlink db/d14/d1d/d7b/d7f/d2a/c57 0 2024-04-22T23:45:44.895 INFO:tasks.workunit.client.0.smithi031.stdout:2/544: mknod d6/dc/d17/d69/cb3 0 2024-04-22T23:45:44.897 INFO:tasks.workunit.client.0.smithi031.stdout:3/447: creat d4/d54/d2c/d89/d2e/f97 x:0 0 0 2024-04-22T23:45:44.898 INFO:tasks.workunit.client.0.smithi031.stdout:6/578: creat d3/d1f/fce x:0 0 0 2024-04-22T23:45:44.898 INFO:tasks.workunit.client.0.smithi031.stdout:3/448: write d4/d6/d11/d25/d44/d7f/d5f/f63 [272728,89048] 0 2024-04-22T23:45:44.899 INFO:tasks.workunit.client.0.smithi031.stdout:4/660: readlink db/l97 0 2024-04-22T23:45:44.900 INFO:tasks.workunit.client.0.smithi031.stdout:7/526: dread d9/d14/f75 [0,4194304] 0 2024-04-22T23:45:44.901 INFO:tasks.workunit.client.0.smithi031.stdout:6/579: write d3/d2a/d3c/d40/d7a/f9a [1266320,27774] 0 2024-04-22T23:45:44.914 INFO:tasks.workunit.client.0.smithi031.stdout:1/512: dread d1/de/d1e/d34/d41/f42 [0,4194304] 0 2024-04-22T23:45:44.924 INFO:tasks.workunit.client.0.smithi031.stdout:2/545: mkdir d6/dc/d42/d28/d94/db4 0 2024-04-22T23:45:44.927 INFO:tasks.workunit.client.0.smithi031.stdout:5/494: creat db/d16/fa2 x:0 0 0 2024-04-22T23:45:44.932 INFO:tasks.workunit.client.0.smithi031.stdout:4/661: creat db/d14/d1d/d7b/d7f/fe8 x:0 0 0 2024-04-22T23:45:44.935 INFO:tasks.workunit.client.0.smithi031.stdout:7/527: write d9/d14/d1f/d7f/f8c [3448167,96509] 0 2024-04-22T23:45:44.935 INFO:tasks.workunit.client.0.smithi031.stdout:3/449: dwrite d4/d6/d11/d25/d44/f7e [0,4194304] 0 2024-04-22T23:45:44.937 INFO:tasks.workunit.client.0.smithi031.stdout:6/580: unlink d3/d35/d86/c8b 0 2024-04-22T23:45:44.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/453: sync 2024-04-22T23:45:44.939 INFO:tasks.workunit.client.0.smithi031.stdout:0/482: sync 2024-04-22T23:45:44.940 INFO:tasks.workunit.client.0.smithi031.stdout:1/513: symlink d1/de/d1e/d34/d66/lb8 0 2024-04-22T23:45:44.941 INFO:tasks.workunit.client.0.smithi031.stdout:0/483: fsync d0/f61 0 2024-04-22T23:45:44.945 INFO:tasks.workunit.client.0.smithi031.stdout:1/514: read d1/de/d1e/d34/d41/f42 [3719047,557] 0 2024-04-22T23:45:44.946 INFO:tasks.workunit.client.0.smithi031.stdout:5/495: dread db/d16/d1d/f47 [0,4194304] 0 2024-04-22T23:45:44.946 INFO:tasks.workunit.client.0.smithi031.stdout:4/662: unlink db/d14/d2d/f37 0 2024-04-22T23:45:44.946 INFO:tasks.workunit.client.0.smithi031.stdout:8/454: write d5/da/d12/d37/d34/d6f/f6d [415726,79355] 0 2024-04-22T23:45:44.948 INFO:tasks.workunit.client.0.smithi031.stdout:3/450: mkdir d4/d54/d6d/d98 0 2024-04-22T23:45:44.949 INFO:tasks.workunit.client.0.smithi031.stdout:6/581: dwrite d3/d2a/d85/f9d [0,4194304] 0 2024-04-22T23:45:44.951 INFO:tasks.workunit.client.0.smithi031.stdout:1/515: truncate d1/db/d16/d1f/f8b 326858 0 2024-04-22T23:45:44.954 INFO:tasks.workunit.client.0.smithi031.stdout:8/455: read d5/da/d12/d37/d49/f54 [2577322,84024] 0 2024-04-22T23:45:44.958 INFO:tasks.workunit.client.0.smithi031.stdout:7/528: dread d9/d14/f18 [0,4194304] 0 2024-04-22T23:45:44.958 INFO:tasks.workunit.client.0.smithi031.stdout:0/484: mknod d0/d13/d18/d1e/caf 0 2024-04-22T23:45:44.959 INFO:tasks.workunit.client.0.smithi031.stdout:0/485: read - d0/d1/d3f/f93 zero size 2024-04-22T23:45:44.961 INFO:tasks.workunit.client.0.smithi031.stdout:5/496: creat db/d16/d2a/d36/d74/fa3 x:0 0 0 2024-04-22T23:45:44.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/546: rmdir d6/dc/d49/d71 0 2024-04-22T23:45:44.963 INFO:tasks.workunit.client.0.smithi031.stdout:2/547: readlink l5 0 2024-04-22T23:45:44.966 INFO:tasks.workunit.client.0.smithi031.stdout:6/582: creat d3/d35/fcf x:0 0 0 2024-04-22T23:45:44.968 INFO:tasks.workunit.client.0.smithi031.stdout:2/548: dwrite d6/dc/f9f [0,4194304] 0 2024-04-22T23:45:44.973 INFO:tasks.workunit.client.0.smithi031.stdout:0/486: creat d0/d1/d3f/fb0 x:0 0 0 2024-04-22T23:45:44.974 INFO:tasks.workunit.client.0.smithi031.stdout:8/456: dread d5/f9 [4194304,4194304] 0 2024-04-22T23:45:44.975 INFO:tasks.workunit.client.0.smithi031.stdout:0/487: stat d0/d1/d2/d36/d11/d1a/d56/f7e 0 2024-04-22T23:45:44.975 INFO:tasks.workunit.client.0.smithi031.stdout:8/457: chown d5/da/d12/d1d/l29 1102106 1 2024-04-22T23:45:44.978 INFO:tasks.workunit.client.0.smithi031.stdout:3/451: symlink d4/d54/d2c/d89/l99 0 2024-04-22T23:45:44.980 INFO:tasks.workunit.client.0.smithi031.stdout:1/516: mknod d1/cb9 0 2024-04-22T23:45:44.984 INFO:tasks.workunit.client.0.smithi031.stdout:2/549: rename d6/dc/d48/d72 to d6/dc/d57/d52/db5 0 2024-04-22T23:45:44.992 INFO:tasks.workunit.client.0.smithi031.stdout:8/458: mkdir d5/da/d12/d37/d34/d6f/d85 0 2024-04-22T23:45:44.997 INFO:tasks.workunit.client.0.smithi031.stdout:1/517: dread d1/d21/d4e/f71 [0,4194304] 0 2024-04-22T23:45:44.997 INFO:tasks.workunit.client.0.smithi031.stdout:1/518: chown d1/d21 30999 1 2024-04-22T23:45:45.000 INFO:tasks.workunit.client.0.smithi031.stdout:6/583: symlink d3/d2a/d3c/ld0 0 2024-04-22T23:45:45.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/663: getdents db/d31 0 2024-04-22T23:45:45.005 INFO:tasks.workunit.client.0.smithi031.stdout:1/519: dwrite d1/de/d1e/d34/fa2 [0,4194304] 0 2024-04-22T23:45:45.008 INFO:tasks.workunit.client.0.smithi031.stdout:4/664: dwrite db/d14/fad [0,4194304] 0 2024-04-22T23:45:45.009 INFO:tasks.workunit.client.0.smithi031.stdout:4/665: chown db/d14/d2d/d49/d59/d6a/f92 111425 1 2024-04-22T23:45:45.012 INFO:tasks.workunit.client.0.smithi031.stdout:8/459: dread d5/fb [0,4194304] 0 2024-04-22T23:45:45.013 INFO:tasks.workunit.client.0.smithi031.stdout:4/666: dread db/d14/fad [0,4194304] 0 2024-04-22T23:45:45.015 INFO:tasks.workunit.client.0.smithi031.stdout:6/584: symlink d3/d35/d86/ld1 0 2024-04-22T23:45:45.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/497: write f7 [1403869,60831] 0 2024-04-22T23:45:45.020 INFO:tasks.workunit.client.0.smithi031.stdout:7/529: dwrite d9/f3b [0,4194304] 0 2024-04-22T23:45:45.026 INFO:tasks.workunit.client.0.smithi031.stdout:8/460: fdatasync d5/da/df/d33/f5c 0 2024-04-22T23:45:45.027 INFO:tasks.workunit.client.0.smithi031.stdout:6/585: dread d3/d1f/d4c/d8e/f93 [0,4194304] 0 2024-04-22T23:45:45.028 INFO:tasks.workunit.client.0.smithi031.stdout:4/667: rmdir db/d14/d2d/d49/d59/d8a 39 2024-04-22T23:45:45.032 INFO:tasks.workunit.client.0.smithi031.stdout:5/498: dread - db/d16/d1d/f56 zero size 2024-04-22T23:45:45.034 INFO:tasks.workunit.client.0.smithi031.stdout:7/530: mkdir d9/d14/d1f/d45/da6/daa/dbb 0 2024-04-22T23:45:45.038 INFO:tasks.workunit.client.0.smithi031.stdout:8/461: symlink d5/d48/l86 0 2024-04-22T23:45:45.039 INFO:tasks.workunit.client.0.smithi031.stdout:3/452: dwrite d4/d6/d11/d25/d44/d7f/f8f [0,4194304] 0 2024-04-22T23:45:45.039 INFO:tasks.workunit.client.0.smithi031.stdout:3/453: readlink d4/l14 0 2024-04-22T23:45:45.044 INFO:tasks.workunit.client.0.smithi031.stdout:4/668: mknod db/d14/d1d/d7b/d7f/d2a/da1/da7/ce9 0 2024-04-22T23:45:45.044 INFO:tasks.workunit.client.0.smithi031.stdout:0/488: truncate d0/d1/d2/d36/d11/f9e 279219 0 2024-04-22T23:45:45.046 INFO:tasks.workunit.client.0.smithi031.stdout:5/499: fdatasync db/fd 0 2024-04-22T23:45:45.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/462: stat d5/c7 0 2024-04-22T23:45:45.051 INFO:tasks.workunit.client.0.smithi031.stdout:3/454: dread d4/d6/da/f3d [0,4194304] 0 2024-04-22T23:45:45.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/463: chown c3 14566 1 2024-04-22T23:45:45.052 INFO:tasks.workunit.client.0.smithi031.stdout:3/455: truncate d4/d6/da/f67 788315 0 2024-04-22T23:45:45.053 INFO:tasks.workunit.client.0.smithi031.stdout:3/456: dread - d4/d54/d2c/d89/d2e/f97 zero size 2024-04-22T23:45:45.054 INFO:tasks.workunit.client.0.smithi031.stdout:6/586: rename d3/d2a/d3c/f52 to d3/d77/db5/fd2 0 2024-04-22T23:45:45.058 INFO:tasks.workunit.client.0.smithi031.stdout:0/489: creat d0/d1/d2/d36/fb1 x:0 0 0 2024-04-22T23:45:45.059 INFO:tasks.workunit.client.0.smithi031.stdout:3/457: dwrite d4/d54/d56/f87 [0,4194304] 0 2024-04-22T23:45:45.060 INFO:tasks.workunit.client.0.smithi031.stdout:5/500: mkdir db/d16/d2a/d59/d90/da4 0 2024-04-22T23:45:45.061 INFO:tasks.workunit.client.0.smithi031.stdout:3/458: write d4/d6/d11/d3e/f7c [624750,16776] 0 2024-04-22T23:45:45.063 INFO:tasks.workunit.client.0.smithi031.stdout:7/531: symlink d9/d14/d1f/d45/da6/lbc 0 2024-04-22T23:45:45.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/459: chown d4/d6/d11/c34 3885 1 2024-04-22T23:45:45.148 INFO:tasks.workunit.client.0.smithi031.stdout:5/501: creat db/d16/d1d/d24/d27/d9e/fa5 x:0 0 0 2024-04-22T23:45:45.149 INFO:tasks.workunit.client.0.smithi031.stdout:3/460: write d4/d6/f23 [983014,16432] 0 2024-04-22T23:45:45.149 INFO:tasks.workunit.client.0.smithi031.stdout:5/502: chown l6 1429758 1 2024-04-22T23:45:45.150 INFO:tasks.workunit.client.0.smithi031.stdout:7/532: symlink d9/d14/d1f/d45/da6/daa/lbd 0 2024-04-22T23:45:45.153 INFO:tasks.workunit.client.0.smithi031.stdout:3/461: dread d4/d6/d11/f1c [0,4194304] 0 2024-04-22T23:45:45.170 INFO:tasks.workunit.client.0.smithi031.stdout:3/462: dwrite d4/d49/f6f [0,4194304] 0 2024-04-22T23:45:45.173 INFO:tasks.workunit.client.0.smithi031.stdout:9/526: sync 2024-04-22T23:45:45.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/527: read d2/dd/d75/d64/d5d/d59/d8e/f9d [1258561,37617] 0 2024-04-22T23:45:45.177 INFO:tasks.workunit.client.0.smithi031.stdout:9/528: chown d2/dc/d3e/f41 1177 1 2024-04-22T23:45:45.180 INFO:tasks.workunit.client.0.smithi031.stdout:3/463: rename d4/d54/d2c/d89/d2e/d45/c66 to d4/d49/d72/c9a 0 2024-04-22T23:45:45.192 INFO:tasks.workunit.client.0.smithi031.stdout:9/529: creat d2/dd/d75/d64/fa8 x:0 0 0 2024-04-22T23:45:45.199 INFO:tasks.workunit.client.0.smithi031.stdout:9/530: symlink d2/la9 0 2024-04-22T23:45:45.231 INFO:tasks.workunit.client.0.smithi031.stdout:7/533: mkdir d9/d14/d22/dbe 0 2024-04-22T23:45:45.231 INFO:tasks.workunit.client.0.smithi031.stdout:7/534: chown d9/cc 1097230 1 2024-04-22T23:45:45.240 INFO:tasks.workunit.client.0.smithi031.stdout:8/464: sync 2024-04-22T23:45:45.240 INFO:tasks.workunit.client.0.smithi031.stdout:1/520: sync 2024-04-22T23:45:45.240 INFO:tasks.workunit.client.0.smithi031.stdout:2/550: sync 2024-04-22T23:45:45.244 INFO:tasks.workunit.client.0.smithi031.stdout:2/551: creat d6/dc/d42/d28/fb6 x:0 0 0 2024-04-22T23:45:45.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/521: dread d1/d21/d4e/f71 [0,4194304] 0 2024-04-22T23:45:45.245 INFO:tasks.workunit.client.0.smithi031.stdout:2/552: read d6/dc/d42/d28/f37 [1406899,3230] 0 2024-04-22T23:45:45.247 INFO:tasks.workunit.client.0.smithi031.stdout:1/522: write d1/db/d16/f83 [927242,1772] 0 2024-04-22T23:45:45.250 INFO:tasks.workunit.client.0.smithi031.stdout:6/587: creat d3/d1f/fd3 x:0 0 0 2024-04-22T23:45:45.250 INFO:tasks.workunit.client.0.smithi031.stdout:2/553: readlink d6/dc/d42/l47 0 2024-04-22T23:45:45.252 INFO:tasks.workunit.client.0.smithi031.stdout:5/503: symlink db/d16/d1d/d24/d32/la6 0 2024-04-22T23:45:45.259 INFO:tasks.workunit.client.0.smithi031.stdout:2/554: creat d6/dc/d9a/d6c/d86/fb7 x:0 0 0 2024-04-22T23:45:45.261 INFO:tasks.workunit.client.0.smithi031.stdout:6/588: symlink d3/d2a/d3c/d5e/ld4 0 2024-04-22T23:45:45.262 INFO:tasks.workunit.client.0.smithi031.stdout:2/555: symlink d6/dc/d57/d52/db5/lb8 0 2024-04-22T23:45:45.263 INFO:tasks.workunit.client.0.smithi031.stdout:2/556: write d6/dc/d49/d83/d85/da9/fad [1011639,15645] 0 2024-04-22T23:45:45.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/557: chown d6/dc/d42/d28/f37 15 1 2024-04-22T23:45:45.265 INFO:tasks.workunit.client.0.smithi031.stdout:6/589: creat d3/d1f/da3/fd5 x:0 0 0 2024-04-22T23:45:45.271 INFO:tasks.workunit.client.0.smithi031.stdout:0/490: dwrite d0/d13/d18/d1e/d25/f82 [0,4194304] 0 2024-04-22T23:45:45.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/491: write d0/d1/d3f/f8b [148050,21192] 0 2024-04-22T23:45:45.274 INFO:tasks.workunit.client.0.smithi031.stdout:0/492: dread - d0/d13/d18/d1e/d25/d8e/da7/d2b/f90 zero size 2024-04-22T23:45:45.277 INFO:tasks.workunit.client.0.smithi031.stdout:4/669: truncate db/d14/d2d/d49/d59/d8a/d89/f8f 466676 0 2024-04-22T23:45:45.277 INFO:tasks.workunit.client.0.smithi031.stdout:4/670: chown db/d31/d40/fb2 0 1 2024-04-22T23:45:45.284 INFO:tasks.workunit.client.0.smithi031.stdout:4/671: dwrite db/d14/d1d/d7b/d7f/d2a/d47/d80/f95 [0,4194304] 0 2024-04-22T23:45:45.285 INFO:tasks.workunit.client.0.smithi031.stdout:4/672: chown db/d14/d2d/d49/d59/d6a/d41 292 1 2024-04-22T23:45:45.290 INFO:tasks.workunit.client.0.smithi031.stdout:4/673: write db/d14/fad [3481765,77919] 0 2024-04-22T23:45:45.295 INFO:tasks.workunit.client.0.smithi031.stdout:4/674: symlink db/d31/d40/lea 0 2024-04-22T23:45:45.298 INFO:tasks.workunit.client.0.smithi031.stdout:4/675: write db/d14/d2d/d49/fb3 [821817,117674] 0 2024-04-22T23:45:45.302 INFO:tasks.workunit.client.0.smithi031.stdout:8/465: creat d5/f87 x:0 0 0 2024-04-22T23:45:45.302 INFO:tasks.workunit.client.0.smithi031.stdout:8/466: stat d5/da/d12/d2e/c3d 0 2024-04-22T23:45:45.305 INFO:tasks.workunit.client.0.smithi031.stdout:3/464: stat d4/d49/d72/c9a 0 2024-04-22T23:45:45.320 INFO:tasks.workunit.client.0.smithi031.stdout:9/531: write d2/dc/d8b/d3d/d40/f44 [564092,30665] 0 2024-04-22T23:45:45.337 INFO:tasks.workunit.client.0.smithi031.stdout:1/523: rename d1/d21/d29 to d1/d3f/d54/dba 0 2024-04-22T23:45:45.338 INFO:tasks.workunit.client.0.smithi031.stdout:1/524: dread - d1/d35/d36/f9e zero size 2024-04-22T23:45:45.340 INFO:tasks.workunit.client.0.smithi031.stdout:5/504: rename db/d16/d1d/d24/d58/d5d/f7e to db/d16/d2a/d7c/fa7 0 2024-04-22T23:45:45.343 INFO:tasks.workunit.client.0.smithi031.stdout:7/535: mkdir d9/d15/dbf 0 2024-04-22T23:45:45.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/536: dread - d9/d14/d1f/d47/f7c zero size 2024-04-22T23:45:45.346 INFO:tasks.workunit.client.0.smithi031.stdout:4/676: unlink db/d14/d2d/d49/f53 0 2024-04-22T23:45:45.347 INFO:tasks.workunit.client.0.smithi031.stdout:6/590: rename d3/d1f/c2d to d3/d35/d75/d7d/dc4/cd6 0 2024-04-22T23:45:45.349 INFO:tasks.workunit.client.0.smithi031.stdout:1/525: dread d1/d21/fb5 [0,4194304] 0 2024-04-22T23:45:45.350 INFO:tasks.workunit.client.0.smithi031.stdout:1/526: fsync d1/de/d1e/f64 0 2024-04-22T23:45:45.351 INFO:tasks.workunit.client.0.smithi031.stdout:4/677: mkdir db/d14/d1d/d7b/d7f/d2a/d47/d80/deb 0 2024-04-22T23:45:45.358 INFO:tasks.workunit.client.0.smithi031.stdout:7/537: symlink d9/d15/dac/lc0 0 2024-04-22T23:45:45.359 INFO:tasks.workunit.client.0.smithi031.stdout:7/538: write d9/d14/d22/d5c/f9a [976513,58625] 0 2024-04-22T23:45:45.359 INFO:tasks.workunit.client.0.smithi031.stdout:1/527: mkdir d1/d35/dbb 0 2024-04-22T23:45:45.359 INFO:tasks.workunit.client.0.smithi031.stdout:7/539: fsync d9/d15/f99 0 2024-04-22T23:45:45.363 INFO:tasks.workunit.client.0.smithi031.stdout:5/505: getdents db/d16/d2a/d36/d74/d82/d86 0 2024-04-22T23:45:45.365 INFO:tasks.workunit.client.0.smithi031.stdout:1/528: mkdir d1/d21/d4b/dbc 0 2024-04-22T23:45:45.366 INFO:tasks.workunit.client.0.smithi031.stdout:2/558: truncate d6/f1c 3028849 0 2024-04-22T23:45:45.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/540: creat d9/d14/d1f/d47/fc1 x:0 0 0 2024-04-22T23:45:45.368 INFO:tasks.workunit.client.0.smithi031.stdout:2/559: write d6/dc/d48/d4b/f64 [1016482,69308] 0 2024-04-22T23:45:45.373 INFO:tasks.workunit.client.0.smithi031.stdout:1/529: creat d1/d35/d36/fbd x:0 0 0 2024-04-22T23:45:45.375 INFO:tasks.workunit.client.0.smithi031.stdout:4/678: unlink db/d14/d2d/d49/d59/d83/d9c/f55 0 2024-04-22T23:45:45.379 INFO:tasks.workunit.client.0.smithi031.stdout:5/506: dread db/d16/f3e [4194304,4194304] 0 2024-04-22T23:45:45.381 INFO:tasks.workunit.client.0.smithi031.stdout:1/530: symlink d1/d21/d4e/lbe 0 2024-04-22T23:45:45.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/679: creat db/d14/d1d/d7b/d7f/d2a/d48/fec x:0 0 0 2024-04-22T23:45:45.388 INFO:tasks.workunit.client.0.smithi031.stdout:0/493: symlink d0/d1/d2/d36/d11/lb2 0 2024-04-22T23:45:45.389 INFO:tasks.workunit.client.0.smithi031.stdout:5/507: fdatasync db/f29 0 2024-04-22T23:45:45.390 INFO:tasks.workunit.client.0.smithi031.stdout:1/531: rename d1/db/d16/d1f/l33 to d1/d3f/d54/lbf 0 2024-04-22T23:45:45.392 INFO:tasks.workunit.client.0.smithi031.stdout:0/494: unlink d0/l62 0 2024-04-22T23:45:45.394 INFO:tasks.workunit.client.0.smithi031.stdout:5/508: dread db/d16/d2a/d36/f38 [0,4194304] 0 2024-04-22T23:45:45.395 INFO:tasks.workunit.client.0.smithi031.stdout:5/509: write db/d16/f65 [242028,110651] 0 2024-04-22T23:45:45.396 INFO:tasks.workunit.client.0.smithi031.stdout:1/532: creat d1/db/d16/d2f/fc0 x:0 0 0 2024-04-22T23:45:45.397 INFO:tasks.workunit.client.0.smithi031.stdout:4/680: creat db/d14/d1d/d7b/dc1/fed x:0 0 0 2024-04-22T23:45:45.400 INFO:tasks.workunit.client.0.smithi031.stdout:0/495: unlink d0/d1/d2/d36/f8d 0 2024-04-22T23:45:45.401 INFO:tasks.workunit.client.0.smithi031.stdout:3/465: truncate d4/d54/d2c/d89/f2b 314541 0 2024-04-22T23:45:45.404 INFO:tasks.workunit.client.0.smithi031.stdout:8/467: dwrite d5/da/d12/d37/d49/f4f [0,4194304] 0 2024-04-22T23:45:45.408 INFO:tasks.workunit.client.0.smithi031.stdout:3/466: rmdir d4/d54/d6d 39 2024-04-22T23:45:45.412 INFO:tasks.workunit.client.0.smithi031.stdout:3/467: write d4/d54/d59/f6e [3595618,49807] 0 2024-04-22T23:45:45.412 INFO:tasks.workunit.client.0.smithi031.stdout:4/681: dwrite db/d14/d1d/d7b/d7f/d2a/d47/d80/f95 [0,4194304] 0 2024-04-22T23:45:45.414 INFO:tasks.workunit.client.0.smithi031.stdout:0/496: dwrite d0/d13/d18/d1e/f8c [0,4194304] 0 2024-04-22T23:45:45.415 INFO:tasks.workunit.client.0.smithi031.stdout:3/468: chown d4/d6/d11/f3c 113960 1 2024-04-22T23:45:45.415 INFO:tasks.workunit.client.0.smithi031.stdout:5/510: dread db/d16/f65 [0,4194304] 0 2024-04-22T23:45:45.416 INFO:tasks.workunit.client.0.smithi031.stdout:8/468: rmdir d5/da/d12/d37/d34/d6f/d5b 39 2024-04-22T23:45:45.417 INFO:tasks.workunit.client.0.smithi031.stdout:0/497: write d0/d13/f67 [833250,4816] 0 2024-04-22T23:45:45.418 INFO:tasks.workunit.client.0.smithi031.stdout:5/511: chown db/d16/d1d/d24/d58/d5d/fa0 497976958 1 2024-04-22T23:45:45.418 INFO:tasks.workunit.client.0.smithi031.stdout:0/498: fdatasync d0/d13/d18/f48 0 2024-04-22T23:45:45.419 INFO:tasks.workunit.client.0.smithi031.stdout:4/682: truncate db/d14/d2d/d49/d59/d6a/d63/fe0 660310 0 2024-04-22T23:45:45.428 INFO:tasks.workunit.client.0.smithi031.stdout:5/512: mkdir db/d16/d2a/d59/da8 0 2024-04-22T23:45:45.430 INFO:tasks.workunit.client.0.smithi031.stdout:5/513: chown db/d16/d1d/f56 240321492 1 2024-04-22T23:45:45.432 INFO:tasks.workunit.client.0.smithi031.stdout:0/499: rmdir d0/d13/d18/d1e/d25/d4c 39 2024-04-22T23:45:45.432 INFO:tasks.workunit.client.0.smithi031.stdout:9/532: sync 2024-04-22T23:45:45.434 INFO:tasks.workunit.client.0.smithi031.stdout:8/469: dwrite d5/da/d12/d37/d34/d6f/d5b/f84 [0,4194304] 0 2024-04-22T23:45:45.434 INFO:tasks.workunit.client.0.smithi031.stdout:3/469: creat d4/d54/d6d/f9b x:0 0 0 2024-04-22T23:45:45.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/500: fdatasync d0/d13/d18/d1e/d25/d8e/da7/d2b/fa3 0 2024-04-22T23:45:45.438 INFO:tasks.workunit.client.0.smithi031.stdout:6/591: write d3/f45 [5193914,11171] 0 2024-04-22T23:45:45.443 INFO:tasks.workunit.client.0.smithi031.stdout:5/514: rename db/d16/d1d/d24/d32/d62/l70 to db/d16/d2a/d36/d74/la9 0 2024-04-22T23:45:45.444 INFO:tasks.workunit.client.0.smithi031.stdout:8/470: dread d5/da/d12/f44 [0,4194304] 0 2024-04-22T23:45:45.450 INFO:tasks.workunit.client.0.smithi031.stdout:0/501: readlink d0/d13/d18/d1e/d25/d4c/l77 0 2024-04-22T23:45:45.450 INFO:tasks.workunit.client.0.smithi031.stdout:0/502: chown d0/d13/d18/d6c/f74 7087459 1 2024-04-22T23:45:45.458 INFO:tasks.workunit.client.0.smithi031.stdout:3/470: mkdir d4/d54/d2c/d89/d9c 0 2024-04-22T23:45:45.458 INFO:tasks.workunit.client.0.smithi031.stdout:3/471: fsync d4/d6/da/f67 0 2024-04-22T23:45:45.460 INFO:tasks.workunit.client.0.smithi031.stdout:0/503: write d0/d1/f6b [654688,127193] 0 2024-04-22T23:45:45.461 INFO:tasks.workunit.client.0.smithi031.stdout:0/504: chown d0/c91 49330 1 2024-04-22T23:45:45.463 INFO:tasks.workunit.client.0.smithi031.stdout:7/541: truncate d9/d15/d6d/d65/f86 1801291 0 2024-04-22T23:45:45.468 INFO:tasks.workunit.client.0.smithi031.stdout:3/472: creat d4/d6/d11/d25/d44/d7f/d5f/f9d x:0 0 0 2024-04-22T23:45:45.468 INFO:tasks.workunit.client.0.smithi031.stdout:7/542: dread d9/d14/d1f/d47/f69 [0,4194304] 0 2024-04-22T23:45:45.468 INFO:tasks.workunit.client.0.smithi031.stdout:6/592: dread d3/f1a [0,4194304] 0 2024-04-22T23:45:45.474 INFO:tasks.workunit.client.0.smithi031.stdout:0/505: symlink d0/d1/d2/d36/d11/d1c/lb3 0 2024-04-22T23:45:45.478 INFO:tasks.workunit.client.0.smithi031.stdout:9/533: rename d2/dc/d8b/d3d/f8f to d2/dc/faa 0 2024-04-22T23:45:45.478 INFO:tasks.workunit.client.0.smithi031.stdout:0/506: dread d0/d13/d18/d1e/f8c [0,4194304] 0 2024-04-22T23:45:45.484 INFO:tasks.workunit.client.0.smithi031.stdout:1/533: unlink d1/d3f/d54/lbf 0 2024-04-22T23:45:45.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/593: creat d3/d77/d8d/fd7 x:0 0 0 2024-04-22T23:45:45.489 INFO:tasks.workunit.client.0.smithi031.stdout:6/594: readlink d3/d1f/d4c/d8e/l9c 0 2024-04-22T23:45:45.496 INFO:tasks.workunit.client.0.smithi031.stdout:3/473: rename d4/d6/d22/c84 to d4/d54/d56/c9e 0 2024-04-22T23:45:45.496 INFO:tasks.workunit.client.0.smithi031.stdout:3/474: chown d4/d6/d11/d25/d44/f71 84 1 2024-04-22T23:45:45.499 INFO:tasks.workunit.client.0.smithi031.stdout:5/515: getdents db/d16/d1d/d24/d32/d41 0 2024-04-22T23:45:45.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/534: mkdir d1/db/d16/d2f/dc1 0 2024-04-22T23:45:45.503 INFO:tasks.workunit.client.0.smithi031.stdout:6/595: truncate d3/d2a/f53 114080 0 2024-04-22T23:45:45.503 INFO:tasks.workunit.client.0.smithi031.stdout:4/683: write db/d14/f45 [1231629,86843] 0 2024-04-22T23:45:45.504 INFO:tasks.workunit.client.0.smithi031.stdout:5/516: dread db/d16/d1d/d24/d32/d62/f67 [0,4194304] 0 2024-04-22T23:45:45.504 INFO:tasks.workunit.client.0.smithi031.stdout:6/596: dread - d3/d2a/d85/d96/fbf zero size 2024-04-22T23:45:45.505 INFO:tasks.workunit.client.0.smithi031.stdout:4/684: read db/fc [4086667,54717] 0 2024-04-22T23:45:45.505 INFO:tasks.workunit.client.0.smithi031.stdout:6/597: chown d3/d2a/d3c/d40/c6e 1 1 2024-04-22T23:45:45.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/507: unlink d0/d13/d18/d1e/f8c 0 2024-04-22T23:45:45.508 INFO:tasks.workunit.client.0.smithi031.stdout:3/475: symlink d4/d6/d22/l9f 0 2024-04-22T23:45:45.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/517: dread db/d16/d1d/d24/d58/f4f [0,4194304] 0 2024-04-22T23:45:45.510 INFO:tasks.workunit.client.0.smithi031.stdout:9/534: creat d2/dc/d8b/d3d/d40/da4/fab x:0 0 0 2024-04-22T23:45:45.510 INFO:tasks.workunit.client.0.smithi031.stdout:5/518: chown db/d16/d1d/d24/d32/d62 7 1 2024-04-22T23:45:45.516 INFO:tasks.workunit.client.0.smithi031.stdout:6/598: creat d3/d35/d86/dae/fd8 x:0 0 0 2024-04-22T23:45:45.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/535: dwrite d2/dc/f12 [0,4194304] 0 2024-04-22T23:45:45.517 INFO:tasks.workunit.client.0.smithi031.stdout:6/599: stat d3/d2a/d3c/d40/f4e 0 2024-04-22T23:45:45.519 INFO:tasks.workunit.client.0.smithi031.stdout:6/600: write d3/d2a/d3c/f73 [1093840,32905] 0 2024-04-22T23:45:45.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/508: dread d0/d1/f6b [0,4194304] 0 2024-04-22T23:45:45.527 INFO:tasks.workunit.client.0.smithi031.stdout:1/535: mknod d1/d21/d4b/d6f/da6/cc2 0 2024-04-22T23:45:45.529 INFO:tasks.workunit.client.0.smithi031.stdout:1/536: write d1/de/d1e/d34/d41/f4c [2658973,12078] 0 2024-04-22T23:45:45.531 INFO:tasks.workunit.client.0.smithi031.stdout:5/519: rename db/d16/d1d/d24/d27/l4e to db/d16/d1d/d24/d32/d62/laa 0 2024-04-22T23:45:45.546 INFO:tasks.workunit.client.0.smithi031.stdout:1/537: symlink d1/de/d1e/d34/d66/lc3 0 2024-04-22T23:45:45.547 INFO:tasks.workunit.client.0.smithi031.stdout:5/520: mknod db/d16/d2a/d36/d74/cab 0 2024-04-22T23:45:45.547 INFO:tasks.workunit.client.0.smithi031.stdout:1/538: write d1/d21/d4b/f5d [312872,32283] 0 2024-04-22T23:45:45.552 INFO:tasks.workunit.client.0.smithi031.stdout:6/601: fsync d3/f38 0 2024-04-22T23:45:45.553 INFO:tasks.workunit.client.0.smithi031.stdout:7/543: dwrite d9/d15/d3a/d41/f54 [0,4194304] 0 2024-04-22T23:45:45.554 INFO:tasks.workunit.client.0.smithi031.stdout:7/544: write d9/d15/d6d/f8e [852077,37858] 0 2024-04-22T23:45:45.560 INFO:tasks.workunit.client.0.smithi031.stdout:7/545: dwrite d9/d14/d22/d5c/f80 [0,4194304] 0 2024-04-22T23:45:45.562 INFO:tasks.workunit.client.0.smithi031.stdout:7/546: readlink d9/d15/d6d/d65/l67 0 2024-04-22T23:45:45.562 INFO:tasks.workunit.client.0.smithi031.stdout:1/539: creat d1/d21/fc4 x:0 0 0 2024-04-22T23:45:45.570 INFO:tasks.workunit.client.0.smithi031.stdout:1/540: dwrite d1/d35/f96 [0,4194304] 0 2024-04-22T23:45:45.570 INFO:tasks.workunit.client.0.smithi031.stdout:7/547: dwrite d9/d14/f18 [0,4194304] 0 2024-04-22T23:45:45.573 INFO:tasks.workunit.client.0.smithi031.stdout:7/548: write d9/d14/d22/d5c/f80 [3249700,125081] 0 2024-04-22T23:45:45.581 INFO:tasks.workunit.client.0.smithi031.stdout:1/541: dwrite d1/db/d16/d1f/d52/f5c [0,4194304] 0 2024-04-22T23:45:45.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/602: symlink d3/ld9 0 2024-04-22T23:45:45.586 INFO:tasks.workunit.client.0.smithi031.stdout:3/476: write d4/d54/d56/f87 [5029076,37118] 0 2024-04-22T23:45:45.588 INFO:tasks.workunit.client.0.smithi031.stdout:3/477: truncate d4/d6/f23 1672687 0 2024-04-22T23:45:45.588 INFO:tasks.workunit.client.0.smithi031.stdout:0/509: mkdir d0/d1/d2/db4 0 2024-04-22T23:45:45.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/510: fdatasync d0/d1/d3f/f93 0 2024-04-22T23:45:45.590 INFO:tasks.workunit.client.0.smithi031.stdout:7/549: creat d9/d14/d1f/fc2 x:0 0 0 2024-04-22T23:45:45.592 INFO:tasks.workunit.client.0.smithi031.stdout:5/521: link db/f23 db/d16/d2a/d59/da8/fac 0 2024-04-22T23:45:45.595 INFO:tasks.workunit.client.0.smithi031.stdout:5/522: dread f5 [0,4194304] 0 2024-04-22T23:45:45.595 INFO:tasks.workunit.client.0.smithi031.stdout:6/603: creat d3/d2a/d3c/d5e/fda x:0 0 0 2024-04-22T23:45:45.599 INFO:tasks.workunit.client.0.smithi031.stdout:7/550: symlink d9/d15/d6d/lc3 0 2024-04-22T23:45:45.602 INFO:tasks.workunit.client.0.smithi031.stdout:1/542: link d1/db/d16/fa9 d1/db/d16/d1f/fc5 0 2024-04-22T23:45:45.603 INFO:tasks.workunit.client.0.smithi031.stdout:1/543: stat d1/de/c3a 0 2024-04-22T23:45:45.603 INFO:tasks.workunit.client.0.smithi031.stdout:1/544: chown d1/d3f/d54/dba/d28/c49 79 1 2024-04-22T23:45:45.608 INFO:tasks.workunit.client.0.smithi031.stdout:2/560: sync 2024-04-22T23:45:45.609 INFO:tasks.workunit.client.0.smithi031.stdout:7/551: write d9/d15/d6d/d65/fb7 [1328025,116162] 0 2024-04-22T23:45:45.611 INFO:tasks.workunit.client.0.smithi031.stdout:6/604: creat d3/d2a/d3c/db6/fdb x:0 0 0 2024-04-22T23:45:45.611 INFO:tasks.workunit.client.0.smithi031.stdout:3/478: link d4/d6/c7a d4/d54/d6d/ca0 0 2024-04-22T23:45:45.612 INFO:tasks.workunit.client.0.smithi031.stdout:1/545: dwrite d1/db/d16/d2f/f3b [0,4194304] 0 2024-04-22T23:45:45.613 INFO:tasks.workunit.client.0.smithi031.stdout:9/536: dwrite d2/d15/d5a/fa3 [0,4194304] 0 2024-04-22T23:45:45.619 INFO:tasks.workunit.client.0.smithi031.stdout:7/552: chown d9/d14/d22/c6a 8936154 1 2024-04-22T23:45:45.622 INFO:tasks.workunit.client.0.smithi031.stdout:6/605: creat d3/d2a/d3c/d40/fdc x:0 0 0 2024-04-22T23:45:45.624 INFO:tasks.workunit.client.0.smithi031.stdout:3/479: symlink d4/d6/d4c/d8c/la1 0 2024-04-22T23:45:45.625 INFO:tasks.workunit.client.0.smithi031.stdout:3/480: write d4/d6/d11/d3e/f5a [1382791,29523] 0 2024-04-22T23:45:45.630 INFO:tasks.workunit.client.0.smithi031.stdout:5/523: getdents db 0 2024-04-22T23:45:45.635 INFO:tasks.workunit.client.0.smithi031.stdout:6/606: fsync d3/f38 0 2024-04-22T23:45:45.641 INFO:tasks.workunit.client.0.smithi031.stdout:2/561: rename d6/d45/f95 to d6/dc/d9a/fb9 0 2024-04-22T23:45:45.642 INFO:tasks.workunit.client.0.smithi031.stdout:1/546: dread d1/db/d16/f83 [0,4194304] 0 2024-04-22T23:45:45.642 INFO:tasks.workunit.client.0.smithi031.stdout:5/524: mknod db/d16/d2a/d36/d74/d82/d86/cad 0 2024-04-22T23:45:45.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/537: dread d2/dd/d75/d4a/f61 [0,4194304] 0 2024-04-22T23:45:45.649 INFO:tasks.workunit.client.0.smithi031.stdout:3/481: mkdir d4/d54/d88/da2 0 2024-04-22T23:45:45.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/538: write d2/dc/d8b/d3d/d40/f44 [2991870,36717] 0 2024-04-22T23:45:45.653 INFO:tasks.workunit.client.0.smithi031.stdout:7/553: rename d9/d15/d3a/c40 to d9/d15/dbf/cc4 0 2024-04-22T23:45:45.655 INFO:tasks.workunit.client.0.smithi031.stdout:1/547: mkdir d1/d3f/d54/dc6 0 2024-04-22T23:45:45.656 INFO:tasks.workunit.client.0.smithi031.stdout:5/525: creat db/d16/d1d/d24/d27/d9e/fae x:0 0 0 2024-04-22T23:45:45.658 INFO:tasks.workunit.client.0.smithi031.stdout:0/511: unlink d0/d1/d2/d36/d11/d1a/d3c/c8f 0 2024-04-22T23:45:45.660 INFO:tasks.workunit.client.0.smithi031.stdout:2/562: dwrite d6/dc/d9a/fb9 [0,4194304] 0 2024-04-22T23:45:45.660 INFO:tasks.workunit.client.0.smithi031.stdout:5/526: chown db/d16/d2a/d36/d74/d82/d86/f91 6 1 2024-04-22T23:45:45.663 INFO:tasks.workunit.client.0.smithi031.stdout:3/482: rename d4/d6/d11/d25/d44/f50 to d4/d6/d4c/d8c/fa3 0 2024-04-22T23:45:45.670 INFO:tasks.workunit.client.0.smithi031.stdout:0/512: write d0/d13/d18/d1e/d25/d8e/fa8 [307836,29976] 0 2024-04-22T23:45:45.672 INFO:tasks.workunit.client.0.smithi031.stdout:0/513: write d0/d13/d18/f6d [904832,74450] 0 2024-04-22T23:45:45.676 INFO:tasks.workunit.client.0.smithi031.stdout:3/483: creat d4/d6/d11/d25/d44/fa4 x:0 0 0 2024-04-22T23:45:45.677 INFO:tasks.workunit.client.0.smithi031.stdout:5/527: truncate db/d16/d2a/d36/d74/d82/d86/f95 2529950 0 2024-04-22T23:45:45.678 INFO:tasks.workunit.client.0.smithi031.stdout:2/563: getdents d6/dc/d42/d24/d9c 0 2024-04-22T23:45:45.678 INFO:tasks.workunit.client.0.smithi031.stdout:6/607: dwrite d3/d2a/d3c/f98 [0,4194304] 0 2024-04-22T23:45:45.680 INFO:tasks.workunit.client.0.smithi031.stdout:0/514: mkdir d0/db5 0 2024-04-22T23:45:45.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/685: sync 2024-04-22T23:45:45.682 INFO:tasks.workunit.client.0.smithi031.stdout:8/471: sync 2024-04-22T23:45:45.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/528: symlink db/d16/d1d/d24/d58/d5d/laf 0 2024-04-22T23:45:45.685 INFO:tasks.workunit.client.0.smithi031.stdout:6/608: creat d3/d1f/d4c/d8e/fdd x:0 0 0 2024-04-22T23:45:45.686 INFO:tasks.workunit.client.0.smithi031.stdout:4/686: chown db/d14/d1d/l75 44787 1 2024-04-22T23:45:45.686 INFO:tasks.workunit.client.0.smithi031.stdout:9/539: rename d2/dc/d3e/f6c to d2/dd/fac 0 2024-04-22T23:45:45.687 INFO:tasks.workunit.client.0.smithi031.stdout:3/484: truncate d4/d6/d11/f37 541172 0 2024-04-22T23:45:45.689 INFO:tasks.workunit.client.0.smithi031.stdout:9/540: readlink d2/d15/d5a/l84 0 2024-04-22T23:45:45.690 INFO:tasks.workunit.client.0.smithi031.stdout:8/472: creat d5/da/d12/d2e/d4d/d56/f88 x:0 0 0 2024-04-22T23:45:45.690 INFO:tasks.workunit.client.0.smithi031.stdout:6/609: mknod d3/d35/d75/cde 0 2024-04-22T23:45:45.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/554: dwrite d9/f3e [4194304,4194304] 0 2024-04-22T23:45:45.693 INFO:tasks.workunit.client.0.smithi031.stdout:1/548: rename d1/de/d1e/f95 to d1/d21/d4b/d6f/da6/fc7 0 2024-04-22T23:45:45.695 INFO:tasks.workunit.client.0.smithi031.stdout:3/485: mkdir d4/d6/d11/d25/d44/d7f/da5 0 2024-04-22T23:45:45.695 INFO:tasks.workunit.client.0.smithi031.stdout:5/529: dread db/d16/d2a/d59/da8/fac [0,4194304] 0 2024-04-22T23:45:45.701 INFO:tasks.workunit.client.0.smithi031.stdout:3/486: rmdir d4/d6/d4c 39 2024-04-22T23:45:45.703 INFO:tasks.workunit.client.0.smithi031.stdout:6/610: dwrite d3/da4/fcc [0,4194304] 0 2024-04-22T23:45:45.704 INFO:tasks.workunit.client.0.smithi031.stdout:9/541: mknod d2/dd/d75/d4a/da0/cad 0 2024-04-22T23:45:45.705 INFO:tasks.workunit.client.0.smithi031.stdout:1/549: dwrite d1/db/d16/d1f/f26 [0,4194304] 0 2024-04-22T23:45:45.707 INFO:tasks.workunit.client.0.smithi031.stdout:9/542: chown d2/dd/d75/d64/d5d/d59/d8e/f91 15 1 2024-04-22T23:45:45.712 INFO:tasks.workunit.client.0.smithi031.stdout:4/687: truncate db/d14/d1d/d7b/d7f/d2a/d30/f60 1716835 0 2024-04-22T23:45:45.718 INFO:tasks.workunit.client.0.smithi031.stdout:4/688: dwrite db/d14/d1d/fdc [0,4194304] 0 2024-04-22T23:45:45.722 INFO:tasks.workunit.client.0.smithi031.stdout:2/564: rename d6/d45/c81 to d6/dc/d42/d24/d32/cba 0 2024-04-22T23:45:45.726 INFO:tasks.workunit.client.0.smithi031.stdout:6/611: readlink d3/l3b 0 2024-04-22T23:45:45.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/565: dwrite d6/dc/d57/fa2 [0,4194304] 0 2024-04-22T23:45:45.734 INFO:tasks.workunit.client.0.smithi031.stdout:4/689: dread db/d31/d40/d62/f77 [0,4194304] 0 2024-04-22T23:45:45.734 INFO:tasks.workunit.client.0.smithi031.stdout:5/530: write db/d16/d2a/d59/da8/fac [351348,35586] 0 2024-04-22T23:45:45.740 INFO:tasks.workunit.client.0.smithi031.stdout:6/612: unlink d3/d2a/d3c/f43 0 2024-04-22T23:45:45.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/550: symlink d1/db/d16/d2f/dc1/lc8 0 2024-04-22T23:45:45.747 INFO:tasks.workunit.client.0.smithi031.stdout:8/473: rename d5/l69 to d5/da/d12/d2e/d4d/l89 0 2024-04-22T23:45:45.748 INFO:tasks.workunit.client.0.smithi031.stdout:7/555: dwrite d9/d15/d6d/d65/f86 [0,4194304] 0 2024-04-22T23:45:45.749 INFO:tasks.workunit.client.0.smithi031.stdout:7/556: fsync d9/d15/f99 0 2024-04-22T23:45:45.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/531: readlink db/d16/d2a/d36/d74/la9 0 2024-04-22T23:45:45.751 INFO:tasks.workunit.client.0.smithi031.stdout:6/613: mknod d3/d35/cdf 0 2024-04-22T23:45:45.752 INFO:tasks.workunit.client.0.smithi031.stdout:5/532: stat db/d16/d1d/d24/d27/l6e 0 2024-04-22T23:45:45.754 INFO:tasks.workunit.client.0.smithi031.stdout:5/533: fsync db/d16/d1d/d24/d27/d9e/fa5 0 2024-04-22T23:45:45.755 INFO:tasks.workunit.client.0.smithi031.stdout:9/543: dwrite d2/dd/d75/d4a/f61 [0,4194304] 0 2024-04-22T23:45:45.755 INFO:tasks.workunit.client.0.smithi031.stdout:1/551: mknod d1/d21/d4b/cc9 0 2024-04-22T23:45:45.756 INFO:tasks.workunit.client.0.smithi031.stdout:1/552: readlink d1/d3f/d90/l89 0 2024-04-22T23:45:45.763 INFO:tasks.workunit.client.0.smithi031.stdout:3/487: rename d4/d6/d11/c17 to d4/d54/d56/ca6 0 2024-04-22T23:45:45.769 INFO:tasks.workunit.client.0.smithi031.stdout:4/690: dread db/d14/d2d/d49/f99 [0,4194304] 0 2024-04-22T23:45:45.770 INFO:tasks.workunit.client.0.smithi031.stdout:7/557: symlink d9/d14/d22/d28/d7a/lc5 0 2024-04-22T23:45:45.772 INFO:tasks.workunit.client.0.smithi031.stdout:6/614: write d3/d1f/f4f [353509,66575] 0 2024-04-22T23:45:45.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/566: dread d6/f1c [0,4194304] 0 2024-04-22T23:45:45.777 INFO:tasks.workunit.client.0.smithi031.stdout:8/474: write d5/da/d12/d37/d34/d6f/f4b [3915707,58701] 0 2024-04-22T23:45:45.779 INFO:tasks.workunit.client.0.smithi031.stdout:4/691: creat db/d14/d2d/d49/d59/d6a/fee x:0 0 0 2024-04-22T23:45:45.783 INFO:tasks.workunit.client.0.smithi031.stdout:6/615: creat d3/d77/d8d/fe0 x:0 0 0 2024-04-22T23:45:45.786 INFO:tasks.workunit.client.0.smithi031.stdout:9/544: rename d2/dd/d65/l92 to d2/dc/d8b/d3d/lae 0 2024-04-22T23:45:45.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/534: dwrite db/d16/d1d/f56 [0,4194304] 0 2024-04-22T23:45:45.790 INFO:tasks.workunit.client.0.smithi031.stdout:8/475: creat d5/da/d12/d37/d34/d6f/d5b/f8a x:0 0 0 2024-04-22T23:45:45.791 INFO:tasks.workunit.client.0.smithi031.stdout:4/692: symlink db/d31/d40/d62/lef 0 2024-04-22T23:45:45.792 INFO:tasks.workunit.client.0.smithi031.stdout:4/693: write db/d14/d1d/d7b/d7f/d2a/d47/d80/fcf [549446,15386] 0 2024-04-22T23:45:45.795 INFO:tasks.workunit.client.0.smithi031.stdout:9/545: mkdir d2/d15/daf 0 2024-04-22T23:45:45.798 INFO:tasks.workunit.client.0.smithi031.stdout:5/535: rename db/d16/d2a/d36/d74/d82/d86/f9a to db/d16/d1d/d24/d27/fb0 0 2024-04-22T23:45:45.800 INFO:tasks.workunit.client.0.smithi031.stdout:4/694: dwrite db/d14/d2d/d49/d59/d6a/d63/fe0 [0,4194304] 0 2024-04-22T23:45:45.803 INFO:tasks.workunit.client.0.smithi031.stdout:7/558: mkdir d9/d14/d1f/dc6 0 2024-04-22T23:45:45.803 INFO:tasks.workunit.client.0.smithi031.stdout:6/616: mknod d3/d2a/d85/d96/dcb/ce1 0 2024-04-22T23:45:45.804 INFO:tasks.workunit.client.0.smithi031.stdout:7/559: dread - d9/d14/d22/d5c/fa7 zero size 2024-04-22T23:45:45.805 INFO:tasks.workunit.client.0.smithi031.stdout:1/553: getdents d1/db/d16/da0 0 2024-04-22T23:45:45.806 INFO:tasks.workunit.client.0.smithi031.stdout:4/695: dread db/d14/fad [0,4194304] 0 2024-04-22T23:45:45.811 INFO:tasks.workunit.client.0.smithi031.stdout:9/546: rename d2/dd/d75/d64/d5d/f77 to d2/dd/d75/d64/d5d/d59/fb0 0 2024-04-22T23:45:45.812 INFO:tasks.workunit.client.0.smithi031.stdout:9/547: chown d2/dc/d6d/fa6 37675 1 2024-04-22T23:45:45.818 INFO:tasks.workunit.client.0.smithi031.stdout:6/617: mkdir d3/d1f/d4c/de2 0 2024-04-22T23:45:45.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/536: dread db/d16/f54 [0,4194304] 0 2024-04-22T23:45:45.819 INFO:tasks.workunit.client.0.smithi031.stdout:7/560: fdatasync d9/d15/d3a/d41/f5a 0 2024-04-22T23:45:45.820 INFO:tasks.workunit.client.0.smithi031.stdout:5/537: chown db/d16/d1d/d24/d58/f85 287232603 1 2024-04-22T23:45:45.820 INFO:tasks.workunit.client.0.smithi031.stdout:5/538: chown db/d16/d2a/d7c/fa7 0 1 2024-04-22T23:45:45.823 INFO:tasks.workunit.client.0.smithi031.stdout:2/567: getdents d6/dc/d57 0 2024-04-22T23:45:45.827 INFO:tasks.workunit.client.0.smithi031.stdout:4/696: creat db/d14/d1d/d7b/d7f/ff0 x:0 0 0 2024-04-22T23:45:45.829 INFO:tasks.workunit.client.0.smithi031.stdout:8/476: truncate d5/da/df/f79 717152 0 2024-04-22T23:45:45.829 INFO:tasks.workunit.client.0.smithi031.stdout:4/697: chown db/d31/d40/d43/c71 14 1 2024-04-22T23:45:45.829 INFO:tasks.workunit.client.0.smithi031.stdout:4/698: fsync db/d14/d1d/d7b/d7f/d2a/d30/f42 0 2024-04-22T23:45:45.830 INFO:tasks.workunit.client.0.smithi031.stdout:4/699: fdatasync db/d14/d2d/d49/d59/d6a/fbb 0 2024-04-22T23:45:45.833 INFO:tasks.workunit.client.0.smithi031.stdout:9/548: mkdir d2/dc/d8b/d3d/d40/da4/db1 0 2024-04-22T23:45:45.837 INFO:tasks.workunit.client.0.smithi031.stdout:6/618: stat d3/d1f/l3e 0 2024-04-22T23:45:45.838 INFO:tasks.workunit.client.0.smithi031.stdout:6/619: readlink d3/d35/d75/d7d/l7f 0 2024-04-22T23:45:45.840 INFO:tasks.workunit.client.0.smithi031.stdout:7/561: readlink d9/d14/d1f/d45/da6/daa/lb4 0 2024-04-22T23:45:45.841 INFO:tasks.workunit.client.0.smithi031.stdout:5/539: rmdir db/d16/d2a 39 2024-04-22T23:45:45.842 INFO:tasks.workunit.client.0.smithi031.stdout:5/540: dread - db/d16/d1d/d24/d32/f73 zero size 2024-04-22T23:45:45.845 INFO:tasks.workunit.client.0.smithi031.stdout:1/554: unlink d1/db/d16/d2f/d81/lac 0 2024-04-22T23:45:45.845 INFO:tasks.workunit.client.0.smithi031.stdout:1/555: chown d1/d21/d4b/dbc 23273 1 2024-04-22T23:45:45.849 INFO:tasks.workunit.client.0.smithi031.stdout:9/549: chown d2/l4 22 1 2024-04-22T23:45:45.851 INFO:tasks.workunit.client.0.smithi031.stdout:0/515: sync 2024-04-22T23:45:45.851 INFO:tasks.workunit.client.0.smithi031.stdout:3/488: sync 2024-04-22T23:45:45.853 INFO:tasks.workunit.client.0.smithi031.stdout:0/516: rename d0/d1/d2 to d0/d1/d2/d36/d11/d1a/db6 22 2024-04-22T23:45:45.854 INFO:tasks.workunit.client.0.smithi031.stdout:5/541: chown db/d16/d2a/d36/d74/d82 283695858 1 2024-04-22T23:45:45.858 INFO:tasks.workunit.client.0.smithi031.stdout:5/542: chown db/c49 0 1 2024-04-22T23:45:45.858 INFO:tasks.workunit.client.0.smithi031.stdout:2/568: link d6/dc/d17/d69/f97 d6/dc/d49/d83/fbb 0 2024-04-22T23:45:45.859 INFO:tasks.workunit.client.0.smithi031.stdout:1/556: mkdir d1/db/dca 0 2024-04-22T23:45:45.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/700: mknod db/d14/d2d/d49/d59/d83/d9c/d72/dc6/cf1 0 2024-04-22T23:45:45.859 INFO:tasks.workunit.client.0.smithi031.stdout:8/477: dread d5/da/d12/d37/d34/f4c [0,4194304] 0 2024-04-22T23:45:45.862 INFO:tasks.workunit.client.0.smithi031.stdout:7/562: creat d9/d14/d1f/d7f/fc7 x:0 0 0 2024-04-22T23:45:45.866 INFO:tasks.workunit.client.0.smithi031.stdout:5/543: dwrite db/d16/d2a/d36/d74/f87 [0,4194304] 0 2024-04-22T23:45:45.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/569: dwrite d6/dc/d9a/d6c/f76 [0,4194304] 0 2024-04-22T23:45:45.869 INFO:tasks.workunit.client.0.smithi031.stdout:5/544: chown db/d16/d2a/d59/d90/l7a 9270 1 2024-04-22T23:45:45.869 INFO:tasks.workunit.client.0.smithi031.stdout:9/550: mkdir d2/dc/d8b/d3d/d40/da4/db1/db2 0 2024-04-22T23:45:45.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/489: mknod d4/d54/d88/da2/ca7 0 2024-04-22T23:45:45.873 INFO:tasks.workunit.client.0.smithi031.stdout:2/570: dread d6/dc/d9a/d6c/f76 [0,4194304] 0 2024-04-22T23:45:45.884 INFO:tasks.workunit.client.0.smithi031.stdout:6/620: write d3/f25 [477417,58426] 0 2024-04-22T23:45:45.884 INFO:tasks.workunit.client.0.smithi031.stdout:6/621: readlink d3/d1f/l1e 0 2024-04-22T23:45:45.888 INFO:tasks.workunit.client.0.smithi031.stdout:4/701: truncate db/d14/d1d/f76 3271805 0 2024-04-22T23:45:45.889 INFO:tasks.workunit.client.0.smithi031.stdout:4/702: chown db/d14/d2d/d49/d59/d8a/d89 3570 1 2024-04-22T23:45:45.892 INFO:tasks.workunit.client.0.smithi031.stdout:5/545: mkdir db/d16/d1d/d24/d58/db1 0 2024-04-22T23:45:45.894 INFO:tasks.workunit.client.0.smithi031.stdout:9/551: dread - d2/dd/d75/d64/f5f zero size 2024-04-22T23:45:45.901 INFO:tasks.workunit.client.0.smithi031.stdout:1/557: creat d1/db/d16/fcb x:0 0 0 2024-04-22T23:45:45.901 INFO:tasks.workunit.client.0.smithi031.stdout:9/552: chown d2/dd/d75/d4a 34248 1 2024-04-22T23:45:45.901 INFO:tasks.workunit.client.0.smithi031.stdout:4/703: dread db/d14/f45 [0,4194304] 0 2024-04-22T23:45:45.903 INFO:tasks.workunit.client.0.smithi031.stdout:6/622: rename d3/f34 to d3/d35/d75/fe3 0 2024-04-22T23:45:45.904 INFO:tasks.workunit.client.0.smithi031.stdout:3/490: dread d4/d54/d59/f6e [0,4194304] 0 2024-04-22T23:45:45.905 INFO:tasks.workunit.client.0.smithi031.stdout:4/704: fsync db/d14/d1d/fd7 0 2024-04-22T23:45:45.905 INFO:tasks.workunit.client.0.smithi031.stdout:8/478: write d5/f8 [759704,83686] 0 2024-04-22T23:45:45.907 INFO:tasks.workunit.client.0.smithi031.stdout:9/553: dread - d2/dd/d75/d64/d5d/d59/f78 zero size 2024-04-22T23:45:45.907 INFO:tasks.workunit.client.0.smithi031.stdout:5/546: dwrite db/f25 [0,4194304] 0 2024-04-22T23:45:45.908 INFO:tasks.workunit.client.0.smithi031.stdout:6/623: write d3/d35/d86/dae/fd8 [351552,42166] 0 2024-04-22T23:45:45.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/554: chown d2/dd/l7a 194112697 1 2024-04-22T23:45:45.914 INFO:tasks.workunit.client.0.smithi031.stdout:4/705: dwrite db/d31/d4f/f78 [0,4194304] 0 2024-04-22T23:45:45.919 INFO:tasks.workunit.client.0.smithi031.stdout:7/563: truncate d9/fa 7662178 0 2024-04-22T23:45:45.920 INFO:tasks.workunit.client.0.smithi031.stdout:7/564: chown d9/d14/l92 32 1 2024-04-22T23:45:45.931 INFO:tasks.workunit.client.0.smithi031.stdout:2/571: symlink d6/dc/d42/d24/d9c/lbc 0 2024-04-22T23:45:45.932 INFO:tasks.workunit.client.0.smithi031.stdout:0/517: getdents d0/d1/d2/d36/d11/d1a/d3c 0 2024-04-22T23:45:45.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/491: chown d4/d54/d56/ca6 1561 1 2024-04-22T23:45:45.940 INFO:tasks.workunit.client.0.smithi031.stdout:3/492: truncate d4/d6/d11/d25/d44/d7f/f69 4649430 0 2024-04-22T23:45:45.945 INFO:tasks.workunit.client.0.smithi031.stdout:7/565: rename d9/d14/f5e to d9/d15/d6d/fc8 0 2024-04-22T23:45:45.946 INFO:tasks.workunit.client.0.smithi031.stdout:2/572: fsync d6/dc/d48/d4b/fb0 0 2024-04-22T23:45:45.956 INFO:tasks.workunit.client.0.smithi031.stdout:3/493: dwrite d4/d54/d59/f6e [0,4194304] 0 2024-04-22T23:45:45.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/566: chown d9/d14/d1f/l8b 0 1 2024-04-22T23:45:45.968 INFO:tasks.workunit.client.0.smithi031.stdout:3/494: creat d4/d54/d2c/fa8 x:0 0 0 2024-04-22T23:45:45.971 INFO:tasks.workunit.client.0.smithi031.stdout:2/573: mknod d6/dc/cbd 0 2024-04-22T23:45:45.974 INFO:tasks.workunit.client.0.smithi031.stdout:1/558: link d1/de/d1e/d34/d41/c32 d1/db3/ccc 0 2024-04-22T23:45:45.974 INFO:tasks.workunit.client.0.smithi031.stdout:7/567: dwrite d9/ff [0,4194304] 0 2024-04-22T23:45:45.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/559: chown d1/de/d1e/d34/d41/f42 3052 1 2024-04-22T23:45:45.977 INFO:tasks.workunit.client.0.smithi031.stdout:6/624: getdents d3 0 2024-04-22T23:45:45.978 INFO:tasks.workunit.client.0.smithi031.stdout:3/495: creat d4/d6/d11/d25/d44/d7f/d5f/fa9 x:0 0 0 2024-04-22T23:45:45.978 INFO:tasks.workunit.client.0.smithi031.stdout:6/625: dread - d3/d2a/d85/d96/fbf zero size 2024-04-22T23:45:45.980 INFO:tasks.workunit.client.0.smithi031.stdout:6/626: truncate d3/d2a/d3c/d40/d5b/fc5 557862 0 2024-04-22T23:45:45.980 INFO:tasks.workunit.client.0.smithi031.stdout:2/574: symlink d6/dc/d48/d4b/lbe 0 2024-04-22T23:45:45.983 INFO:tasks.workunit.client.0.smithi031.stdout:0/518: getdents d0/d13/d18/d1e/d25 0 2024-04-22T23:45:45.984 INFO:tasks.workunit.client.0.smithi031.stdout:0/519: fdatasync d0/d13/d18/d1e/d25/d8e/da7/d2b/f64 0 2024-04-22T23:45:45.984 INFO:tasks.workunit.client.0.smithi031.stdout:4/706: write db/d14/d1d/d7b/d7f/d2a/f79 [3080244,105810] 0 2024-04-22T23:45:45.986 INFO:tasks.workunit.client.0.smithi031.stdout:0/520: write d0/d1/d2/d36/fb1 [650648,12896] 0 2024-04-22T23:45:45.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/560: creat d1/d21/d4e/fcd x:0 0 0 2024-04-22T23:45:45.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/561: dread - d1/db/d16/d2f/fc0 zero size 2024-04-22T23:45:45.989 INFO:tasks.workunit.client.0.smithi031.stdout:3/496: truncate d4/d54/d2c/d89/d2e/d45/f60 494346 0 2024-04-22T23:45:45.992 INFO:tasks.workunit.client.0.smithi031.stdout:6/627: write d3/f1a [4619217,122517] 0 2024-04-22T23:45:45.993 INFO:tasks.workunit.client.0.smithi031.stdout:2/575: symlink d6/dc/d9a/d6c/lbf 0 2024-04-22T23:45:45.994 INFO:tasks.workunit.client.0.smithi031.stdout:5/547: truncate db/d16/d2a/d59/d90/f7b 3181363 0 2024-04-22T23:45:45.994 INFO:tasks.workunit.client.0.smithi031.stdout:0/521: dwrite d0/d13/d18/d1e/f6a [4194304,4194304] 0 2024-04-22T23:45:45.996 INFO:tasks.workunit.client.0.smithi031.stdout:5/548: readlink db/d16/l28 0 2024-04-22T23:45:46.003 INFO:tasks.workunit.client.0.smithi031.stdout:3/497: creat d4/d54/d6d/faa x:0 0 0 2024-04-22T23:45:46.004 INFO:tasks.workunit.client.0.smithi031.stdout:0/522: dwrite d0/d13/d18/d1e/f6f [0,4194304] 0 2024-04-22T23:45:46.006 INFO:tasks.workunit.client.0.smithi031.stdout:6/628: creat d3/d77/fe4 x:0 0 0 2024-04-22T23:45:46.007 INFO:tasks.workunit.client.0.smithi031.stdout:3/498: write d4/d54/d2c/d89/d2e/d45/f6c [2358923,114158] 0 2024-04-22T23:45:46.008 INFO:tasks.workunit.client.0.smithi031.stdout:0/523: chown d0/d1/d2/d36/d11/f14 1 1 2024-04-22T23:45:46.009 INFO:tasks.workunit.client.0.smithi031.stdout:4/707: dread db/d31/f33 [0,4194304] 0 2024-04-22T23:45:46.009 INFO:tasks.workunit.client.0.smithi031.stdout:0/524: read d0/d13/f67 [928053,43432] 0 2024-04-22T23:45:46.010 INFO:tasks.workunit.client.0.smithi031.stdout:1/562: dread d1/de/d1e/f64 [0,4194304] 0 2024-04-22T23:45:46.011 INFO:tasks.workunit.client.0.smithi031.stdout:1/563: chown d1/d35/f7b 22 1 2024-04-22T23:45:46.017 INFO:tasks.workunit.client.0.smithi031.stdout:5/549: unlink db/d16/d1d/f56 0 2024-04-22T23:45:46.025 INFO:tasks.workunit.client.0.smithi031.stdout:3/499: stat d4/d54/d56/c9e 0 2024-04-22T23:45:46.025 INFO:tasks.workunit.client.0.smithi031.stdout:3/500: stat d4/d6/d11/d25/f36 0 2024-04-22T23:45:46.033 INFO:tasks.workunit.client.0.smithi031.stdout:1/564: write d1/d35/f7b [1026454,40520] 0 2024-04-22T23:45:46.033 INFO:tasks.workunit.client.0.smithi031.stdout:1/565: chown d1/db/d16/d2f 207 1 2024-04-22T23:45:46.035 INFO:tasks.workunit.client.0.smithi031.stdout:7/568: getdents d9/d14/d1f/d45/da6/daa 0 2024-04-22T23:45:46.036 INFO:tasks.workunit.client.0.smithi031.stdout:7/569: write d9/d14/d1f/d7f/fc7 [834038,86698] 0 2024-04-22T23:45:46.042 INFO:tasks.workunit.client.0.smithi031.stdout:5/550: rename db/l40 to db/d16/d1d/d24/d32/d41/lb2 0 2024-04-22T23:45:46.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/576: chown d6/dc/d48/d4b/l6e 130131 1 2024-04-22T23:45:46.046 INFO:tasks.workunit.client.0.smithi031.stdout:2/577: write d6/dc/d9a/d6c/f7e [389430,53798] 0 2024-04-22T23:45:46.046 INFO:tasks.workunit.client.0.smithi031.stdout:3/501: creat d4/d83/fab x:0 0 0 2024-04-22T23:45:46.051 INFO:tasks.workunit.client.0.smithi031.stdout:7/570: mkdir d9/d14/dc9 0 2024-04-22T23:45:46.055 INFO:tasks.workunit.client.0.smithi031.stdout:2/578: dwrite d6/dc/d17/f3c [0,4194304] 0 2024-04-22T23:45:46.055 INFO:tasks.workunit.client.0.smithi031.stdout:3/502: dwrite d4/d83/f8b [0,4194304] 0 2024-04-22T23:45:46.056 INFO:tasks.workunit.client.0.smithi031.stdout:2/579: dread - d6/dc/d9a/d6c/d86/f9b zero size 2024-04-22T23:45:46.057 INFO:tasks.workunit.client.0.smithi031.stdout:2/580: chown d6/dc/d57/d52 536 1 2024-04-22T23:45:46.063 INFO:tasks.workunit.client.0.smithi031.stdout:7/571: mknod d9/d14/d1f/d45/da6/daa/db0/cca 0 2024-04-22T23:45:46.064 INFO:tasks.workunit.client.0.smithi031.stdout:7/572: chown d9/d15/c73 172 1 2024-04-22T23:45:46.064 INFO:tasks.workunit.client.0.smithi031.stdout:3/503: dwrite d4/d6/d11/d25/d44/d7f/f69 [0,4194304] 0 2024-04-22T23:45:46.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/566: link d1/d35/d36/c59 d1/de/d1e/d34/d41/cce 0 2024-04-22T23:45:46.072 INFO:tasks.workunit.client.0.smithi031.stdout:4/708: dwrite db/fc [0,4194304] 0 2024-04-22T23:45:46.090 INFO:tasks.workunit.client.0.smithi031.stdout:2/581: rmdir d6/dc/d42/d24/d32/d79 0 2024-04-22T23:45:46.095 INFO:tasks.workunit.client.0.smithi031.stdout:7/573: dread d9/d15/d6d/d65/f6b [4194304,4194304] 0 2024-04-22T23:45:46.097 INFO:tasks.workunit.client.0.smithi031.stdout:1/567: rename d1/d21/d69 to d1/de/d1e/dcf 0 2024-04-22T23:45:46.099 INFO:tasks.workunit.client.0.smithi031.stdout:6/629: getdents d3 0 2024-04-22T23:45:46.100 INFO:tasks.workunit.client.0.smithi031.stdout:6/630: chown d3/d1f/d60/cc8 25364750 1 2024-04-22T23:45:46.100 INFO:tasks.workunit.client.0.smithi031.stdout:6/631: chown d3/d2a/d3c 7 1 2024-04-22T23:45:46.101 INFO:tasks.workunit.client.0.smithi031.stdout:6/632: dread - d3/da4/fb9 zero size 2024-04-22T23:45:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:2/582: rename d6/dc/d42/f88 to d6/dc/d57/d52/db5/fc0 0 2024-04-22T23:45:46.104 INFO:tasks.workunit.client.0.smithi031.stdout:2/583: fsync d6/dc/d49/d83/d85/fa1 0 2024-04-22T23:45:46.104 INFO:tasks.workunit.client.0.smithi031.stdout:2/584: readlink l5 0 2024-04-22T23:45:46.105 INFO:tasks.workunit.client.0.smithi031.stdout:5/551: getdents db/d16/d2a/d7c/d89 0 2024-04-22T23:45:46.106 INFO:tasks.workunit.client.0.smithi031.stdout:5/552: chown db/d16/d2a/d7c/fa7 9 1 2024-04-22T23:45:46.106 INFO:tasks.workunit.client.0.smithi031.stdout:4/709: write db/d14/d1d/f76 [2264328,65400] 0 2024-04-22T23:45:46.109 INFO:tasks.workunit.client.0.smithi031.stdout:4/710: chown db/d14/d2d/d49/d59/d6a/d63/d86/ce3 4 1 2024-04-22T23:45:46.111 INFO:tasks.workunit.client.0.smithi031.stdout:2/585: dwrite d6/dc/d42/d28/f54 [0,4194304] 0 2024-04-22T23:45:46.111 INFO:tasks.workunit.client.0.smithi031.stdout:7/574: rename d9/d14/d22/d28/d7a/la5 to d9/d14/lcb 0 2024-04-22T23:45:46.114 INFO:tasks.workunit.client.0.smithi031.stdout:7/575: dread d9/d14/d22/d28/d7a/f95 [0,4194304] 0 2024-04-22T23:45:46.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/633: dread d3/d2a/d3c/d40/d4d/f61 [0,4194304] 0 2024-04-22T23:45:46.116 INFO:tasks.workunit.client.0.smithi031.stdout:6/634: dread - d3/d1f/d4c/d8e/fdd zero size 2024-04-22T23:45:46.119 INFO:tasks.workunit.client.0.smithi031.stdout:4/711: mkdir db/d14/d2d/d49/d59/d6a/d63/d86/df2 0 2024-04-22T23:45:46.123 INFO:tasks.workunit.client.0.smithi031.stdout:2/586: truncate d6/dc/d42/d28/f37 86920 0 2024-04-22T23:45:46.125 INFO:tasks.workunit.client.0.smithi031.stdout:6/635: chown d3/c23 3215 1 2024-04-22T23:45:46.127 INFO:tasks.workunit.client.0.smithi031.stdout:5/553: symlink db/d8d/d9b/lb3 0 2024-04-22T23:45:46.128 INFO:tasks.workunit.client.0.smithi031.stdout:2/587: mkdir d6/dc/d48/dc1 0 2024-04-22T23:45:46.133 INFO:tasks.workunit.client.0.smithi031.stdout:2/588: mkdir d6/dc/d49/d83/dc2 0 2024-04-22T23:45:46.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/636: rename d3/d1f/f20 to d3/d35/d75/d7d/fe5 0 2024-04-22T23:45:46.135 INFO:tasks.workunit.client.0.smithi031.stdout:4/712: getdents db/d14/d1d/d7b/d7f/d2a/d30 0 2024-04-22T23:45:46.136 INFO:tasks.workunit.client.0.smithi031.stdout:4/713: chown db/d31/d40/d43/f51 49118381 1 2024-04-22T23:45:46.140 INFO:tasks.workunit.client.0.smithi031.stdout:7/576: dread f3 [0,4194304] 0 2024-04-22T23:45:46.140 INFO:tasks.workunit.client.0.smithi031.stdout:2/589: rename d6/dc/d42/d24/l53 to d6/dc/d9a/lc3 0 2024-04-22T23:45:46.142 INFO:tasks.workunit.client.0.smithi031.stdout:4/714: read db/d14/d2d/db7/fb8 [3659830,113783] 0 2024-04-22T23:45:46.146 INFO:tasks.workunit.client.0.smithi031.stdout:4/715: dread db/d14/d1d/d7b/d7f/d2a/fb1 [0,4194304] 0 2024-04-22T23:45:46.148 INFO:tasks.workunit.client.0.smithi031.stdout:2/590: rename d6/dc/d17/f25 to d6/dc/d42/d24/d32/fc4 0 2024-04-22T23:45:46.148 INFO:tasks.workunit.client.0.smithi031.stdout:5/554: link db/d16/d2a/d59/d90/f7b db/d16/d2a/d7c/d89/fb4 0 2024-04-22T23:45:46.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/716: link f9 db/d14/ff3 0 2024-04-22T23:45:46.154 INFO:tasks.workunit.client.0.smithi031.stdout:4/717: dread - db/d14/d2d/d49/dcd/fd3 zero size 2024-04-22T23:45:46.154 INFO:tasks.workunit.client.0.smithi031.stdout:5/555: creat db/d16/d1d/d24/d58/db1/fb5 x:0 0 0 2024-04-22T23:45:46.155 INFO:tasks.workunit.client.0.smithi031.stdout:4/718: write db/d14/d1d/fd7 [620396,6707] 0 2024-04-22T23:45:46.161 INFO:tasks.workunit.client.0.smithi031.stdout:4/719: symlink db/d14/d2d/d49/d59/d83/d9c/d72/dc6/dd2/lf4 0 2024-04-22T23:45:46.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/577: dwrite d9/d14/d22/d28/f36 [0,4194304] 0 2024-04-22T23:45:46.167 INFO:tasks.workunit.client.0.smithi031.stdout:4/720: dread db/d14/d2d/d49/fb3 [0,4194304] 0 2024-04-22T23:45:46.172 INFO:tasks.workunit.client.0.smithi031.stdout:4/721: mknod db/d31/d40/d62/cf5 0 2024-04-22T23:45:46.172 INFO:tasks.workunit.client.0.smithi031.stdout:5/556: dwrite db/f29 [0,4194304] 0 2024-04-22T23:45:46.177 INFO:tasks.workunit.client.0.smithi031.stdout:4/722: mknod db/d14/d2d/d49/d59/d6a/d63/cf6 0 2024-04-22T23:45:46.177 INFO:tasks.workunit.client.0.smithi031.stdout:4/723: chown db/d14/d2d/d49 646225 1 2024-04-22T23:45:46.178 INFO:tasks.workunit.client.0.smithi031.stdout:5/557: read db/d16/d1d/d24/d27/fb0 [1301892,101856] 0 2024-04-22T23:45:46.181 INFO:tasks.workunit.client.0.smithi031.stdout:5/558: mknod db/d16/d1d/d24/d32/d41/cb6 0 2024-04-22T23:45:46.186 INFO:tasks.workunit.client.0.smithi031.stdout:4/724: dwrite db/d14/d1d/d7b/d7f/d2a/d30/f42 [0,4194304] 0 2024-04-22T23:45:46.191 INFO:tasks.workunit.client.0.smithi031.stdout:4/725: creat db/d14/d2d/d49/d59/d6a/d63/d8b/ff7 x:0 0 0 2024-04-22T23:45:46.194 INFO:tasks.workunit.client.0.smithi031.stdout:5/559: write db/d16/d1d/f93 [1335667,91595] 0 2024-04-22T23:45:46.196 INFO:tasks.workunit.client.0.smithi031.stdout:4/726: mknod db/d14/d2d/d49/d59/d83/d9c/d72/dde/cf8 0 2024-04-22T23:45:46.198 INFO:tasks.workunit.client.0.smithi031.stdout:5/560: rename db/d16/d1d/d24/d58/c4c to db/d8d/d9b/cb7 0 2024-04-22T23:45:46.199 INFO:tasks.workunit.client.0.smithi031.stdout:4/727: unlink db/d14/d1d/d7b/d7f/d2a/d30/f5e 0 2024-04-22T23:45:46.204 INFO:tasks.workunit.client.0.smithi031.stdout:4/728: dread db/d14/d1d/d7b/d7f/d2a/d30/f42 [0,4194304] 0 2024-04-22T23:45:46.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/729: write db/d14/d2d/d49/d59/d83/d9c/d72/dc6/dd2/fd9 [977400,21898] 0 2024-04-22T23:45:46.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/730: write db/d14/d2d/d49/d59/d6a/fee [579217,130946] 0 2024-04-22T23:45:46.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/731: mknod db/d31/d40/cf9 0 2024-04-22T23:45:46.217 INFO:tasks.workunit.client.0.smithi031.stdout:5/561: dread db/d16/d2a/f7f [0,4194304] 0 2024-04-22T23:45:46.219 INFO:tasks.workunit.client.0.smithi031.stdout:4/732: mkdir db/d31/d40/d43/dfa 0 2024-04-22T23:45:46.221 INFO:tasks.workunit.client.0.smithi031.stdout:5/562: mkdir db/d16/d2a/d36/d74/d82/d86/db8 0 2024-04-22T23:45:46.224 INFO:tasks.workunit.client.0.smithi031.stdout:4/733: creat db/d31/d40/d43/dfa/ffb x:0 0 0 2024-04-22T23:45:46.226 INFO:tasks.workunit.client.0.smithi031.stdout:5/563: dwrite db/d16/fa2 [0,4194304] 0 2024-04-22T23:45:46.227 INFO:tasks.workunit.client.0.smithi031.stdout:5/564: chown db/d16/d2a/d7c 18 1 2024-04-22T23:45:46.228 INFO:tasks.workunit.client.0.smithi031.stdout:4/734: mknod db/d14/d1d/d7b/cfc 0 2024-04-22T23:45:46.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/565: rmdir db/d16/d2a/d59 39 2024-04-22T23:45:46.231 INFO:tasks.workunit.client.0.smithi031.stdout:5/566: write db/d16/d1d/d24/d27/d9e/fae [815133,31815] 0 2024-04-22T23:45:46.237 INFO:tasks.workunit.client.0.smithi031.stdout:4/735: rename db/d31/c4a to db/d14/d1d/d7b/d7f/d2a/d47/cfd 0 2024-04-22T23:45:46.247 INFO:tasks.workunit.client.0.smithi031.stdout:5/567: dread db/d16/d1d/d24/d58/d5d/f77 [0,4194304] 0 2024-04-22T23:45:46.248 INFO:tasks.workunit.client.0.smithi031.stdout:4/736: write db/d14/d2d/d49/d59/d6a/d63/d86/f8c [3528650,98436] 0 2024-04-22T23:45:46.253 INFO:tasks.workunit.client.0.smithi031.stdout:4/737: link db/d14/d2d/d49/d59/d6a/d63/d86/f8c db/d14/d1d/dd8/ffe 0 2024-04-22T23:45:46.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/568: dwrite db/d16/d2a/f2c [0,4194304] 0 2024-04-22T23:45:46.255 INFO:tasks.workunit.client.0.smithi031.stdout:5/569: read db/d16/d2a/f7f [3869511,66823] 0 2024-04-22T23:45:46.266 INFO:tasks.workunit.client.0.smithi031.stdout:5/570: symlink db/d16/d2a/d59/d90/da4/lb9 0 2024-04-22T23:45:46.267 INFO:tasks.workunit.client.0.smithi031.stdout:5/571: fsync db/d16/d1d/d24/f2e 0 2024-04-22T23:45:46.267 INFO:tasks.workunit.client.0.smithi031.stdout:4/738: dwrite db/d14/d2d/d49/d59/d6a/d63/d86/f8c [0,4194304] 0 2024-04-22T23:45:46.269 INFO:tasks.workunit.client.0.smithi031.stdout:5/572: symlink db/d8d/lba 0 2024-04-22T23:45:46.272 INFO:tasks.workunit.client.0.smithi031.stdout:4/739: creat db/d14/d2d/d49/d59/fff x:0 0 0 2024-04-22T23:45:46.273 INFO:tasks.workunit.client.0.smithi031.stdout:5/573: mknod db/d16/d1d/d24/d32/cbb 0 2024-04-22T23:45:46.274 INFO:tasks.workunit.client.0.smithi031.stdout:5/574: dread - db/d16/d1d/d24/d58/d5d/fa0 zero size 2024-04-22T23:45:46.277 INFO:tasks.workunit.client.0.smithi031.stdout:5/575: creat db/d16/d2a/d36/d74/fbc x:0 0 0 2024-04-22T23:45:46.278 INFO:tasks.workunit.client.0.smithi031.stdout:5/576: chown db/d16/c5c 1467 1 2024-04-22T23:45:46.279 INFO:tasks.workunit.client.0.smithi031.stdout:4/740: dwrite db/d14/d1d/fd7 [0,4194304] 0 2024-04-22T23:45:46.281 INFO:tasks.workunit.client.0.smithi031.stdout:4/741: fdatasync db/d14/d1d/d7b/d7f/d2a/d47/fa3 0 2024-04-22T23:45:46.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/742: rmdir db/d14/d2d/d49/d59/d6a 39 2024-04-22T23:45:46.289 INFO:tasks.workunit.client.0.smithi031.stdout:4/743: read db/d14/f45 [385868,76127] 0 2024-04-22T23:45:46.293 INFO:tasks.workunit.client.0.smithi031.stdout:4/744: symlink db/d14/d2d/l100 0 2024-04-22T23:45:46.298 INFO:tasks.workunit.client.0.smithi031.stdout:4/745: dread db/d14/d1d/d7b/d7f/d2a/d30/f42 [0,4194304] 0 2024-04-22T23:45:46.298 INFO:tasks.workunit.client.0.smithi031.stdout:5/577: truncate db/d16/f18 1130977 0 2024-04-22T23:45:46.299 INFO:tasks.workunit.client.0.smithi031.stdout:4/746: chown db/d14/d1d/d7b/d7f/d2a/da1/da7/ce9 0 1 2024-04-22T23:45:46.300 INFO:tasks.workunit.client.0.smithi031.stdout:5/578: creat db/d16/d1d/d24/d58/d5d/fbd x:0 0 0 2024-04-22T23:45:46.303 INFO:tasks.workunit.client.0.smithi031.stdout:5/579: creat db/d16/d2a/d7c/fbe x:0 0 0 2024-04-22T23:45:46.305 INFO:tasks.workunit.client.0.smithi031.stdout:4/747: dwrite db/d14/d1d/fd7 [0,4194304] 0 2024-04-22T23:45:46.316 INFO:tasks.workunit.client.0.smithi031.stdout:5/580: getdents db/d16/d1d/d24/d58/db1 0 2024-04-22T23:45:46.318 INFO:tasks.workunit.client.0.smithi031.stdout:4/748: getdents db/d14/d1d/d7b 0 2024-04-22T23:45:46.318 INFO:tasks.workunit.client.0.smithi031.stdout:5/581: mknod db/d16/d2a/d59/cbf 0 2024-04-22T23:45:46.325 INFO:tasks.workunit.client.0.smithi031.stdout:5/582: dwrite db/d16/d1d/d24/d27/f5b [0,4194304] 0 2024-04-22T23:45:46.326 INFO:tasks.workunit.client.0.smithi031.stdout:5/583: write db/d16/f3e [4869279,105420] 0 2024-04-22T23:45:46.328 INFO:tasks.workunit.client.0.smithi031.stdout:5/584: chown db/d16/d2a/d7c/fbe 581669 1 2024-04-22T23:45:46.328 INFO:tasks.workunit.client.0.smithi031.stdout:5/585: chown db/d16/d2a/d36/f6f 327226846 1 2024-04-22T23:45:46.339 INFO:tasks.workunit.client.0.smithi031.stdout:8/479: sync 2024-04-22T23:45:46.339 INFO:tasks.workunit.client.0.smithi031.stdout:9/555: sync 2024-04-22T23:45:46.340 INFO:tasks.workunit.client.0.smithi031.stdout:5/586: truncate db/d16/d1d/f93 2062200 0 2024-04-22T23:45:46.347 INFO:tasks.workunit.client.0.smithi031.stdout:8/480: dwrite d5/da/d12/d37/d34/d6f/f6d [0,4194304] 0 2024-04-22T23:45:46.348 INFO:tasks.workunit.client.0.smithi031.stdout:9/556: write d2/d15/f54 [496925,1672] 0 2024-04-22T23:45:46.350 INFO:tasks.workunit.client.0.smithi031.stdout:5/587: dread f7 [0,4194304] 0 2024-04-22T23:45:46.351 INFO:tasks.workunit.client.0.smithi031.stdout:5/588: stat db/d16/d1d/d24/f76 0 2024-04-22T23:45:46.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/557: creat d2/dc/d8b/d3d/d40/da4/fb3 x:0 0 0 2024-04-22T23:45:46.356 INFO:tasks.workunit.client.0.smithi031.stdout:9/558: mknod d2/d15/daf/cb4 0 2024-04-22T23:45:46.359 INFO:tasks.workunit.client.0.smithi031.stdout:9/559: rename d2/dd/d75/d64/l3f to d2/d15/lb5 0 2024-04-22T23:45:46.361 INFO:tasks.workunit.client.0.smithi031.stdout:9/560: mknod d2/dc/d3e/da1/cb6 0 2024-04-22T23:45:46.362 INFO:tasks.workunit.client.0.smithi031.stdout:9/561: chown d2/d15/f9b 485290378 1 2024-04-22T23:45:46.363 INFO:tasks.workunit.client.0.smithi031.stdout:8/481: truncate d5/fb 1838531 0 2024-04-22T23:45:46.366 INFO:tasks.workunit.client.0.smithi031.stdout:9/562: dread d2/dc/d3e/f41 [4194304,4194304] 0 2024-04-22T23:45:46.369 INFO:tasks.workunit.client.0.smithi031.stdout:5/589: dread db/d16/d2a/d36/d74/d82/d86/f95 [0,4194304] 0 2024-04-22T23:45:46.372 INFO:tasks.workunit.client.0.smithi031.stdout:8/482: dwrite d5/da/d12/d37/d34/d6f/d5b/f8a [0,4194304] 0 2024-04-22T23:45:46.373 INFO:tasks.workunit.client.0.smithi031.stdout:5/590: read db/f42 [4243756,50650] 0 2024-04-22T23:45:46.377 INFO:tasks.workunit.client.0.smithi031.stdout:8/483: creat d5/da/d12/d1d/f8b x:0 0 0 2024-04-22T23:45:46.380 INFO:tasks.workunit.client.0.smithi031.stdout:5/591: dwrite db/d16/fa2 [0,4194304] 0 2024-04-22T23:45:46.382 INFO:tasks.workunit.client.0.smithi031.stdout:9/563: write d2/dc/d3e/f41 [3324551,113645] 0 2024-04-22T23:45:46.383 INFO:tasks.workunit.client.0.smithi031.stdout:8/484: creat d5/da/d83/f8c x:0 0 0 2024-04-22T23:45:46.390 INFO:tasks.workunit.client.0.smithi031.stdout:8/485: rename d5/da/d12/d2e/d30 to d5/da/d83/d8d 0 2024-04-22T23:45:46.390 INFO:tasks.workunit.client.0.smithi031.stdout:8/486: dread - d5/d48/d61/d6b/d6c/f75 zero size 2024-04-22T23:45:46.396 INFO:tasks.workunit.client.0.smithi031.stdout:9/564: creat d2/dc/fb7 x:0 0 0 2024-04-22T23:45:46.396 INFO:tasks.workunit.client.0.smithi031.stdout:8/487: dwrite d5/da/d12/d37/d34/d6f/f4b [0,4194304] 0 2024-04-22T23:45:46.405 INFO:tasks.workunit.client.0.smithi031.stdout:9/565: creat d2/fb8 x:0 0 0 2024-04-22T23:45:46.406 INFO:tasks.workunit.client.0.smithi031.stdout:5/592: truncate db/d16/d1d/f6a 3585629 0 2024-04-22T23:45:46.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/488: dwrite d5/da/d12/d2e/d4d/d56/f71 [0,4194304] 0 2024-04-22T23:45:46.407 INFO:tasks.workunit.client.0.smithi031.stdout:9/566: chown d2/dd/d65/l7b 65401486 1 2024-04-22T23:45:46.410 INFO:tasks.workunit.client.0.smithi031.stdout:0/525: sync 2024-04-22T23:45:46.410 INFO:tasks.workunit.client.0.smithi031.stdout:3/504: sync 2024-04-22T23:45:46.411 INFO:tasks.workunit.client.0.smithi031.stdout:5/593: symlink db/d16/d1d/d24/d32/lc0 0 2024-04-22T23:45:46.412 INFO:tasks.workunit.client.0.smithi031.stdout:8/489: symlink d5/da/d12/d37/d4e/l8e 0 2024-04-22T23:45:46.413 INFO:tasks.workunit.client.0.smithi031.stdout:9/567: mknod d2/dd/d75/d4a/da0/cb9 0 2024-04-22T23:45:46.416 INFO:tasks.workunit.client.0.smithi031.stdout:3/505: mknod d4/d83/cac 0 2024-04-22T23:45:46.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/490: dwrite d5/da/d12/d37/d34/d6f/d5b/f84 [0,4194304] 0 2024-04-22T23:45:46.427 INFO:tasks.workunit.client.0.smithi031.stdout:0/526: dwrite d0/d1/d2/f23 [0,4194304] 0 2024-04-22T23:45:46.436 INFO:tasks.workunit.client.0.smithi031.stdout:9/568: dread d2/f8 [0,4194304] 0 2024-04-22T23:45:46.439 INFO:tasks.workunit.client.0.smithi031.stdout:3/506: creat d4/d54/d2c/d89/fad x:0 0 0 2024-04-22T23:45:46.442 INFO:tasks.workunit.client.0.smithi031.stdout:9/569: creat d2/dd/d75/d4a/fba x:0 0 0 2024-04-22T23:45:46.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:46 smithi031 ceph-mon[26721]: pgmap v29: 65 pgs: 65 active+clean; 1.2 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 32 MiB/s rd, 162 MiB/s wr, 466 op/s 2024-04-22T23:45:46.446 INFO:tasks.workunit.client.0.smithi031.stdout:0/527: rename d0/d1/d2/d17/c78 to d0/d1/d2/cb7 0 2024-04-22T23:45:46.447 INFO:tasks.workunit.client.0.smithi031.stdout:9/570: symlink d2/dc/d8b/d3d/d40/da4/lbb 0 2024-04-22T23:45:46.449 INFO:tasks.workunit.client.0.smithi031.stdout:5/594: truncate f7 635870 0 2024-04-22T23:45:46.450 INFO:tasks.workunit.client.0.smithi031.stdout:3/507: write d4/d6/d4c/d8c/fa3 [458564,37453] 0 2024-04-22T23:45:46.451 INFO:tasks.workunit.client.0.smithi031.stdout:0/528: dread - d0/d1/d2/d36/d11/d1a/d56/fa4 zero size 2024-04-22T23:45:46.452 INFO:tasks.workunit.client.0.smithi031.stdout:0/529: truncate d0/d1/d3f/fb0 560319 0 2024-04-22T23:45:46.453 INFO:tasks.workunit.client.0.smithi031.stdout:9/571: symlink d2/dd/d75/d80/lbc 0 2024-04-22T23:45:46.454 INFO:tasks.workunit.client.0.smithi031.stdout:9/572: stat d2/d15/f86 0 2024-04-22T23:45:46.454 INFO:tasks.workunit.client.0.smithi031.stdout:5/595: dread db/d16/d1d/d24/d27/d9e/fae [0,4194304] 0 2024-04-22T23:45:46.454 INFO:tasks.workunit.client.0.smithi031.stdout:9/573: dread - d2/d15/f86 zero size 2024-04-22T23:45:46.455 INFO:tasks.workunit.client.0.smithi031.stdout:3/508: mkdir d4/d54/d56/dae 0 2024-04-22T23:45:46.455 INFO:tasks.workunit.client.0.smithi031.stdout:5/596: readlink db/d16/d2a/d59/d90/da4/lb9 0 2024-04-22T23:45:46.457 INFO:tasks.workunit.client.0.smithi031.stdout:0/530: symlink d0/lb8 0 2024-04-22T23:45:46.459 INFO:tasks.workunit.client.0.smithi031.stdout:8/491: truncate d5/da/d12/d37/d34/f46 2327823 0 2024-04-22T23:45:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:46 smithi044 ceph-mon[34463]: pgmap v29: 65 pgs: 65 active+clean; 1.2 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 32 MiB/s rd, 162 MiB/s wr, 466 op/s 2024-04-22T23:45:46.460 INFO:tasks.workunit.client.0.smithi031.stdout:1/568: sync 2024-04-22T23:45:46.460 INFO:tasks.workunit.client.0.smithi031.stdout:6/637: sync 2024-04-22T23:45:46.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/509: mkdir d4/d54/d56/daf 0 2024-04-22T23:45:46.461 INFO:tasks.workunit.client.0.smithi031.stdout:1/569: chown d1/la 6182 1 2024-04-22T23:45:46.461 INFO:tasks.workunit.client.0.smithi031.stdout:6/638: fdatasync d3/d2a/d3c/d40/d7a/f9a 0 2024-04-22T23:45:46.462 INFO:tasks.workunit.client.0.smithi031.stdout:5/597: rename db/d16/d2a/d59/f88 to db/d16/d1d/d24/fc1 0 2024-04-22T23:45:46.463 INFO:tasks.workunit.client.0.smithi031.stdout:9/574: symlink d2/dc/d8b/d3d/d40/lbd 0 2024-04-22T23:45:46.464 INFO:tasks.workunit.client.0.smithi031.stdout:5/598: write db/d16/d2a/d36/d74/f87 [1243144,83394] 0 2024-04-22T23:45:46.464 INFO:tasks.workunit.client.0.smithi031.stdout:0/531: symlink d0/d1/d2/d36/d11/d1c/d54/lb9 0 2024-04-22T23:45:46.466 INFO:tasks.workunit.client.0.smithi031.stdout:3/510: mknod d4/d83/cb0 0 2024-04-22T23:45:46.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/570: rmdir d1/de/d1e 39 2024-04-22T23:45:46.470 INFO:tasks.workunit.client.0.smithi031.stdout:6/639: dwrite d3/d35/d86/f91 [0,4194304] 0 2024-04-22T23:45:46.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/640: write d3/d2a/d85/fab [182162,79844] 0 2024-04-22T23:45:46.475 INFO:tasks.workunit.client.0.smithi031.stdout:0/532: mkdir d0/d13/d18/d6c/dad/dba 0 2024-04-22T23:45:46.478 INFO:tasks.workunit.client.0.smithi031.stdout:5/599: rename db/d16/d2a/d36/d74/cab to db/d16/d2a/d36/d74/d82/d86/db8/cc2 0 2024-04-22T23:45:46.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/571: mknod d1/db/d16/d1f/d52/db0/cd0 0 2024-04-22T23:45:46.479 INFO:tasks.workunit.client.0.smithi031.stdout:5/600: fdatasync db/d16/d1d/d24/f2e 0 2024-04-22T23:45:46.481 INFO:tasks.workunit.client.0.smithi031.stdout:6/641: dwrite d3/d2a/d3c/f73 [0,4194304] 0 2024-04-22T23:45:46.481 INFO:tasks.workunit.client.0.smithi031.stdout:5/601: dread - db/d16/d1d/d24/d58/d5d/fa0 zero size 2024-04-22T23:45:46.526 INFO:tasks.workunit.client.0.smithi031.stdout:5/602: fdatasync db/d16/d1d/d24/d32/f73 0 2024-04-22T23:45:46.528 INFO:tasks.workunit.client.0.smithi031.stdout:9/575: truncate d2/dc/f50 57183 0 2024-04-22T23:45:46.530 INFO:tasks.workunit.client.0.smithi031.stdout:3/511: rename d4/d54/d2c/d89/f2d to d4/d6/d11/d25/d44/d7f/d5f/fb1 0 2024-04-22T23:45:46.532 INFO:tasks.workunit.client.0.smithi031.stdout:3/512: chown d4/d6/da/f3d 1379268 1 2024-04-22T23:45:46.532 INFO:tasks.workunit.client.0.smithi031.stdout:9/576: symlink d2/d15/lbe 0 2024-04-22T23:45:46.536 INFO:tasks.workunit.client.0.smithi031.stdout:9/577: read - d2/d15/f7c zero size 2024-04-22T23:45:46.538 INFO:tasks.workunit.client.0.smithi031.stdout:8/492: dwrite d5/da/f2c [0,4194304] 0 2024-04-22T23:45:46.538 INFO:tasks.workunit.client.0.smithi031.stdout:1/572: dwrite d1/f6 [0,4194304] 0 2024-04-22T23:45:46.541 INFO:tasks.workunit.client.0.smithi031.stdout:3/513: write d4/d6/d11/d25/d44/d7f/d5f/f63 [4059146,43607] 0 2024-04-22T23:45:46.542 INFO:tasks.workunit.client.0.smithi031.stdout:5/603: dwrite db/d16/d2a/d7c/fa7 [0,4194304] 0 2024-04-22T23:45:46.544 INFO:tasks.workunit.client.0.smithi031.stdout:8/493: mknod d5/c8f 0 2024-04-22T23:45:46.550 INFO:tasks.workunit.client.0.smithi031.stdout:6/642: dread d3/d35/d75/fe3 [0,4194304] 0 2024-04-22T23:45:46.551 INFO:tasks.workunit.client.0.smithi031.stdout:5/604: mknod db/d16/d1d/d24/d58/d5d/cc3 0 2024-04-22T23:45:46.556 INFO:tasks.workunit.client.0.smithi031.stdout:8/494: dwrite d5/da/d12/d37/d4e/f81 [0,4194304] 0 2024-04-22T23:45:46.556 INFO:tasks.workunit.client.0.smithi031.stdout:6/643: mkdir d3/d35/db1/de6 0 2024-04-22T23:45:46.558 INFO:tasks.workunit.client.0.smithi031.stdout:6/644: chown d3/d1f/fce 119072 1 2024-04-22T23:45:46.558 INFO:tasks.workunit.client.0.smithi031.stdout:8/495: dread d5/d48/f77 [0,4194304] 0 2024-04-22T23:45:46.562 INFO:tasks.workunit.client.0.smithi031.stdout:5/605: symlink db/d16/d1d/d24/d32/d62/lc4 0 2024-04-22T23:45:46.562 INFO:tasks.workunit.client.0.smithi031.stdout:3/514: link d4/d6/d4c/d8c/la1 d4/d49/d72/lb2 0 2024-04-22T23:45:46.563 INFO:tasks.workunit.client.0.smithi031.stdout:5/606: fsync db/d16/d2a/d36/d74/fbc 0 2024-04-22T23:45:46.569 INFO:tasks.workunit.client.0.smithi031.stdout:6/645: truncate d3/f17 5958752 0 2024-04-22T23:45:46.572 INFO:tasks.workunit.client.0.smithi031.stdout:3/515: mkdir d4/d6/d11/db3 0 2024-04-22T23:45:46.572 INFO:tasks.workunit.client.0.smithi031.stdout:5/607: dwrite db/d16/d1d/d24/d27/f99 [0,4194304] 0 2024-04-22T23:45:46.574 INFO:tasks.workunit.client.0.smithi031.stdout:7/578: sync 2024-04-22T23:45:46.574 INFO:tasks.workunit.client.0.smithi031.stdout:4/749: sync 2024-04-22T23:45:46.574 INFO:tasks.workunit.client.0.smithi031.stdout:2/591: sync 2024-04-22T23:45:46.575 INFO:tasks.workunit.client.0.smithi031.stdout:1/573: dread d1/d3f/d54/dba/d28/f2a [0,4194304] 0 2024-04-22T23:45:46.576 INFO:tasks.workunit.client.0.smithi031.stdout:4/750: chown db/d31/d40/d62/f77 103 1 2024-04-22T23:45:46.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/646: rmdir d3/d1f/d4c 39 2024-04-22T23:45:46.587 INFO:tasks.workunit.client.0.smithi031.stdout:3/516: unlink d4/d6/d11/d25/c33 0 2024-04-22T23:45:46.591 INFO:tasks.workunit.client.0.smithi031.stdout:9/578: getdents d2/d15 0 2024-04-22T23:45:46.593 INFO:tasks.workunit.client.0.smithi031.stdout:6/647: symlink d3/d35/le7 0 2024-04-22T23:45:46.593 INFO:tasks.workunit.client.0.smithi031.stdout:9/579: dread - d2/dd/d75/d64/fa8 zero size 2024-04-22T23:45:46.593 INFO:tasks.workunit.client.0.smithi031.stdout:4/751: dread db/d14/d2d/d49/f99 [0,4194304] 0 2024-04-22T23:45:46.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/752: chown db/d14/d1d/d7b/d7f/d2a/da1/da7/ce9 63 1 2024-04-22T23:45:46.596 INFO:tasks.workunit.client.0.smithi031.stdout:7/579: getdents d9/d14/d22/dbe 0 2024-04-22T23:45:46.600 INFO:tasks.workunit.client.0.smithi031.stdout:6/648: symlink d3/d35/d75/d7d/le8 0 2024-04-22T23:45:46.601 INFO:tasks.workunit.client.0.smithi031.stdout:5/608: creat db/d16/fc5 x:0 0 0 2024-04-22T23:45:46.601 INFO:tasks.workunit.client.0.smithi031.stdout:7/580: dwrite f2 [0,4194304] 0 2024-04-22T23:45:46.602 INFO:tasks.workunit.client.0.smithi031.stdout:6/649: truncate d3/d35/fc0 306783 0 2024-04-22T23:45:46.607 INFO:tasks.workunit.client.0.smithi031.stdout:1/574: dread d1/de/d1e/d34/d41/f58 [0,4194304] 0 2024-04-22T23:45:46.609 INFO:tasks.workunit.client.0.smithi031.stdout:6/650: stat d3/d1f/f5c 0 2024-04-22T23:45:46.611 INFO:tasks.workunit.client.0.smithi031.stdout:7/581: mkdir d9/d15/d6d/dcc 0 2024-04-22T23:45:46.620 INFO:tasks.workunit.client.0.smithi031.stdout:9/580: dread d2/dc/f48 [0,4194304] 0 2024-04-22T23:45:46.621 INFO:tasks.workunit.client.0.smithi031.stdout:9/581: read - d2/dc/d8b/d3d/d40/da4/fb3 zero size 2024-04-22T23:45:46.621 INFO:tasks.workunit.client.0.smithi031.stdout:8/496: truncate d5/d48/d61/d6b/d6c/f73 174971 0 2024-04-22T23:45:46.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/609: unlink db/d16/d1d/d24/d32/d41/lb2 0 2024-04-22T23:45:46.626 INFO:tasks.workunit.client.0.smithi031.stdout:1/575: unlink d1/d99/d63/l73 0 2024-04-22T23:45:46.627 INFO:tasks.workunit.client.0.smithi031.stdout:1/576: write d1/d99/d7d/f8a [562547,45802] 0 2024-04-22T23:45:46.629 INFO:tasks.workunit.client.0.smithi031.stdout:6/651: link d3/d1f/fce d3/da4/fe9 0 2024-04-22T23:45:46.635 INFO:tasks.workunit.client.0.smithi031.stdout:8/497: mknod d5/da/d83/d8d/c90 0 2024-04-22T23:45:46.635 INFO:tasks.workunit.client.0.smithi031.stdout:9/582: dwrite d2/dc/faa [0,4194304] 0 2024-04-22T23:45:46.640 INFO:tasks.workunit.client.0.smithi031.stdout:7/582: rmdir d9/d14/dc9 0 2024-04-22T23:45:46.645 INFO:tasks.workunit.client.0.smithi031.stdout:9/583: unlink d2/f3c 0 2024-04-22T23:45:46.645 INFO:tasks.workunit.client.0.smithi031.stdout:5/610: dwrite db/d16/d1d/d24/d27/fb0 [0,4194304] 0 2024-04-22T23:45:46.645 INFO:tasks.workunit.client.0.smithi031.stdout:8/498: fsync d5/da/d12/d1d/d38/f47 0 2024-04-22T23:45:46.646 INFO:tasks.workunit.client.0.smithi031.stdout:7/583: dread - d9/d14/d1f/d47/fc1 zero size 2024-04-22T23:45:46.647 INFO:tasks.workunit.client.0.smithi031.stdout:1/577: creat d1/db/d16/fd1 x:0 0 0 2024-04-22T23:45:46.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/584: creat d2/dd/d65/fbf x:0 0 0 2024-04-22T23:45:46.649 INFO:tasks.workunit.client.0.smithi031.stdout:1/578: stat d1/d3f/d54/d84 0 2024-04-22T23:45:46.649 INFO:tasks.workunit.client.0.smithi031.stdout:9/585: chown d2/dd/d75/d64 3284 1 2024-04-22T23:45:46.651 INFO:tasks.workunit.client.0.smithi031.stdout:6/652: rename d3/d2a/d3c/d40/d7a/f9a to d3/d1f/fea 0 2024-04-22T23:45:46.652 INFO:tasks.workunit.client.0.smithi031.stdout:5/611: chown db/c49 7756 1 2024-04-22T23:45:46.656 INFO:tasks.workunit.client.0.smithi031.stdout:8/499: creat d5/da/d83/d8d/f91 x:0 0 0 2024-04-22T23:45:46.660 INFO:tasks.workunit.client.0.smithi031.stdout:5/612: dwrite db/d16/d1d/d24/d32/f73 [0,4194304] 0 2024-04-22T23:45:46.665 INFO:tasks.workunit.client.0.smithi031.stdout:1/579: rename d1/d3f/d54/dba/l5b to d1/d21/d4b/d6f/da6/ld2 0 2024-04-22T23:45:46.666 INFO:tasks.workunit.client.0.smithi031.stdout:1/580: write d1/de/f11 [2092105,92630] 0 2024-04-22T23:45:46.669 INFO:tasks.workunit.client.0.smithi031.stdout:8/500: chown d5/l64 2 1 2024-04-22T23:45:46.672 INFO:tasks.workunit.client.0.smithi031.stdout:9/586: symlink d2/dd/d75/d64/lc0 0 2024-04-22T23:45:46.676 INFO:tasks.workunit.client.0.smithi031.stdout:8/501: symlink d5/da/d12/d2e/d4d/d56/l92 0 2024-04-22T23:45:46.677 INFO:tasks.workunit.client.0.smithi031.stdout:1/581: mknod d1/db/d16/cd3 0 2024-04-22T23:45:46.680 INFO:tasks.workunit.client.0.smithi031.stdout:1/582: creat d1/d3f/fd4 x:0 0 0 2024-04-22T23:45:46.681 INFO:tasks.workunit.client.0.smithi031.stdout:9/587: creat d2/d9a/fc1 x:0 0 0 2024-04-22T23:45:46.683 INFO:tasks.workunit.client.0.smithi031.stdout:1/583: fdatasync d1/d35/d36/f9e 0 2024-04-22T23:45:46.683 INFO:tasks.workunit.client.0.smithi031.stdout:9/588: dread - d2/fb8 zero size 2024-04-22T23:45:46.683 INFO:tasks.workunit.client.0.smithi031.stdout:6/653: dread d3/d2a/d3c/d40/f4a [0,4194304] 0 2024-04-22T23:45:46.684 INFO:tasks.workunit.client.0.smithi031.stdout:8/502: dwrite d5/f87 [0,4194304] 0 2024-04-22T23:45:46.687 INFO:tasks.workunit.client.0.smithi031.stdout:8/503: write d5/da/d12/d37/d49/f4f [2026374,34819] 0 2024-04-22T23:45:46.689 INFO:tasks.workunit.client.0.smithi031.stdout:6/654: creat d3/d2a/d3c/d40/d4d/feb x:0 0 0 2024-04-22T23:45:46.690 INFO:tasks.workunit.client.0.smithi031.stdout:7/584: write d9/d15/d6d/d65/faf [3165123,87025] 0 2024-04-22T23:45:46.692 INFO:tasks.workunit.client.0.smithi031.stdout:8/504: creat d5/da/df/d33/f93 x:0 0 0 2024-04-22T23:45:46.699 INFO:tasks.workunit.client.0.smithi031.stdout:7/585: unlink d9/d15/c33 0 2024-04-22T23:45:46.706 INFO:tasks.workunit.client.0.smithi031.stdout:7/586: dwrite d9/d14/d1f/d7f/fc7 [0,4194304] 0 2024-04-22T23:45:46.706 INFO:tasks.workunit.client.0.smithi031.stdout:9/589: dread d2/dc/f83 [0,4194304] 0 2024-04-22T23:45:46.710 INFO:tasks.workunit.client.0.smithi031.stdout:8/505: write d5/da/d12/d37/d34/f46 [3206878,105854] 0 2024-04-22T23:45:46.711 INFO:tasks.workunit.client.0.smithi031.stdout:1/584: dread d1/db/d16/d1f/f26 [0,4194304] 0 2024-04-22T23:45:46.712 INFO:tasks.workunit.client.0.smithi031.stdout:9/590: mknod d2/dd/d75/d64/d5d/d59/cc2 0 2024-04-22T23:45:46.714 INFO:tasks.workunit.client.0.smithi031.stdout:7/587: creat d9/d15/d6d/dcc/fcd x:0 0 0 2024-04-22T23:45:46.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/591: stat d2/dd/d75/d80/c90 0 2024-04-22T23:45:46.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/588: fsync d9/d14/d22/f2d 0 2024-04-22T23:45:46.715 INFO:tasks.workunit.client.0.smithi031.stdout:1/585: dread d1/db/d16/d1f/f8b [0,4194304] 0 2024-04-22T23:45:46.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/655: dread d3/d1f/f5c [0,4194304] 0 2024-04-22T23:45:46.716 INFO:tasks.workunit.client.0.smithi031.stdout:8/506: chown d5/da/df/c18 542198 1 2024-04-22T23:45:46.717 INFO:tasks.workunit.client.0.smithi031.stdout:2/592: sync 2024-04-22T23:45:46.717 INFO:tasks.workunit.client.0.smithi031.stdout:0/533: sync 2024-04-22T23:45:46.717 INFO:tasks.workunit.client.0.smithi031.stdout:9/592: fdatasync d2/d15/f7c 0 2024-04-22T23:45:46.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/656: fdatasync d3/f1a 0 2024-04-22T23:45:46.719 INFO:tasks.workunit.client.0.smithi031.stdout:7/589: symlink d9/d15/dbf/lce 0 2024-04-22T23:45:46.722 INFO:tasks.workunit.client.0.smithi031.stdout:6/657: chown d3/d2a/d3c 3 1 2024-04-22T23:45:46.722 INFO:tasks.workunit.client.0.smithi031.stdout:0/534: write d0/d1/d2/d36/d11/d1a/d56/fae [826221,92794] 0 2024-04-22T23:45:46.723 INFO:tasks.workunit.client.0.smithi031.stdout:9/593: write d2/d15/d5a/fa3 [640101,86669] 0 2024-04-22T23:45:46.724 INFO:tasks.workunit.client.0.smithi031.stdout:1/586: rename d1/db/d16/d1f/c91 to d1/d35/dbb/cd5 0 2024-04-22T23:45:46.727 INFO:tasks.workunit.client.0.smithi031.stdout:6/658: chown d3/d2a/d85/fab 0 1 2024-04-22T23:45:46.733 INFO:tasks.workunit.client.0.smithi031.stdout:0/535: creat d0/d13/d18/d1e/d25/fbb x:0 0 0 2024-04-22T23:45:46.733 INFO:tasks.workunit.client.0.smithi031.stdout:2/593: dwrite d6/dc/d49/d83/d85/da9/fad [0,4194304] 0 2024-04-22T23:45:46.734 INFO:tasks.workunit.client.0.smithi031.stdout:9/594: mkdir d2/dd/d75/d64/d5d/dc3 0 2024-04-22T23:45:46.735 INFO:tasks.workunit.client.0.smithi031.stdout:2/594: chown d6/dc/d57/f4d 0 1 2024-04-22T23:45:46.736 INFO:tasks.workunit.client.0.smithi031.stdout:2/595: fsync d6/f22 0 2024-04-22T23:45:46.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/587: mknod d1/db/d16/d2f/d81/cd6 0 2024-04-22T23:45:46.738 INFO:tasks.workunit.client.0.smithi031.stdout:3/517: sync 2024-04-22T23:45:46.739 INFO:tasks.workunit.client.0.smithi031.stdout:4/753: sync 2024-04-22T23:45:46.741 INFO:tasks.workunit.client.0.smithi031.stdout:2/596: fsync d6/dc/f9f 0 2024-04-22T23:45:46.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/595: unlink d2/dd/d75/d80/lbc 0 2024-04-22T23:45:46.742 INFO:tasks.workunit.client.0.smithi031.stdout:0/536: mknod d0/d13/d18/d6c/dad/dba/cbc 0 2024-04-22T23:45:46.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/590: dread d9/fa [4194304,4194304] 0 2024-04-22T23:45:46.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/597: fsync d6/dc/d9a/d6c/d86/fb7 0 2024-04-22T23:45:46.745 INFO:tasks.workunit.client.0.smithi031.stdout:1/588: mknod d1/db/d16/d1f/d52/db0/cd7 0 2024-04-22T23:45:46.745 INFO:tasks.workunit.client.0.smithi031.stdout:6/659: write d3/d1f/f5c [3129292,100651] 0 2024-04-22T23:45:46.749 INFO:tasks.workunit.client.0.smithi031.stdout:4/754: dread db/d14/d2d/d49/d59/d6a/fee [0,4194304] 0 2024-04-22T23:45:46.749 INFO:tasks.workunit.client.0.smithi031.stdout:9/596: rename d2/dc/d8b/d3d/f68 to d2/dc/d8b/d3d/fc4 0 2024-04-22T23:45:46.752 INFO:tasks.workunit.client.0.smithi031.stdout:6/660: creat d3/d2a/d3c/d40/d7a/fec x:0 0 0 2024-04-22T23:45:46.754 INFO:tasks.workunit.client.0.smithi031.stdout:8/507: dread d5/da/d12/d1d/d38/f3f [0,4194304] 0 2024-04-22T23:45:46.754 INFO:tasks.workunit.client.0.smithi031.stdout:4/755: write db/d14/d2d/d49/d59/fff [720220,111122] 0 2024-04-22T23:45:46.756 INFO:tasks.workunit.client.0.smithi031.stdout:1/589: rename d1/d35/d36/d9f to d1/d21/d4b/d6f/dd8 0 2024-04-22T23:45:46.757 INFO:tasks.workunit.client.0.smithi031.stdout:1/590: readlink d1/d21/d4e/lbe 0 2024-04-22T23:45:46.765 INFO:tasks.workunit.client.0.smithi031.stdout:9/597: mkdir d2/dc/d8b/d3d/d40/da4/db1/dc5 0 2024-04-22T23:45:46.766 INFO:tasks.workunit.client.0.smithi031.stdout:3/518: dwrite d4/d6/da/f29 [0,4194304] 0 2024-04-22T23:45:46.768 INFO:tasks.workunit.client.0.smithi031.stdout:7/591: dread d9/d15/d6d/d65/f89 [0,4194304] 0 2024-04-22T23:45:46.770 INFO:tasks.workunit.client.0.smithi031.stdout:8/508: unlink d5/da/d12/d2e/f57 0 2024-04-22T23:45:46.770 INFO:tasks.workunit.client.0.smithi031.stdout:0/537: dwrite d0/d1/d2/d36/d11/f43 [0,4194304] 0 2024-04-22T23:45:46.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/598: truncate d6/dc/d9a/d6c/f7e 621373 0 2024-04-22T23:45:46.773 INFO:tasks.workunit.client.0.smithi031.stdout:8/509: chown d5/da/d12/d37/d4e/l72 172 1 2024-04-22T23:45:46.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/599: fdatasync d6/dc/d17/d69/d90/f91 0 2024-04-22T23:45:46.777 INFO:tasks.workunit.client.0.smithi031.stdout:3/519: mkdir d4/d6/d11/d25/d44/d7f/d5f/db4 0 2024-04-22T23:45:46.778 INFO:tasks.workunit.client.0.smithi031.stdout:6/661: dwrite d3/d2a/d3c/d40/f44 [4194304,4194304] 0 2024-04-22T23:45:46.781 INFO:tasks.workunit.client.0.smithi031.stdout:4/756: link db/d14/d2d/d49/d59/d6a/d63/dc9/fe6 db/d14/d1d/d7b/d7f/d2a/d30/f101 0 2024-04-22T23:45:46.783 INFO:tasks.workunit.client.0.smithi031.stdout:7/592: mkdir d9/d15/dcf 0 2024-04-22T23:45:46.785 INFO:tasks.workunit.client.0.smithi031.stdout:6/662: readlink d3/d1f/d4c/l55 0 2024-04-22T23:45:46.788 INFO:tasks.workunit.client.0.smithi031.stdout:4/757: chown db/d14/d2d/f35 80262474 1 2024-04-22T23:45:46.790 INFO:tasks.workunit.client.0.smithi031.stdout:1/591: getdents d1/de/d1e/d34/d66 0 2024-04-22T23:45:46.791 INFO:tasks.workunit.client.0.smithi031.stdout:8/510: dwrite d5/da/d83/d8d/f91 [0,4194304] 0 2024-04-22T23:45:46.792 INFO:tasks.workunit.client.0.smithi031.stdout:4/758: chown db/d14/d1d/d7b/d7f/d2a/da1 1948681 1 2024-04-22T23:45:46.792 INFO:tasks.workunit.client.0.smithi031.stdout:3/520: dwrite d4/d6/d11/d25/d44/d7f/f8f [0,4194304] 0 2024-04-22T23:45:46.795 INFO:tasks.workunit.client.0.smithi031.stdout:1/592: chown d1/db/d16/d1f/d52/db0/cd7 140280812 1 2024-04-22T23:45:46.797 INFO:tasks.workunit.client.0.smithi031.stdout:1/593: write d1/db/d16/d1f/faa [396724,70591] 0 2024-04-22T23:45:46.798 INFO:tasks.workunit.client.0.smithi031.stdout:9/598: link d2/d15/daf/cb4 d2/dd/d75/d64/cc6 0 2024-04-22T23:45:46.799 INFO:tasks.workunit.client.0.smithi031.stdout:0/538: creat d0/d13/d18/d1e/d25/d8e/fbd x:0 0 0 2024-04-22T23:45:46.800 INFO:tasks.workunit.client.0.smithi031.stdout:9/599: dread - d2/dc/d8b/d3d/d40/f39 zero size 2024-04-22T23:45:46.800 INFO:tasks.workunit.client.0.smithi031.stdout:0/539: read d0/d13/d18/f6d [680810,10406] 0 2024-04-22T23:45:46.805 INFO:tasks.workunit.client.0.smithi031.stdout:6/663: read d3/d1f/d4c/d8e/f93 [570179,82057] 0 2024-04-22T23:45:46.806 INFO:tasks.workunit.client.0.smithi031.stdout:0/540: dwrite d0/f3d [0,4194304] 0 2024-04-22T23:45:46.809 INFO:tasks.workunit.client.0.smithi031.stdout:0/541: chown d0/d13/d18/d1e/f6e 20660 1 2024-04-22T23:45:46.809 INFO:tasks.workunit.client.0.smithi031.stdout:8/511: mknod d5/da/d12/d2e/d7e/c94 0 2024-04-22T23:45:46.814 INFO:tasks.workunit.client.0.smithi031.stdout:1/594: mkdir d1/d21/d4e/dd9 0 2024-04-22T23:45:46.814 INFO:tasks.workunit.client.0.smithi031.stdout:1/595: fsync d1/d21/f9b 0 2024-04-22T23:45:46.819 INFO:tasks.workunit.client.0.smithi031.stdout:5/613: sync 2024-04-22T23:45:46.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/600: creat d2/dd/d75/d64/d5d/d59/fc7 x:0 0 0 2024-04-22T23:45:46.823 INFO:tasks.workunit.client.0.smithi031.stdout:7/593: write d9/d15/d6d/fc8 [179886,84871] 0 2024-04-22T23:45:46.823 INFO:tasks.workunit.client.0.smithi031.stdout:7/594: chown f2 1 1 2024-04-22T23:45:46.824 INFO:tasks.workunit.client.0.smithi031.stdout:7/595: readlink d9/d14/d1f/d7f/l81 0 2024-04-22T23:45:46.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/664: rename d3/d35/d75/d7d/c97 to d3/d1f/da3/ced 0 2024-04-22T23:45:46.826 INFO:tasks.workunit.client.0.smithi031.stdout:0/542: creat d0/fbe x:0 0 0 2024-04-22T23:45:46.828 INFO:tasks.workunit.client.0.smithi031.stdout:0/543: chown d0/d13/d18/d1e/d25/d8e/fbd 26205786 1 2024-04-22T23:45:46.830 INFO:tasks.workunit.client.0.smithi031.stdout:1/596: fdatasync d1/db/f19 0 2024-04-22T23:45:46.832 INFO:tasks.workunit.client.0.smithi031.stdout:5/614: creat db/d16/d1d/d24/fc6 x:0 0 0 2024-04-22T23:45:46.833 INFO:tasks.workunit.client.0.smithi031.stdout:6/665: dwrite d3/d35/fb7 [0,4194304] 0 2024-04-22T23:45:46.835 INFO:tasks.workunit.client.0.smithi031.stdout:2/600: getdents d6/dc/d42/d24/d9c 0 2024-04-22T23:45:46.837 INFO:tasks.workunit.client.0.smithi031.stdout:3/521: dwrite d4/d6/d11/f3c [0,4194304] 0 2024-04-22T23:45:46.837 INFO:tasks.workunit.client.0.smithi031.stdout:5/615: write db/d16/d2a/d59/f75 [570086,27653] 0 2024-04-22T23:45:46.841 INFO:tasks.workunit.client.0.smithi031.stdout:7/596: fsync d9/d15/d3a/d41/f8f 0 2024-04-22T23:45:46.843 INFO:tasks.workunit.client.0.smithi031.stdout:1/597: dwrite d1/db/d16/f25 [0,4194304] 0 2024-04-22T23:45:46.844 INFO:tasks.workunit.client.0.smithi031.stdout:7/597: chown d9/d14/d1f/dc6 160702010 1 2024-04-22T23:45:46.846 INFO:tasks.workunit.client.0.smithi031.stdout:1/598: dread - d1/d3f/d54/dba/f98 zero size 2024-04-22T23:45:46.846 INFO:tasks.workunit.client.0.smithi031.stdout:8/512: mkdir d5/da/d12/d95 0 2024-04-22T23:45:46.849 INFO:tasks.workunit.client.0.smithi031.stdout:8/513: truncate d5/da/d12/f14 1412497 0 2024-04-22T23:45:46.849 INFO:tasks.workunit.client.0.smithi031.stdout:1/599: read d1/d99/d7d/f8a [1613531,74087] 0 2024-04-22T23:45:46.850 INFO:tasks.workunit.client.0.smithi031.stdout:0/544: mkdir d0/d13/d18/dbf 0 2024-04-22T23:45:46.853 INFO:tasks.workunit.client.0.smithi031.stdout:1/600: write d1/db/d16/d2f/f3b [1872954,117918] 0 2024-04-22T23:45:46.856 INFO:tasks.workunit.client.0.smithi031.stdout:5/616: symlink db/d16/d2a/d36/d74/lc7 0 2024-04-22T23:45:46.857 INFO:tasks.workunit.client.0.smithi031.stdout:4/759: getdents db 0 2024-04-22T23:45:46.860 INFO:tasks.workunit.client.0.smithi031.stdout:4/760: fsync db/d14/d1d/f76 0 2024-04-22T23:45:46.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/514: dwrite d5/da/d12/d2e/d4d/d56/f88 [0,4194304] 0 2024-04-22T23:45:46.862 INFO:tasks.workunit.client.0.smithi031.stdout:0/545: creat d0/d1/d2/d36/d11/d1c/d54/fc0 x:0 0 0 2024-04-22T23:45:46.864 INFO:tasks.workunit.client.0.smithi031.stdout:6/666: unlink d3/d2a/d3c/d65/c80 0 2024-04-22T23:45:46.867 INFO:tasks.workunit.client.0.smithi031.stdout:3/522: mknod d4/d54/d6d/d98/cb5 0 2024-04-22T23:45:46.869 INFO:tasks.workunit.client.0.smithi031.stdout:1/601: rmdir d1/d35/dbb 39 2024-04-22T23:45:46.869 INFO:tasks.workunit.client.0.smithi031.stdout:3/523: stat d4/d54/c46 0 2024-04-22T23:45:46.872 INFO:tasks.workunit.client.0.smithi031.stdout:7/598: dwrite d9/d14/d22/d28/f36 [4194304,4194304] 0 2024-04-22T23:45:46.876 INFO:tasks.workunit.client.0.smithi031.stdout:3/524: write d4/d6/da/f29 [3718823,123521] 0 2024-04-22T23:45:46.876 INFO:tasks.workunit.client.0.smithi031.stdout:5/617: chown db/f3f 231 1 2024-04-22T23:45:46.878 INFO:tasks.workunit.client.0.smithi031.stdout:8/515: dwrite d5/f87 [0,4194304] 0 2024-04-22T23:45:46.880 INFO:tasks.workunit.client.0.smithi031.stdout:4/761: dwrite db/d14/d2d/d49/d59/d83/d9c/d72/dc6/dd2/fd9 [0,4194304] 0 2024-04-22T23:45:46.885 INFO:tasks.workunit.client.0.smithi031.stdout:4/762: dread - db/d14/d2d/d49/d59/d6a/d63/d8b/ff7 zero size 2024-04-22T23:45:46.885 INFO:tasks.workunit.client.0.smithi031.stdout:9/601: getdents d2/dd 0 2024-04-22T23:45:46.888 INFO:tasks.workunit.client.0.smithi031.stdout:6/667: dwrite d3/d1f/fd3 [0,4194304] 0 2024-04-22T23:45:46.888 INFO:tasks.workunit.client.0.smithi031.stdout:0/546: truncate d0/d13/f67 3263309 0 2024-04-22T23:45:46.890 INFO:tasks.workunit.client.0.smithi031.stdout:0/547: stat d0/d13/d18/l89 0 2024-04-22T23:45:46.891 INFO:tasks.workunit.client.0.smithi031.stdout:2/601: dread d6/dc/d49/f70 [0,4194304] 0 2024-04-22T23:45:46.893 INFO:tasks.workunit.client.0.smithi031.stdout:1/602: symlink d1/de/d1e/d34/lda 0 2024-04-22T23:45:46.893 INFO:tasks.workunit.client.0.smithi031.stdout:8/516: write d5/da/d12/d37/d34/d6f/d5b/f67 [325245,105616] 0 2024-04-22T23:45:46.896 INFO:tasks.workunit.client.0.smithi031.stdout:0/548: chown d0/d1/d2/d36/d11/d1c/d54/f72 337409 1 2024-04-22T23:45:46.897 INFO:tasks.workunit.client.0.smithi031.stdout:1/603: stat d1/db/d16/d2f/f3b 0 2024-04-22T23:45:46.903 INFO:tasks.workunit.client.0.smithi031.stdout:7/599: creat d9/d15/d6d/dcc/fd0 x:0 0 0 2024-04-22T23:45:46.922 INFO:tasks.workunit.client.0.smithi031.stdout:4/763: dread db/d14/d2d/f35 [0,4194304] 0 2024-04-22T23:45:46.924 INFO:tasks.workunit.client.0.smithi031.stdout:4/764: fsync db/d14/d1d/d7b/d7f/d2a/d48/fe5 0 2024-04-22T23:45:46.936 INFO:tasks.workunit.client.0.smithi031.stdout:8/517: write d5/da/d12/d2e/d4d/d56/f5a [2514181,22373] 0 2024-04-22T23:45:46.938 INFO:tasks.workunit.client.0.smithi031.stdout:6/668: dwrite d3/d2a/d85/d96/f9e [0,4194304] 0 2024-04-22T23:45:46.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/669: chown d3/d2a/d85/d96/f9e 811 1 2024-04-22T23:45:46.945 INFO:tasks.workunit.client.0.smithi031.stdout:3/525: rename d4/d54/d2c/d89/c20 to d4/d54/d88/da2/cb6 0 2024-04-22T23:45:46.946 INFO:tasks.workunit.client.0.smithi031.stdout:7/600: fsync d9/d15/d3a/d41/f5a 0 2024-04-22T23:45:46.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/549: creat d0/d13/d18/d1e/d25/d4c/fc1 x:0 0 0 2024-04-22T23:45:46.957 INFO:tasks.workunit.client.0.smithi031.stdout:0/550: write d0/d13/d18/d1e/d25/fbb [284637,108389] 0 2024-04-22T23:45:46.957 INFO:tasks.workunit.client.0.smithi031.stdout:1/604: mknod d1/db/dca/cdb 0 2024-04-22T23:45:46.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/605: fdatasync d1/d3f/d54/dba/f1b 0 2024-04-22T23:45:46.960 INFO:tasks.workunit.client.0.smithi031.stdout:0/551: chown d0/d13/d18/d1e/d25/d8e/da7 15890456 1 2024-04-22T23:45:46.960 INFO:tasks.workunit.client.0.smithi031.stdout:1/606: fdatasync d1/d35/f7b 0 2024-04-22T23:45:46.965 INFO:tasks.workunit.client.0.smithi031.stdout:9/602: creat d2/dd/d75/fc8 x:0 0 0 2024-04-22T23:45:46.976 INFO:tasks.workunit.client.0.smithi031.stdout:4/765: dread db/d14/d2d/fb6 [4194304,4194304] 0 2024-04-22T23:45:46.977 INFO:tasks.workunit.client.0.smithi031.stdout:1/607: dread d1/d35/f96 [0,4194304] 0 2024-04-22T23:45:46.978 INFO:tasks.workunit.client.0.smithi031.stdout:1/608: chown d1/de/d1e/d94 8663461 1 2024-04-22T23:45:46.979 INFO:tasks.workunit.client.0.smithi031.stdout:1/609: chown d1/d35/f96 15316947 1 2024-04-22T23:45:46.981 INFO:tasks.workunit.client.0.smithi031.stdout:1/610: write d1/db/d16/d2f/fae [879142,66883] 0 2024-04-22T23:45:46.982 INFO:tasks.workunit.client.0.smithi031.stdout:1/611: stat d1/d3f/d54/l60 0 2024-04-22T23:45:46.983 INFO:tasks.workunit.client.0.smithi031.stdout:1/612: dread - d1/d21/d4e/fcd zero size 2024-04-22T23:45:46.983 INFO:tasks.workunit.client.0.smithi031.stdout:6/670: symlink d3/d35/db1/de6/lee 0 2024-04-22T23:45:46.985 INFO:tasks.workunit.client.0.smithi031.stdout:6/671: write d3/d1f/f2f [2421539,83989] 0 2024-04-22T23:45:46.986 INFO:tasks.workunit.client.0.smithi031.stdout:6/672: chown d3/d1f/d4c/d8e 28 1 2024-04-22T23:45:46.986 INFO:tasks.workunit.client.0.smithi031.stdout:3/526: link d4/d6/d11/d25/d44/f7e d4/d54/d56/daf/fb7 0 2024-04-22T23:45:46.989 INFO:tasks.workunit.client.0.smithi031.stdout:5/618: truncate db/d16/fa2 1367458 0 2024-04-22T23:45:46.991 INFO:tasks.workunit.client.0.smithi031.stdout:8/518: creat d5/da/d12/f96 x:0 0 0 2024-04-22T23:45:46.991 INFO:tasks.workunit.client.0.smithi031.stdout:8/519: readlink d5/d48/l4a 0 2024-04-22T23:45:46.998 INFO:tasks.workunit.client.0.smithi031.stdout:1/613: mkdir d1/d3f/d54/dba/ddc 0 2024-04-22T23:45:47.001 INFO:tasks.workunit.client.0.smithi031.stdout:3/527: creat d4/d54/d59/fb8 x:0 0 0 2024-04-22T23:45:47.008 INFO:tasks.workunit.client.0.smithi031.stdout:2/602: truncate d6/dc/d48/d4b/f64 174931 0 2024-04-22T23:45:47.011 INFO:tasks.workunit.client.0.smithi031.stdout:1/614: read d1/de/d1e/d34/d41/f62 [4099619,58162] 0 2024-04-22T23:45:47.011 INFO:tasks.workunit.client.0.smithi031.stdout:3/528: creat d4/d6/da/fb9 x:0 0 0 2024-04-22T23:45:47.012 INFO:tasks.workunit.client.0.smithi031.stdout:0/552: write d0/f7f [890254,104800] 0 2024-04-22T23:45:47.013 INFO:tasks.workunit.client.0.smithi031.stdout:7/601: getdents d9/d14/d1f/d45/da6/daa/db0 0 2024-04-22T23:45:47.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/615: chown d1/db/d16/d2f/d81/cd6 2 1 2024-04-22T23:45:47.014 INFO:tasks.workunit.client.0.smithi031.stdout:7/602: chown d9/d14/d22/dbe 1192 1 2024-04-22T23:45:47.019 INFO:tasks.workunit.client.0.smithi031.stdout:5/619: rename db/d16/d1d/d24/d27/f2d to db/d16/d2a/d36/d74/d82/d86/fc8 0 2024-04-22T23:45:47.022 INFO:tasks.workunit.client.0.smithi031.stdout:8/520: mknod d5/da/d12/d2e/c97 0 2024-04-22T23:45:47.026 INFO:tasks.workunit.client.0.smithi031.stdout:2/603: mknod d6/dc/d57/d52/cc5 0 2024-04-22T23:45:47.028 INFO:tasks.workunit.client.0.smithi031.stdout:3/529: mknod d4/d6/d11/cba 0 2024-04-22T23:45:47.030 INFO:tasks.workunit.client.0.smithi031.stdout:0/553: truncate d0/d1/d2/f23 4817963 0 2024-04-22T23:45:47.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/616: write d1/db/d16/d1f/f26 [255959,54811] 0 2024-04-22T23:45:47.034 INFO:tasks.workunit.client.0.smithi031.stdout:1/617: truncate d1/d21/d4b/d6f/da6/fc7 4338221 0 2024-04-22T23:45:47.036 INFO:tasks.workunit.client.0.smithi031.stdout:1/618: write d1/d3f/d54/dba/d28/fa8 [389448,113846] 0 2024-04-22T23:45:47.038 INFO:tasks.workunit.client.0.smithi031.stdout:4/766: link db/d14/d1d/d7b/d7f/d2a/d47/cfd db/d14/d1d/d7b/d7f/d2a/d47/d80/deb/c102 0 2024-04-22T23:45:47.041 INFO:tasks.workunit.client.0.smithi031.stdout:2/604: creat d6/fc6 x:0 0 0 2024-04-22T23:45:47.042 INFO:tasks.workunit.client.0.smithi031.stdout:3/530: mknod d4/d49/d72/cbb 0 2024-04-22T23:45:47.043 INFO:tasks.workunit.client.0.smithi031.stdout:7/603: symlink d9/d14/d22/dbe/ld1 0 2024-04-22T23:45:47.045 INFO:tasks.workunit.client.0.smithi031.stdout:6/673: truncate d3/d2a/d3c/f73 3546250 0 2024-04-22T23:45:47.057 INFO:tasks.workunit.client.0.smithi031.stdout:3/531: creat d4/d6/d11/d25/d44/d7f/fbc x:0 0 0 2024-04-22T23:45:47.059 INFO:tasks.workunit.client.0.smithi031.stdout:6/674: mknod d3/d35/db1/de6/cef 0 2024-04-22T23:45:47.060 INFO:tasks.workunit.client.0.smithi031.stdout:8/521: write d5/d48/d61/d6b/d6c/f70 [4190532,63719] 0 2024-04-22T23:45:47.064 INFO:tasks.workunit.client.0.smithi031.stdout:2/605: creat d6/dc/d48/d4b/d6b/fc7 x:0 0 0 2024-04-22T23:45:47.066 INFO:tasks.workunit.client.0.smithi031.stdout:3/532: unlink d4/d6/d22/c52 0 2024-04-22T23:45:47.067 INFO:tasks.workunit.client.0.smithi031.stdout:7/604: creat d9/d15/dcf/fd2 x:0 0 0 2024-04-22T23:45:47.068 INFO:tasks.workunit.client.0.smithi031.stdout:4/767: dread db/d14/d1d/d7b/d7f/f2f [0,4194304] 0 2024-04-22T23:45:47.070 INFO:tasks.workunit.client.0.smithi031.stdout:4/768: write db/d14/d1d/d7b/d7f/d2a/d48/fec [473682,38990] 0 2024-04-22T23:45:47.072 INFO:tasks.workunit.client.0.smithi031.stdout:8/522: fsync d5/da/d12/d1d/d38/f3f 0 2024-04-22T23:45:47.076 INFO:tasks.workunit.client.0.smithi031.stdout:2/606: symlink d6/dc/d57/d52/db5/lc8 0 2024-04-22T23:45:47.077 INFO:tasks.workunit.client.0.smithi031.stdout:0/554: getdents d0/d13/d18/d1e/d25/d8e/da7/d2b 0 2024-04-22T23:45:47.080 INFO:tasks.workunit.client.0.smithi031.stdout:7/605: dwrite d9/f3b [4194304,4194304] 0 2024-04-22T23:45:47.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/620: dwrite db/f81 [0,4194304] 0 2024-04-22T23:45:47.084 INFO:tasks.workunit.client.0.smithi031.stdout:7/606: fsync d9/d14/d1f/d47/fc1 0 2024-04-22T23:45:47.084 INFO:tasks.workunit.client.0.smithi031.stdout:6/675: truncate d3/d35/d86/f91 4614702 0 2024-04-22T23:45:47.086 INFO:tasks.workunit.client.0.smithi031.stdout:7/607: chown d9/d14/d1f/d47/lab 0 1 2024-04-22T23:45:47.090 INFO:tasks.workunit.client.0.smithi031.stdout:7/608: write d9/d15/d6d/f8e [918200,105495] 0 2024-04-22T23:45:47.090 INFO:tasks.workunit.client.0.smithi031.stdout:7/609: chown d9/d14/d1f/d47/fc1 244 1 2024-04-22T23:45:47.097 INFO:tasks.workunit.client.0.smithi031.stdout:6/676: symlink d3/lf0 0 2024-04-22T23:45:47.107 INFO:tasks.workunit.client.0.smithi031.stdout:7/610: chown d9/d15/d6d/c50 688862 1 2024-04-22T23:45:47.107 INFO:tasks.workunit.client.0.smithi031.stdout:1/619: link d1/d35/dbb/cd5 d1/d35/cdd 0 2024-04-22T23:45:47.109 INFO:tasks.workunit.client.0.smithi031.stdout:1/620: chown d1/d21/d4b/cc9 14411061 1 2024-04-22T23:45:47.110 INFO:tasks.workunit.client.0.smithi031.stdout:5/621: mknod db/d16/d2a/cc9 0 2024-04-22T23:45:47.111 INFO:tasks.workunit.client.0.smithi031.stdout:2/607: dwrite d6/dc/d42/d24/d32/f3a [0,4194304] 0 2024-04-22T23:45:47.114 INFO:tasks.workunit.client.0.smithi031.stdout:5/622: chown db/d16/d1d/d24/d32/f39 883919945 1 2024-04-22T23:45:47.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/677: mkdir d3/d1f/d4c/df1 0 2024-04-22T23:45:47.120 INFO:tasks.workunit.client.0.smithi031.stdout:3/533: getdents d4/d54/d2c/d89 0 2024-04-22T23:45:47.122 INFO:tasks.workunit.client.0.smithi031.stdout:5/623: readlink db/d16/d2a/d36/d74/lc7 0 2024-04-22T23:45:47.123 INFO:tasks.workunit.client.0.smithi031.stdout:2/608: creat d6/dc/d9a/d9e/fc9 x:0 0 0 2024-04-22T23:45:47.125 INFO:tasks.workunit.client.0.smithi031.stdout:4/769: dread db/d14/d2d/d49/d59/d83/d9c/fa0 [4194304,4194304] 0 2024-04-22T23:45:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:6/678: fdatasync d3/d35/d75/d7d/fe5 0 2024-04-22T23:45:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/555: dwrite d0/d1/d2/fa [0,4194304] 0 2024-04-22T23:45:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:1/621: dwrite d1/de/d1e/d34/fa2 [0,4194304] 0 2024-04-22T23:45:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:3/534: chown d4/d6/d11/d25/d44/d7f/d5f 8019120 1 2024-04-22T23:45:47.138 INFO:tasks.workunit.client.0.smithi031.stdout:3/535: read d4/d54/d2c/d89/f2b [31055,112145] 0 2024-04-22T23:45:47.138 INFO:tasks.workunit.client.0.smithi031.stdout:0/556: mknod d0/d1/d2/d36/d11/cc2 0 2024-04-22T23:45:47.141 INFO:tasks.workunit.client.0.smithi031.stdout:4/770: chown db/d14/d1d/d7b/d7f/d2a/d30/f60 67452 1 2024-04-22T23:45:47.143 INFO:tasks.workunit.client.0.smithi031.stdout:7/611: dread d9/d14/d1f/d7f/f8c [0,4194304] 0 2024-04-22T23:45:47.145 INFO:tasks.workunit.client.0.smithi031.stdout:3/536: read - d4/d54/d2c/fa8 zero size 2024-04-22T23:45:47.147 INFO:tasks.workunit.client.0.smithi031.stdout:0/557: readlink d0/d1/d2/d36/d11/d1a/l4a 0 2024-04-22T23:45:47.148 INFO:tasks.workunit.client.0.smithi031.stdout:2/609: link d6/c8 d6/dc/d42/d24/d9c/cca 0 2024-04-22T23:45:47.152 INFO:tasks.workunit.client.0.smithi031.stdout:7/612: chown d9/d14/d22/f78 1 1 2024-04-22T23:45:47.154 INFO:tasks.workunit.client.0.smithi031.stdout:6/679: dwrite d3/d2a/d85/d96/fbf [0,4194304] 0 2024-04-22T23:45:47.155 INFO:tasks.workunit.client.0.smithi031.stdout:5/624: dwrite db/d16/d1d/d24/d27/d9e/fae [0,4194304] 0 2024-04-22T23:45:47.157 INFO:tasks.workunit.client.0.smithi031.stdout:3/537: dread d4/d54/d2c/d89/d2e/f39 [0,4194304] 0 2024-04-22T23:45:47.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/680: chown d3/d35/d75 280760 1 2024-04-22T23:45:47.160 INFO:tasks.workunit.client.0.smithi031.stdout:1/622: dwrite d1/d21/d4b/f5d [0,4194304] 0 2024-04-22T23:45:47.160 INFO:tasks.workunit.client.0.smithi031.stdout:4/771: truncate db/d14/d2d/d49/dcd/fd3 152147 0 2024-04-22T23:45:47.162 INFO:tasks.workunit.client.0.smithi031.stdout:2/610: symlink d6/dc/d57/d52/db5/lcb 0 2024-04-22T23:45:47.163 INFO:tasks.workunit.client.0.smithi031.stdout:7/613: chown d9/d14/d22/d5c/f9e 1012919734 1 2024-04-22T23:45:47.164 INFO:tasks.workunit.client.0.smithi031.stdout:5/625: rmdir db/d16/d2a/d59 39 2024-04-22T23:45:47.169 INFO:tasks.workunit.client.0.smithi031.stdout:3/538: unlink d4/d6/d11/cba 0 2024-04-22T23:45:47.171 INFO:tasks.workunit.client.0.smithi031.stdout:6/681: unlink d3/c6b 0 2024-04-22T23:45:47.173 INFO:tasks.workunit.client.0.smithi031.stdout:6/682: chown d3/d1f/d4c 5837945 1 2024-04-22T23:45:47.174 INFO:tasks.workunit.client.0.smithi031.stdout:1/623: rename d1/d21/d4e/lbe to d1/de/d1e/d34/d41/lde 0 2024-04-22T23:45:47.178 INFO:tasks.workunit.client.0.smithi031.stdout:6/683: dread - d3/fba zero size 2024-04-22T23:45:47.178 INFO:tasks.workunit.client.0.smithi031.stdout:4/772: dread db/d14/d2d/d49/fb3 [0,4194304] 0 2024-04-22T23:45:47.179 INFO:tasks.workunit.client.0.smithi031.stdout:2/611: rename d6/dc/d48/d4b/lbe to d6/dc/d49/d83/lcc 0 2024-04-22T23:45:47.184 INFO:tasks.workunit.client.0.smithi031.stdout:5/626: truncate db/d16/d1d/d24/d27/d9e/fa5 846252 0 2024-04-22T23:45:47.188 INFO:tasks.workunit.client.0.smithi031.stdout:3/539: dwrite d4/d83/fab [0,4194304] 0 2024-04-22T23:45:47.189 INFO:tasks.workunit.client.0.smithi031.stdout:9/603: sync 2024-04-22T23:45:47.190 INFO:tasks.workunit.client.0.smithi031.stdout:8/523: sync 2024-04-22T23:45:47.190 INFO:tasks.workunit.client.0.smithi031.stdout:7/614: dwrite d9/d14/d1f/fc2 [0,4194304] 0 2024-04-22T23:45:47.192 INFO:tasks.workunit.client.0.smithi031.stdout:0/558: dread d0/d1/f6b [0,4194304] 0 2024-04-22T23:45:47.195 INFO:tasks.workunit.client.0.smithi031.stdout:6/684: write d3/d1f/fce [428111,116194] 0 2024-04-22T23:45:47.198 INFO:tasks.workunit.client.0.smithi031.stdout:3/540: write d4/d54/d6d/f9b [578430,83685] 0 2024-04-22T23:45:47.200 INFO:tasks.workunit.client.0.smithi031.stdout:9/604: mknod d2/dd/d75/d80/cc9 0 2024-04-22T23:45:47.200 INFO:tasks.workunit.client.0.smithi031.stdout:0/559: fsync d0/d1/d2/fa 0 2024-04-22T23:45:47.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/685: fsync d3/d77/d8d/fa0 0 2024-04-22T23:45:47.204 INFO:tasks.workunit.client.0.smithi031.stdout:5/627: read db/d16/d1d/d24/f2e [6080426,95432] 0 2024-04-22T23:45:47.205 INFO:tasks.workunit.client.0.smithi031.stdout:1/624: dwrite d1/d21/d4b/f5d [0,4194304] 0 2024-04-22T23:45:47.206 INFO:tasks.workunit.client.0.smithi031.stdout:7/615: truncate d9/d15/d6d/d65/faf 3473940 0 2024-04-22T23:45:47.207 INFO:tasks.workunit.client.0.smithi031.stdout:3/541: fdatasync d4/d6/d11/d25/d44/fa4 0 2024-04-22T23:45:47.208 INFO:tasks.workunit.client.0.smithi031.stdout:8/524: rename d5/d48/d61/d6b/d6c/f7a to d5/da/df/d33/f98 0 2024-04-22T23:45:47.209 INFO:tasks.workunit.client.0.smithi031.stdout:6/686: rmdir d3/d1f/d60 39 2024-04-22T23:45:47.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/612: dwrite d6/dc/d9a/d9e/fc9 [0,4194304] 0 2024-04-22T23:45:47.214 INFO:tasks.workunit.client.0.smithi031.stdout:8/525: readlink d5/da/d12/l24 0 2024-04-22T23:45:47.215 INFO:tasks.workunit.client.0.smithi031.stdout:2/613: read d6/dc/d9a/d9e/fc9 [1833995,117792] 0 2024-04-22T23:45:47.216 INFO:tasks.workunit.client.0.smithi031.stdout:0/560: chown d0/d1/d2/d36/f1d 109301098 1 2024-04-22T23:45:47.216 INFO:tasks.workunit.client.0.smithi031.stdout:4/773: rename db/d14/d1d/d7b/d7f/d2a/da1/da7/cda to db/d14/d1d/d7b/d7f/d2a/d48/c103 0 2024-04-22T23:45:47.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/628: rmdir db/d16/d2a/d36/d74/d82 39 2024-04-22T23:45:47.218 INFO:tasks.workunit.client.0.smithi031.stdout:1/625: symlink d1/d35/d36/ldf 0 2024-04-22T23:45:47.220 INFO:tasks.workunit.client.0.smithi031.stdout:8/526: fsync d5/da/d12/d37/d34/d6f/d5b/f84 0 2024-04-22T23:45:47.225 INFO:tasks.workunit.client.0.smithi031.stdout:6/687: link d3/d2a/d85/d96/fbf d3/d2a/d3c/db6/ff2 0 2024-04-22T23:45:47.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/616: rename d9/d14/d1f/d45/da6/daa/lb3 to d9/d14/d22/d28/d7a/ld3 0 2024-04-22T23:45:47.229 INFO:tasks.workunit.client.0.smithi031.stdout:0/561: mkdir d0/d13/d18/dbf/dc3 0 2024-04-22T23:45:47.232 INFO:tasks.workunit.client.0.smithi031.stdout:8/527: unlink d5/da/d12/d2e/d7e/l6a 0 2024-04-22T23:45:47.234 INFO:tasks.workunit.client.0.smithi031.stdout:1/626: write d1/d3f/d54/dba/d28/f2a [3032854,85521] 0 2024-04-22T23:45:47.235 INFO:tasks.workunit.client.0.smithi031.stdout:3/542: dwrite d4/d54/d2c/d89/d2e/d45/f92 [4194304,4194304] 0 2024-04-22T23:45:47.235 INFO:tasks.workunit.client.0.smithi031.stdout:6/688: mknod d3/d2a/d85/cf3 0 2024-04-22T23:45:47.237 INFO:tasks.workunit.client.0.smithi031.stdout:5/629: dwrite db/d16/f3e [8388608,4194304] 0 2024-04-22T23:45:47.238 INFO:tasks.workunit.client.0.smithi031.stdout:4/774: rename db/fc to db/d14/d2d/d49/d59/d6a/d63/dc9/f104 0 2024-04-22T23:45:47.240 INFO:tasks.workunit.client.0.smithi031.stdout:2/614: getdents d6/dc/d17 0 2024-04-22T23:45:47.241 INFO:tasks.workunit.client.0.smithi031.stdout:1/627: fdatasync d1/de/d1e/d34/fa2 0 2024-04-22T23:45:47.250 INFO:tasks.workunit.client.0.smithi031.stdout:6/689: stat d3/d1f/da3/ced 0 2024-04-22T23:45:47.252 INFO:tasks.workunit.client.0.smithi031.stdout:1/628: dwrite d1/de/d1e/dcf/fb7 [0,4194304] 0 2024-04-22T23:45:47.261 INFO:tasks.workunit.client.0.smithi031.stdout:5/630: unlink db/d16/l48 0 2024-04-22T23:45:47.262 INFO:tasks.workunit.client.0.smithi031.stdout:0/562: dwrite d0/d13/d18/d1e/d25/f85 [0,4194304] 0 2024-04-22T23:45:47.265 INFO:tasks.workunit.client.0.smithi031.stdout:0/563: write d0/fbe [156453,80576] 0 2024-04-22T23:45:47.266 INFO:tasks.workunit.client.0.smithi031.stdout:6/690: creat d3/d35/db1/de6/ff4 x:0 0 0 2024-04-22T23:45:47.267 INFO:tasks.workunit.client.0.smithi031.stdout:8/528: dread d5/da/d12/d37/d34/f5e [0,4194304] 0 2024-04-22T23:45:47.271 INFO:tasks.workunit.client.0.smithi031.stdout:1/629: dwrite d1/d21/fc4 [0,4194304] 0 2024-04-22T23:45:47.271 INFO:tasks.workunit.client.0.smithi031.stdout:0/564: dread - d0/d13/d18/d1e/d25/d8e/fbd zero size 2024-04-22T23:45:47.275 INFO:tasks.workunit.client.0.smithi031.stdout:7/617: creat d9/d14/d1f/d45/da6/fd4 x:0 0 0 2024-04-22T23:45:47.276 INFO:tasks.workunit.client.0.smithi031.stdout:8/529: symlink d5/da/d12/d37/d34/d6f/d5b/l99 0 2024-04-22T23:45:47.277 INFO:tasks.workunit.client.0.smithi031.stdout:1/630: write d1/db/d16/f25 [519406,17987] 0 2024-04-22T23:45:47.277 INFO:tasks.workunit.client.0.smithi031.stdout:6/691: truncate d3/d2a/d3c/d40/d4d/f61 2478986 0 2024-04-22T23:45:47.278 INFO:tasks.workunit.client.0.smithi031.stdout:5/631: truncate db/d16/d1d/d24/d58/db1/fb5 575497 0 2024-04-22T23:45:47.283 INFO:tasks.workunit.client.0.smithi031.stdout:0/565: fsync d0/d13/d18/f6d 0 2024-04-22T23:45:47.285 INFO:tasks.workunit.client.0.smithi031.stdout:8/530: read - d5/da/d12/f96 zero size 2024-04-22T23:45:47.287 INFO:tasks.workunit.client.0.smithi031.stdout:7/618: dread d9/f37 [0,4194304] 0 2024-04-22T23:45:47.288 INFO:tasks.workunit.client.0.smithi031.stdout:4/775: write db/d31/d40/d9b/fa6 [1395584,111959] 0 2024-04-22T23:45:47.289 INFO:tasks.workunit.client.0.smithi031.stdout:6/692: dread - d3/d2a/d3c/db6/fdb zero size 2024-04-22T23:45:47.292 INFO:tasks.workunit.client.0.smithi031.stdout:3/543: dwrite d4/d83/f8b [4194304,4194304] 0 2024-04-22T23:45:47.293 INFO:tasks.workunit.client.0.smithi031.stdout:2/615: dwrite d6/ff [0,4194304] 0 2024-04-22T23:45:47.294 INFO:tasks.workunit.client.0.smithi031.stdout:7/619: chown d9/d14/d1f/d47/fc1 2002121 1 2024-04-22T23:45:47.296 INFO:tasks.workunit.client.0.smithi031.stdout:0/566: readlink d0/d1/d2/l98 0 2024-04-22T23:45:47.303 INFO:tasks.workunit.client.0.smithi031.stdout:4/776: chown db/d14/d2d/d49/d59/d83 362657 1 2024-04-22T23:45:47.305 INFO:tasks.workunit.client.0.smithi031.stdout:3/544: mkdir d4/d6/d11/d25/dbd 0 2024-04-22T23:45:47.307 INFO:tasks.workunit.client.0.smithi031.stdout:2/616: symlink d6/dc/d42/d28/d94/db4/lcd 0 2024-04-22T23:45:47.311 INFO:tasks.workunit.client.0.smithi031.stdout:3/545: chown d4/d6/d11/c47 487 1 2024-04-22T23:45:47.311 INFO:tasks.workunit.client.0.smithi031.stdout:4/777: fdatasync db/d14/d1d/d7b/d7f/d2a/da1/fbc 0 2024-04-22T23:45:47.314 INFO:tasks.workunit.client.0.smithi031.stdout:7/620: dwrite d9/d15/d6d/d65/f90 [0,4194304] 0 2024-04-22T23:45:47.315 INFO:tasks.workunit.client.0.smithi031.stdout:1/631: truncate d1/d3f/d54/dba/f1a 3548265 0 2024-04-22T23:45:47.317 INFO:tasks.workunit.client.0.smithi031.stdout:2/617: unlink d6/dc/d42/d24/d32/f3a 0 2024-04-22T23:45:47.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/567: dwrite d0/d1/d3f/f8b [4194304,4194304] 0 2024-04-22T23:45:47.320 INFO:tasks.workunit.client.0.smithi031.stdout:1/632: stat d1/de/d1e/d34/d41/l77 0 2024-04-22T23:45:47.320 INFO:tasks.workunit.client.0.smithi031.stdout:7/621: chown d9/d15/f99 75 1 2024-04-22T23:45:47.320 INFO:tasks.workunit.client.0.smithi031.stdout:3/546: creat d4/d6/d11/d3e/fbe x:0 0 0 2024-04-22T23:45:47.321 INFO:tasks.workunit.client.0.smithi031.stdout:8/531: truncate d5/f76 2301215 0 2024-04-22T23:45:47.325 INFO:tasks.workunit.client.0.smithi031.stdout:6/693: dread d3/f15 [4194304,4194304] 0 2024-04-22T23:45:47.326 INFO:tasks.workunit.client.0.smithi031.stdout:2/618: mkdir d6/dce 0 2024-04-22T23:45:47.326 INFO:tasks.workunit.client.0.smithi031.stdout:3/547: read d4/d6/da/f3d [3126572,62193] 0 2024-04-22T23:45:47.326 INFO:tasks.workunit.client.0.smithi031.stdout:4/778: mknod db/d14/d2d/d49/d59/d6a/d63/d86/c105 0 2024-04-22T23:45:47.329 INFO:tasks.workunit.client.0.smithi031.stdout:0/568: write d0/d13/d18/d1e/d25/d8e/da7/f79 [2406502,83067] 0 2024-04-22T23:45:47.329 INFO:tasks.workunit.client.0.smithi031.stdout:1/633: mknod d1/db/d16/d1f/ce0 0 2024-04-22T23:45:47.331 INFO:tasks.workunit.client.0.smithi031.stdout:3/548: dread - d4/d54/d2c/fa8 zero size 2024-04-22T23:45:47.334 INFO:tasks.workunit.client.0.smithi031.stdout:5/632: dwrite db/d16/d1d/d24/d58/f51 [0,4194304] 0 2024-04-22T23:45:47.342 INFO:tasks.workunit.client.0.smithi031.stdout:3/549: write d4/d6/d11/f3c [4269376,23775] 0 2024-04-22T23:45:47.342 INFO:tasks.workunit.client.0.smithi031.stdout:7/622: getdents d9/d14/d1f/d45/da6/daa/db0 0 2024-04-22T23:45:47.342 INFO:tasks.workunit.client.0.smithi031.stdout:1/634: read d1/de/d1e/d34/d41/f4c [2904916,68888] 0 2024-04-22T23:45:47.343 INFO:tasks.workunit.client.0.smithi031.stdout:1/635: fdatasync d1/d99/d63/f9c 0 2024-04-22T23:45:47.343 INFO:tasks.workunit.client.0.smithi031.stdout:2/619: truncate d6/dc/d48/d4b/fb0 790027 0 2024-04-22T23:45:47.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/779: dwrite db/d14/d2d/d49/dcd/fd3 [0,4194304] 0 2024-04-22T23:45:47.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/780: readlink db/d31/d40/d62/lef 0 2024-04-22T23:45:47.348 INFO:tasks.workunit.client.0.smithi031.stdout:4/781: stat db/d14/d1d/d7b 0 2024-04-22T23:45:47.348 INFO:tasks.workunit.client.0.smithi031.stdout:9/605: sync 2024-04-22T23:45:47.352 INFO:tasks.workunit.client.0.smithi031.stdout:8/532: dread d5/da/d12/d37/d34/f4c [0,4194304] 0 2024-04-22T23:45:47.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/550: mknod d4/d54/d88/cbf 0 2024-04-22T23:45:47.352 INFO:tasks.workunit.client.0.smithi031.stdout:8/533: readlink d5/d48/l86 0 2024-04-22T23:45:47.354 INFO:tasks.workunit.client.0.smithi031.stdout:9/606: read d2/d15/d5a/fa3 [2194690,28112] 0 2024-04-22T23:45:47.354 INFO:tasks.workunit.client.0.smithi031.stdout:8/534: write d5/da/d12/d37/d49/f54 [3329007,60563] 0 2024-04-22T23:45:47.355 INFO:tasks.workunit.client.0.smithi031.stdout:8/535: chown d5/da/l25 13443619 1 2024-04-22T23:45:47.357 INFO:tasks.workunit.client.0.smithi031.stdout:5/633: mkdir db/d16/d2a/dca 0 2024-04-22T23:45:47.357 INFO:tasks.workunit.client.0.smithi031.stdout:3/551: dread d4/d83/fab [0,4194304] 0 2024-04-22T23:45:47.359 INFO:tasks.workunit.client.0.smithi031.stdout:6/694: rename d3/d2a/d3c/d40/d4d/l5a to d3/d1f/lf5 0 2024-04-22T23:45:47.368 INFO:tasks.workunit.client.0.smithi031.stdout:4/782: dread db/d31/d40/d62/f77 [0,4194304] 0 2024-04-22T23:45:47.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/569: truncate d0/d1/d2/d36/f2f 1670970 0 2024-04-22T23:45:47.372 INFO:tasks.workunit.client.0.smithi031.stdout:9/607: creat d2/d15/daf/fca x:0 0 0 2024-04-22T23:45:47.375 INFO:tasks.workunit.client.0.smithi031.stdout:3/552: mkdir d4/d6/d11/dc0 0 2024-04-22T23:45:47.379 INFO:tasks.workunit.client.0.smithi031.stdout:1/636: rename d1/d3f/d54/dba/c8e to d1/d21/d4b/d6f/dd8/ce1 0 2024-04-22T23:45:47.379 INFO:tasks.workunit.client.0.smithi031.stdout:9/608: dwrite d2/d9a/fc1 [0,4194304] 0 2024-04-22T23:45:47.380 INFO:tasks.workunit.client.0.smithi031.stdout:1/637: readlink d1/d3f/d54/l6b 0 2024-04-22T23:45:47.385 INFO:tasks.workunit.client.0.smithi031.stdout:0/570: chown d0/d1/d2/d17/cab 456 1 2024-04-22T23:45:47.385 INFO:tasks.workunit.client.0.smithi031.stdout:5/634: mknod db/d16/d1d/d24/d32/ccb 0 2024-04-22T23:45:47.386 INFO:tasks.workunit.client.0.smithi031.stdout:4/783: dread db/d14/d1d/d7b/d7f/d2a/d30/f101 [0,4194304] 0 2024-04-22T23:45:47.388 INFO:tasks.workunit.client.0.smithi031.stdout:5/635: write db/d16/d1d/d24/f5a [930305,113614] 0 2024-04-22T23:45:47.389 INFO:tasks.workunit.client.0.smithi031.stdout:9/609: rmdir d2/dc/d3e/da1 39 2024-04-22T23:45:47.392 INFO:tasks.workunit.client.0.smithi031.stdout:7/623: getdents d9/d15/d6d/dcc 0 2024-04-22T23:45:47.395 INFO:tasks.workunit.client.0.smithi031.stdout:0/571: fsync d0/d13/d18/d1e/d25/f38 0 2024-04-22T23:45:47.399 INFO:tasks.workunit.client.0.smithi031.stdout:3/553: mkdir d4/d54/d85/dc1 0 2024-04-22T23:45:47.400 INFO:tasks.workunit.client.0.smithi031.stdout:0/572: dread d0/d13/d18/d6c/f74 [0,4194304] 0 2024-04-22T23:45:47.400 INFO:tasks.workunit.client.0.smithi031.stdout:6/695: creat d3/d2a/d3c/ff6 x:0 0 0 2024-04-22T23:45:47.401 INFO:tasks.workunit.client.0.smithi031.stdout:0/573: chown d0/d1/d2/d36/d11/lb2 67900448 1 2024-04-22T23:45:47.403 INFO:tasks.workunit.client.0.smithi031.stdout:2/620: rename d6/dc/d42/d28/d94/lb1 to d6/dc/d9a/d6c/lcf 0 2024-04-22T23:45:47.405 INFO:tasks.workunit.client.0.smithi031.stdout:5/636: creat db/d8d/d9b/fcc x:0 0 0 2024-04-22T23:45:47.405 INFO:tasks.workunit.client.0.smithi031.stdout:2/621: write d6/dc/f9f [4412147,128895] 0 2024-04-22T23:45:47.409 INFO:tasks.workunit.client.0.smithi031.stdout:0/574: dwrite d0/d13/f5b [0,4194304] 0 2024-04-22T23:45:47.409 INFO:tasks.workunit.client.0.smithi031.stdout:1/638: link d1/f1d d1/db/d16/d1f/fe2 0 2024-04-22T23:45:47.410 INFO:tasks.workunit.client.0.smithi031.stdout:8/536: truncate d5/da/d12/d2e/d4d/d56/f71 2362359 0 2024-04-22T23:45:47.413 INFO:tasks.workunit.client.0.smithi031.stdout:8/537: write d5/da/df/f13 [1123662,29862] 0 2024-04-22T23:45:47.415 INFO:tasks.workunit.client.0.smithi031.stdout:8/538: stat d5/da/d12/d37/d49 0 2024-04-22T23:45:47.416 INFO:tasks.workunit.client.0.smithi031.stdout:6/696: rename d3/d1f/d4c/df1 to d3/d1f/d4c/df7 0 2024-04-22T23:45:47.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/539: write d5/da/d12/d37/d34/d6f/d5b/f8a [1879130,31416] 0 2024-04-22T23:45:47.420 INFO:tasks.workunit.client.0.smithi031.stdout:1/639: fsync d1/d21/d4e/f71 0 2024-04-22T23:45:47.420 INFO:tasks.workunit.client.0.smithi031.stdout:9/610: dread d2/dd/d75/d64/d5d/d59/fb0 [0,4194304] 0 2024-04-22T23:45:47.424 INFO:tasks.workunit.client.0.smithi031.stdout:0/575: creat d0/d1/d2/d36/d11/d1a/fc4 x:0 0 0 2024-04-22T23:45:47.425 INFO:tasks.workunit.client.0.smithi031.stdout:0/576: chown d0/d13/d18/f48 568266453 1 2024-04-22T23:45:47.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/611: read d2/dc/f43 [2876931,118712] 0 2024-04-22T23:45:47.426 INFO:tasks.workunit.client.0.smithi031.stdout:9/612: stat d2/dd/d75/d4a/c31 0 2024-04-22T23:45:47.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/697: stat d3/d1f/c41 0 2024-04-22T23:45:47.454 INFO:tasks.workunit.client.0.smithi031.stdout:4/784: rename db/d14/d2d/l9f to db/d14/d2d/d49/d59/d8a/d89/l106 0 2024-04-22T23:45:47.455 INFO:tasks.workunit.client.0.smithi031.stdout:5/637: symlink db/d16/d2a/d36/d74/d82/d86/db8/lcd 0 2024-04-22T23:45:47.455 INFO:tasks.workunit.client.0.smithi031.stdout:8/540: rmdir d5/da/d12/d37/d4e 39 2024-04-22T23:45:47.457 INFO:tasks.workunit.client.0.smithi031.stdout:9/613: unlink d2/dc/d8b/d3d/c70 0 2024-04-22T23:45:47.462 INFO:tasks.workunit.client.0.smithi031.stdout:6/698: dwrite d3/d35/db1/de6/ff4 [0,4194304] 0 2024-04-22T23:45:47.463 INFO:tasks.workunit.client.0.smithi031.stdout:2/622: write d6/d45/d50/f58 [4368320,45712] 0 2024-04-22T23:45:47.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/638: chown db/d16/d1d/d24/d27/l64 1593 1 2024-04-22T23:45:47.465 INFO:tasks.workunit.client.0.smithi031.stdout:0/577: mknod d0/db5/cc5 0 2024-04-22T23:45:47.466 INFO:tasks.workunit.client.0.smithi031.stdout:7/624: getdents d9 0 2024-04-22T23:45:47.471 INFO:tasks.workunit.client.0.smithi031.stdout:4/785: rename db/d31 to db/d14/d2d/d49/d59/d6a/d63/d107 0 2024-04-22T23:45:47.472 INFO:tasks.workunit.client.0.smithi031.stdout:3/554: dwrite d4/f1e [0,4194304] 0 2024-04-22T23:45:47.477 INFO:tasks.workunit.client.0.smithi031.stdout:2/623: chown d6/dc/d17/c30 1360 1 2024-04-22T23:45:47.477 INFO:tasks.workunit.client.0.smithi031.stdout:8/541: dwrite d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:47.479 INFO:tasks.workunit.client.0.smithi031.stdout:3/555: fsync d4/d6/d11/d25/d44/fa4 0 2024-04-22T23:45:47.480 INFO:tasks.workunit.client.0.smithi031.stdout:8/542: stat d5/da/d12/d1d/d38/f3f 0 2024-04-22T23:45:47.481 INFO:tasks.workunit.client.0.smithi031.stdout:6/699: creat d3/d2a/d3c/d40/d7a/ff8 x:0 0 0 2024-04-22T23:45:47.484 INFO:tasks.workunit.client.0.smithi031.stdout:2/624: unlink d6/dc/d42/f5c 0 2024-04-22T23:45:47.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/700: readlink d3/l3b 0 2024-04-22T23:45:47.485 INFO:tasks.workunit.client.0.smithi031.stdout:5/639: creat db/d16/fce x:0 0 0 2024-04-22T23:45:47.486 INFO:tasks.workunit.client.0.smithi031.stdout:0/578: link d0/d13/d18/d1e/d25/f85 d0/d1/d2/d36/d11/d1a/d56/fc6 0 2024-04-22T23:45:47.487 INFO:tasks.workunit.client.0.smithi031.stdout:8/543: mkdir d5/da/d12/d2e/d4d/d9a 0 2024-04-22T23:45:47.492 INFO:tasks.workunit.client.0.smithi031.stdout:9/614: dwrite d2/dd/d75/d64/d5d/d59/fc7 [0,4194304] 0 2024-04-22T23:45:47.500 INFO:tasks.workunit.client.0.smithi031.stdout:7/625: dwrite d9/d14/d1f/d7f/f8c [0,4194304] 0 2024-04-22T23:45:47.500 INFO:tasks.workunit.client.0.smithi031.stdout:6/701: mkdir d3/d1f/d4c/df9 0 2024-04-22T23:45:47.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/556: dread d4/d6/f7 [0,4194304] 0 2024-04-22T23:45:47.503 INFO:tasks.workunit.client.0.smithi031.stdout:0/579: mkdir d0/d13/d18/d1e/d25/d8e/dc7 0 2024-04-22T23:45:47.505 INFO:tasks.workunit.client.0.smithi031.stdout:7/626: dread - d9/d15/f60 zero size 2024-04-22T23:45:47.505 INFO:tasks.workunit.client.0.smithi031.stdout:9/615: mknod d2/dd/d75/d64/ccb 0 2024-04-22T23:45:47.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/580: stat d0/d1/d2/d36/d11/d1a/d56/fc6 0 2024-04-22T23:45:47.508 INFO:tasks.workunit.client.0.smithi031.stdout:7/627: write d9/d15/f56 [1930509,122702] 0 2024-04-22T23:45:47.515 INFO:tasks.workunit.client.0.smithi031.stdout:9/616: creat d2/d15/fcc x:0 0 0 2024-04-22T23:45:47.516 INFO:tasks.workunit.client.0.smithi031.stdout:4/786: mknod db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/c108 0 2024-04-22T23:45:47.518 INFO:tasks.workunit.client.0.smithi031.stdout:6/702: truncate d3/f25 432174 0 2024-04-22T23:45:47.518 INFO:tasks.workunit.client.0.smithi031.stdout:8/544: getdents d5/da/d12/d1d/d38 0 2024-04-22T23:45:47.519 INFO:tasks.workunit.client.0.smithi031.stdout:2/625: dwrite d6/dc/d9a/d6c/f7e [0,4194304] 0 2024-04-22T23:45:47.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/581: rename d0/d1/d2/d36/d11/d1a/d56/fa4 to d0/d13/d18/d6c/dad/fc8 0 2024-04-22T23:45:47.520 INFO:tasks.workunit.client.0.smithi031.stdout:9/617: dread d2/f6e [0,4194304] 0 2024-04-22T23:45:47.521 INFO:tasks.workunit.client.0.smithi031.stdout:0/582: chown d0/d1/d3f 942351585 1 2024-04-22T23:45:47.521 INFO:tasks.workunit.client.0.smithi031.stdout:2/626: write d6/fc6 [702447,55653] 0 2024-04-22T23:45:47.524 INFO:tasks.workunit.client.0.smithi031.stdout:3/557: truncate d4/d54/d56/daf/fb7 1942662 0 2024-04-22T23:45:47.525 INFO:tasks.workunit.client.0.smithi031.stdout:4/787: mknod db/d14/d2d/d49/d59/d6a/c109 0 2024-04-22T23:45:47.526 INFO:tasks.workunit.client.0.smithi031.stdout:0/583: dread - d0/d13/d18/d1e/d25/d8e/da7/d2b/fa3 zero size 2024-04-22T23:45:47.527 INFO:tasks.workunit.client.0.smithi031.stdout:5/640: dread db/d16/d2a/d7c/fa7 [0,4194304] 0 2024-04-22T23:45:47.528 INFO:tasks.workunit.client.0.smithi031.stdout:7/628: link d9/d14/d1f/d45/da6/daa/db0/fb6 d9/d14/d22/d28/fd5 0 2024-04-22T23:45:47.530 INFO:tasks.workunit.client.0.smithi031.stdout:1/640: sync 2024-04-22T23:45:47.530 INFO:tasks.workunit.client.0.smithi031.stdout:9/618: mkdir d2/dd/d65/dcd 0 2024-04-22T23:45:47.532 INFO:tasks.workunit.client.0.smithi031.stdout:7/629: readlink d9/d14/d1f/d45/da6/lb8 0 2024-04-22T23:45:47.533 INFO:tasks.workunit.client.0.smithi031.stdout:8/545: rename d5/da/d12/d37/d49 to d5/da/d12/d37/d34/d6f/d85/d9b 0 2024-04-22T23:45:47.534 INFO:tasks.workunit.client.0.smithi031.stdout:1/641: chown d1/db/d16/d2f/l8c 590525741 1 2024-04-22T23:45:47.543 INFO:tasks.workunit.client.0.smithi031.stdout:3/558: dwrite d4/d54/d2c/d89/f62 [0,4194304] 0 2024-04-22T23:45:47.543 INFO:tasks.workunit.client.0.smithi031.stdout:6/703: dwrite d3/d35/d86/dae/fd8 [0,4194304] 0 2024-04-22T23:45:47.544 INFO:tasks.workunit.client.0.smithi031.stdout:9/619: rmdir d2/dc/d6d 39 2024-04-22T23:45:47.546 INFO:tasks.workunit.client.0.smithi031.stdout:7/630: mkdir d9/d14/dd6 0 2024-04-22T23:45:47.548 INFO:tasks.workunit.client.0.smithi031.stdout:3/559: dread - d4/d6/d11/d25/d44/d7f/d5f/fa9 zero size 2024-04-22T23:45:47.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/620: write d2/dd/d75/d4a/fba [25729,96047] 0 2024-04-22T23:45:47.550 INFO:tasks.workunit.client.0.smithi031.stdout:2/627: rename d6/dc/f5f to d6/dc/d9a/d9e/da4/fd0 0 2024-04-22T23:45:47.553 INFO:tasks.workunit.client.0.smithi031.stdout:3/560: truncate d4/d6/d11/d25/d44/d7f/d5f/f68 591893 0 2024-04-22T23:45:47.557 INFO:tasks.workunit.client.0.smithi031.stdout:5/641: symlink db/d16/d2a/d59/d90/lcf 0 2024-04-22T23:45:47.564 INFO:tasks.workunit.client.0.smithi031.stdout:0/584: dread d0/d13/d18/d1e/d25/d8e/da7/f79 [0,4194304] 0 2024-04-22T23:45:47.566 INFO:tasks.workunit.client.0.smithi031.stdout:0/585: write d0/d1/d2/d36/f9c [1160159,50749] 0 2024-04-22T23:45:47.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/788: rename db/d14/d1d/l58 to db/d14/d2d/d49/d59/d6a/d41/l10a 0 2024-04-22T23:45:47.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/789: chown db/d14/d1d/d7b/d7f/f2f 1840723 1 2024-04-22T23:45:47.568 INFO:tasks.workunit.client.0.smithi031.stdout:2/628: rmdir d6/d45/d50 39 2024-04-22T23:45:47.569 INFO:tasks.workunit.client.0.smithi031.stdout:8/546: symlink d5/da/d12/d37/d4e/l9c 0 2024-04-22T23:45:47.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/642: symlink d1/de/d1e/le3 0 2024-04-22T23:45:47.571 INFO:tasks.workunit.client.0.smithi031.stdout:2/629: truncate d6/dc/d42/d24/f8f 4370362 0 2024-04-22T23:45:47.573 INFO:tasks.workunit.client.0.smithi031.stdout:0/586: dread d0/d13/d18/f6d [0,4194304] 0 2024-04-22T23:45:47.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/621: truncate d2/dc/d8b/f58 628128 0 2024-04-22T23:45:47.580 INFO:tasks.workunit.client.0.smithi031.stdout:2/630: dwrite d6/dc/d49/d83/d85/fa1 [0,4194304] 0 2024-04-22T23:45:47.582 INFO:tasks.workunit.client.0.smithi031.stdout:7/631: rename d9/d15/d6d/d65/f89 to d9/d14/d22/d5c/d5f/fd7 0 2024-04-22T23:45:47.583 INFO:tasks.workunit.client.0.smithi031.stdout:0/587: dwrite d0/d13/d18/d1e/d25/d8e/da7/f55 [0,4194304] 0 2024-04-22T23:45:47.583 INFO:tasks.workunit.client.0.smithi031.stdout:2/631: write d6/dc/d42/d24/f8f [3297950,124250] 0 2024-04-22T23:45:47.585 INFO:tasks.workunit.client.0.smithi031.stdout:1/643: unlink d1/d3f/d90/l89 0 2024-04-22T23:45:47.586 INFO:tasks.workunit.client.0.smithi031.stdout:2/632: write d6/dc/d17/d69/f7f [869249,124467] 0 2024-04-22T23:45:47.592 INFO:tasks.workunit.client.0.smithi031.stdout:2/633: dread d6/dc/d49/f70 [0,4194304] 0 2024-04-22T23:45:47.595 INFO:tasks.workunit.client.0.smithi031.stdout:0/588: rmdir d0/d1/d2/d36 39 2024-04-22T23:45:47.597 INFO:tasks.workunit.client.0.smithi031.stdout:5/642: getdents db/d16/d2a/d36 0 2024-04-22T23:45:47.598 INFO:tasks.workunit.client.0.smithi031.stdout:5/643: chown db/d16/d2a/d59/d90 3226807 1 2024-04-22T23:45:47.603 INFO:tasks.workunit.client.0.smithi031.stdout:6/704: getdents d3/d2a/d85/d96 0 2024-04-22T23:45:47.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/705: fdatasync d3/d2a/d3c/db6/fdb 0 2024-04-22T23:45:47.608 INFO:tasks.workunit.client.0.smithi031.stdout:7/632: mkdir d9/d14/d1f/d45/da6/daa/dbb/dd8 0 2024-04-22T23:45:47.608 INFO:tasks.workunit.client.0.smithi031.stdout:7/633: dread - d9/d14/d1f/d47/fc1 zero size 2024-04-22T23:45:47.609 INFO:tasks.workunit.client.0.smithi031.stdout:0/589: mkdir d0/d13/d18/d1e/d25/d8e/da7/dc9 0 2024-04-22T23:45:47.611 INFO:tasks.workunit.client.0.smithi031.stdout:5/644: creat db/d16/d1d/d24/d32/d41/fd0 x:0 0 0 2024-04-22T23:45:47.611 INFO:tasks.workunit.client.0.smithi031.stdout:5/645: readlink l1 0 2024-04-22T23:45:47.613 INFO:tasks.workunit.client.0.smithi031.stdout:8/547: link d5/da/d83/d8d/c80 d5/da/d12/d2e/d7e/c9d 0 2024-04-22T23:45:47.614 INFO:tasks.workunit.client.0.smithi031.stdout:8/548: chown d5/da/d12/d2e 6859 1 2024-04-22T23:45:47.617 INFO:tasks.workunit.client.0.smithi031.stdout:2/634: mknod d6/dc/d48/d4b/d6b/cd1 0 2024-04-22T23:45:47.620 INFO:tasks.workunit.client.0.smithi031.stdout:5/646: rmdir db/d16/d1d 39 2024-04-22T23:45:47.624 INFO:tasks.workunit.client.0.smithi031.stdout:9/622: link d2/dd/d75/d64/d5d/l98 d2/dc/d6d/lce 0 2024-04-22T23:45:47.624 INFO:tasks.workunit.client.0.smithi031.stdout:2/635: read d6/f19 [12298,102042] 0 2024-04-22T23:45:47.625 INFO:tasks.workunit.client.0.smithi031.stdout:6/706: creat d3/d1f/d4c/de2/ffa x:0 0 0 2024-04-22T23:45:47.653 INFO:tasks.workunit.client.0.smithi031.stdout:0/590: rename d0/d1/d2/d36/d11/d1c/l46 to d0/d13/d18/d6c/dad/dba/lca 0 2024-04-22T23:45:47.655 INFO:tasks.workunit.client.0.smithi031.stdout:2/636: mknod d6/cd2 0 2024-04-22T23:45:47.656 INFO:tasks.workunit.client.0.smithi031.stdout:6/707: dread - d3/d2a/d3c/d65/fb4 zero size 2024-04-22T23:45:47.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/644: dwrite d1/f1d [0,4194304] 0 2024-04-22T23:45:47.661 INFO:tasks.workunit.client.0.smithi031.stdout:8/549: link d5/da/d12/d37/d34/l35 d5/da/d83/l9e 0 2024-04-22T23:45:47.664 INFO:tasks.workunit.client.0.smithi031.stdout:9/623: getdents d2/d15/d5a 0 2024-04-22T23:45:47.664 INFO:tasks.workunit.client.0.smithi031.stdout:8/550: read d5/da/d12/d2e/d4d/f52 [3961232,58329] 0 2024-04-22T23:45:47.666 INFO:tasks.workunit.client.0.smithi031.stdout:8/551: dread - d5/da/df/d33/f5c zero size 2024-04-22T23:45:47.667 INFO:tasks.workunit.client.0.smithi031.stdout:6/708: unlink d3/d2a/d3c/d40/d5b/faa 0 2024-04-22T23:45:47.669 INFO:tasks.workunit.client.0.smithi031.stdout:4/790: dread db/d14/f6e [0,4194304] 0 2024-04-22T23:45:47.670 INFO:tasks.workunit.client.0.smithi031.stdout:9/624: symlink d2/d9a/lcf 0 2024-04-22T23:45:47.677 INFO:tasks.workunit.client.0.smithi031.stdout:6/709: creat d3/da4/ffb x:0 0 0 2024-04-22T23:45:47.678 INFO:tasks.workunit.client.0.smithi031.stdout:8/552: mknod d5/da/df/d33/c9f 0 2024-04-22T23:45:47.681 INFO:tasks.workunit.client.0.smithi031.stdout:3/561: sync 2024-04-22T23:45:47.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/791: rmdir db/d14/d2d/d49/d59/d83/d9c/d72/dc6 39 2024-04-22T23:45:47.684 INFO:tasks.workunit.client.0.smithi031.stdout:3/562: rename d4/d6 to d4/d6/d11/d25/d44/d7f/d5f/dc2 22 2024-04-22T23:45:47.684 INFO:tasks.workunit.client.0.smithi031.stdout:2/637: creat d6/dc/d17/d69/d78/fd3 x:0 0 0 2024-04-22T23:45:47.685 INFO:tasks.workunit.client.0.smithi031.stdout:6/710: dwrite d3/d1f/d4c/d8e/fdd [0,4194304] 0 2024-04-22T23:45:47.688 INFO:tasks.workunit.client.0.smithi031.stdout:8/553: write d5/da/f59 [3940597,111002] 0 2024-04-22T23:45:47.691 INFO:tasks.workunit.client.0.smithi031.stdout:4/792: truncate db/d14/d2d/d49/d59/d83/d9c/f88 1992255 0 2024-04-22T23:45:47.692 INFO:tasks.workunit.client.0.smithi031.stdout:1/645: getdents d1/de/d1e 0 2024-04-22T23:45:47.693 INFO:tasks.workunit.client.0.smithi031.stdout:4/793: write db/d14/d1d/d7b/d7f/f32 [360458,57070] 0 2024-04-22T23:45:47.698 INFO:tasks.workunit.client.0.smithi031.stdout:3/563: rename d4/d54/d2c/d89/d9c to d4/d54/d2c/d89/d2e/d45/d96/dc3 0 2024-04-22T23:45:47.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/554: write d5/da/df/d33/f98 [836212,44151] 0 2024-04-22T23:45:47.703 INFO:tasks.workunit.client.0.smithi031.stdout:3/564: truncate d4/d6/d11/d25/d44/d7f/d5f/f9d 910418 0 2024-04-22T23:45:47.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/646: unlink d1/d35/f96 0 2024-04-22T23:45:47.704 INFO:tasks.workunit.client.0.smithi031.stdout:1/647: chown d1/d3f/d54/dba/d28 1785058 1 2024-04-22T23:45:47.707 INFO:tasks.workunit.client.0.smithi031.stdout:6/711: unlink d3/d35/d86/cc1 0 2024-04-22T23:45:47.708 INFO:tasks.workunit.client.0.smithi031.stdout:0/591: truncate d0/d1/d2/d36/d11/f14 2949585 0 2024-04-22T23:45:47.710 INFO:tasks.workunit.client.0.smithi031.stdout:4/794: rename db/d14/d1d/d7b/d7f/d2a/d47/f7a to db/d14/d2d/d49/d59/d6a/f10b 0 2024-04-22T23:45:47.711 INFO:tasks.workunit.client.0.smithi031.stdout:8/555: creat d5/da/d12/d1d/fa0 x:0 0 0 2024-04-22T23:45:47.713 INFO:tasks.workunit.client.0.smithi031.stdout:9/625: write d2/dc/f43 [2974099,82328] 0 2024-04-22T23:45:47.714 INFO:tasks.workunit.client.0.smithi031.stdout:8/556: chown d5/da/df/d33/f65 1177406118 1 2024-04-22T23:45:47.714 INFO:tasks.workunit.client.0.smithi031.stdout:3/565: chown d4/d6/d11/d25/d44/f7e 47 1 2024-04-22T23:45:47.715 INFO:tasks.workunit.client.0.smithi031.stdout:0/592: dread d0/d13/f5b [0,4194304] 0 2024-04-22T23:45:47.715 INFO:tasks.workunit.client.0.smithi031.stdout:9/626: fdatasync d2/dc/f25 0 2024-04-22T23:45:47.715 INFO:tasks.workunit.client.0.smithi031.stdout:1/648: unlink d1/db/d16/fa9 0 2024-04-22T23:45:47.716 INFO:tasks.workunit.client.0.smithi031.stdout:6/712: mknod d3/d2a/d3c/d40/d4d/cfc 0 2024-04-22T23:45:47.717 INFO:tasks.workunit.client.0.smithi031.stdout:9/627: chown d2/dd/d75/d80/cc9 114263 1 2024-04-22T23:45:47.720 INFO:tasks.workunit.client.0.smithi031.stdout:2/638: getdents d6/dc/d48 0 2024-04-22T23:45:47.722 INFO:tasks.workunit.client.0.smithi031.stdout:4/795: creat db/d14/d2d/d49/d59/d8a/f10c x:0 0 0 2024-04-22T23:45:47.724 INFO:tasks.workunit.client.0.smithi031.stdout:4/796: chown l0 1777 1 2024-04-22T23:45:47.725 INFO:tasks.workunit.client.0.smithi031.stdout:8/557: creat d5/d48/d61/d6b/fa1 x:0 0 0 2024-04-22T23:45:47.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/649: dread d1/d21/fc4 [0,4194304] 0 2024-04-22T23:45:47.728 INFO:tasks.workunit.client.0.smithi031.stdout:0/593: readlink d0/d13/d18/d1e/d25/l2e 0 2024-04-22T23:45:47.729 INFO:tasks.workunit.client.0.smithi031.stdout:3/566: creat d4/d6/d11/d25/d44/d7f/fc4 x:0 0 0 2024-04-22T23:45:47.730 INFO:tasks.workunit.client.0.smithi031.stdout:6/713: dwrite d3/d35/fbd [0,4194304] 0 2024-04-22T23:45:47.731 INFO:tasks.workunit.client.0.smithi031.stdout:9/628: dwrite d2/d15/fcc [0,4194304] 0 2024-04-22T23:45:47.736 INFO:tasks.workunit.client.0.smithi031.stdout:2/639: mkdir d6/dc/d42/d24/dd4 0 2024-04-22T23:45:47.740 INFO:tasks.workunit.client.0.smithi031.stdout:1/650: dwrite d1/de/d1e/d34/fa2 [0,4194304] 0 2024-04-22T23:45:47.742 INFO:tasks.workunit.client.0.smithi031.stdout:1/651: fsync d1/de/d1e/dcf/fb7 0 2024-04-22T23:45:47.743 INFO:tasks.workunit.client.0.smithi031.stdout:4/797: creat db/d14/d1d/d7b/dc1/f10d x:0 0 0 2024-04-22T23:45:47.747 INFO:tasks.workunit.client.0.smithi031.stdout:6/714: rmdir d3/d2a/d85/d96/dcb 39 2024-04-22T23:45:47.751 INFO:tasks.workunit.client.0.smithi031.stdout:0/594: truncate d0/d1/d2/d36/d11/d1c/d54/f69 643203 0 2024-04-22T23:45:47.753 INFO:tasks.workunit.client.0.smithi031.stdout:0/595: fdatasync d0/d1/d2/d36/f9c 0 2024-04-22T23:45:47.753 INFO:tasks.workunit.client.0.smithi031.stdout:0/596: readlink d0/d13/l80 0 2024-04-22T23:45:47.755 INFO:tasks.workunit.client.0.smithi031.stdout:9/629: mkdir d2/dd/d75/d4a/da0/dd0 0 2024-04-22T23:45:47.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/640: unlink d6/dc/d17/d69/c8d 0 2024-04-22T23:45:47.762 INFO:tasks.workunit.client.0.smithi031.stdout:1/652: fdatasync d1/db/f19 0 2024-04-22T23:45:47.763 INFO:tasks.workunit.client.0.smithi031.stdout:3/567: stat d4/d54/d2c/d89/d2e/d45 0 2024-04-22T23:45:47.763 INFO:tasks.workunit.client.0.smithi031.stdout:1/653: fdatasync d1/db/d16/f25 0 2024-04-22T23:45:47.764 INFO:tasks.workunit.client.0.smithi031.stdout:4/798: fdatasync db/d14/d2d/d49/d59/d6a/d63/d107/d40/fb2 0 2024-04-22T23:45:47.764 INFO:tasks.workunit.client.0.smithi031.stdout:3/568: write d4/d54/d2c/d89/d2e/d45/f6c [7741697,19070] 0 2024-04-22T23:45:47.765 INFO:tasks.workunit.client.0.smithi031.stdout:6/715: creat d3/d35/db1/de6/ffd x:0 0 0 2024-04-22T23:45:47.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/634: sync 2024-04-22T23:45:47.793 INFO:tasks.workunit.client.0.smithi031.stdout:5/647: sync 2024-04-22T23:45:47.815 INFO:tasks.workunit.client.0.smithi031.stdout:0/597: unlink d0/d13/d18/d1e/d25/f38 0 2024-04-22T23:45:47.816 INFO:tasks.workunit.client.0.smithi031.stdout:9/630: truncate d2/d15/f7c 68495 0 2024-04-22T23:45:47.819 INFO:tasks.workunit.client.0.smithi031.stdout:3/569: dwrite d4/d54/d2c/d89/d2e/f97 [0,4194304] 0 2024-04-22T23:45:47.820 INFO:tasks.workunit.client.0.smithi031.stdout:6/716: dwrite d3/d35/fcf [0,4194304] 0 2024-04-22T23:45:47.821 INFO:tasks.workunit.client.0.smithi031.stdout:7/635: write d9/d14/d22/d28/d7a/fa8 [3030379,62031] 0 2024-04-22T23:45:47.824 INFO:tasks.workunit.client.0.smithi031.stdout:8/558: getdents d5/da/d12/d37 0 2024-04-22T23:45:47.826 INFO:tasks.workunit.client.0.smithi031.stdout:9/631: mknod d2/dd/d75/d64/d5d/d59/cd1 0 2024-04-22T23:45:47.827 INFO:tasks.workunit.client.0.smithi031.stdout:4/799: creat db/d14/d2d/d49/d59/d83/d9c/d72/dc6/dd2/f10e x:0 0 0 2024-04-22T23:45:47.827 INFO:tasks.workunit.client.0.smithi031.stdout:0/598: chown d0/d1/d2/c63 674 1 2024-04-22T23:45:47.828 INFO:tasks.workunit.client.0.smithi031.stdout:6/717: mkdir d3/d2a/d3c/d40/d5b/dfe 0 2024-04-22T23:45:47.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/654: write d1/db/d16/d1f/d52/f5c [1651488,116454] 0 2024-04-22T23:45:47.837 INFO:tasks.workunit.client.0.smithi031.stdout:3/570: dwrite d4/d54/d2c/d89/d2e/f97 [0,4194304] 0 2024-04-22T23:45:47.839 INFO:tasks.workunit.client.0.smithi031.stdout:9/632: dread d2/dc/d8b/d3d/d40/f39 [0,4194304] 0 2024-04-22T23:45:47.840 INFO:tasks.workunit.client.0.smithi031.stdout:4/800: read db/d14/d1d/f25 [123961,101037] 0 2024-04-22T23:45:47.842 INFO:tasks.workunit.client.0.smithi031.stdout:3/571: dread d4/d54/d2c/d89/d2e/f4f [0,4194304] 0 2024-04-22T23:45:47.843 INFO:tasks.workunit.client.0.smithi031.stdout:1/655: dwrite d1/d3f/d54/dba/d28/fa5 [0,4194304] 0 2024-04-22T23:45:47.844 INFO:tasks.workunit.client.0.smithi031.stdout:1/656: chown d1/d3f/d54 0 1 2024-04-22T23:45:47.854 INFO:tasks.workunit.client.0.smithi031.stdout:6/718: mknod d3/d2a/d3c/d40/d5b/dfe/cff 0 2024-04-22T23:45:47.855 INFO:tasks.workunit.client.0.smithi031.stdout:6/719: readlink d3/d2a/d3c/d40/d4d/lac 0 2024-04-22T23:45:47.857 INFO:tasks.workunit.client.0.smithi031.stdout:6/720: chown d3/d2a/d3c/d40/d5b/fc5 48 1 2024-04-22T23:45:47.857 INFO:tasks.workunit.client.0.smithi031.stdout:8/559: rename d5/d48/d61/d6b/d6c/f75 to d5/da/fa2 0 2024-04-22T23:45:47.859 INFO:tasks.workunit.client.0.smithi031.stdout:6/721: chown d3/f45 6468 1 2024-04-22T23:45:47.859 INFO:tasks.workunit.client.0.smithi031.stdout:7/636: truncate d9/d14/d22/d28/d7a/f7d 1101538 0 2024-04-22T23:45:47.865 INFO:tasks.workunit.client.0.smithi031.stdout:8/560: dwrite d5/da/d12/d37/d34/f46 [0,4194304] 0 2024-04-22T23:45:47.866 INFO:tasks.workunit.client.0.smithi031.stdout:4/801: mkdir db/d14/d2d/d49/d59/d6a/d63/d86/df2/d10f 0 2024-04-22T23:45:47.868 INFO:tasks.workunit.client.0.smithi031.stdout:1/657: dread d1/db/d16/f25 [0,4194304] 0 2024-04-22T23:45:47.869 INFO:tasks.workunit.client.0.smithi031.stdout:6/722: mknod d3/d1f/d4c/d8e/c100 0 2024-04-22T23:45:47.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/633: dwrite d2/f8 [0,4194304] 0 2024-04-22T23:45:47.874 INFO:tasks.workunit.client.0.smithi031.stdout:5/648: rename db/cc to db/d16/d1d/d24/d32/cd1 0 2024-04-22T23:45:47.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/634: dread - d2/fb8 zero size 2024-04-22T23:45:47.874 INFO:tasks.workunit.client.0.smithi031.stdout:3/572: creat d4/d54/d2c/d89/d2e/d45/fc5 x:0 0 0 2024-04-22T23:45:47.875 INFO:tasks.workunit.client.0.smithi031.stdout:9/635: write d2/dc/d3e/f41 [4012241,103798] 0 2024-04-22T23:45:47.878 INFO:tasks.workunit.client.0.smithi031.stdout:4/802: symlink db/d14/d2d/d49/d59/d6a/d63/l110 0 2024-04-22T23:45:47.879 INFO:tasks.workunit.client.0.smithi031.stdout:6/723: dwrite d3/f45 [4194304,4194304] 0 2024-04-22T23:45:47.880 INFO:tasks.workunit.client.0.smithi031.stdout:1/658: symlink d1/d21/d4b/d6f/da6/le4 0 2024-04-22T23:45:47.882 INFO:tasks.workunit.client.0.smithi031.stdout:9/636: write d2/d15/daf/fca [160149,37022] 0 2024-04-22T23:45:47.883 INFO:tasks.workunit.client.0.smithi031.stdout:8/561: symlink d5/da/d12/d2e/d4d/d9a/la3 0 2024-04-22T23:45:47.884 INFO:tasks.workunit.client.0.smithi031.stdout:7/637: rename d9/d15/d3a/c59 to d9/d14/d1f/d45/da6/daa/cd9 0 2024-04-22T23:45:47.885 INFO:tasks.workunit.client.0.smithi031.stdout:9/637: write d2/dd/d75/d4a/f36 [1699498,74929] 0 2024-04-22T23:45:47.887 INFO:tasks.workunit.client.0.smithi031.stdout:7/638: chown d9/d15/f35 195 1 2024-04-22T23:45:47.892 INFO:tasks.workunit.client.0.smithi031.stdout:6/724: dwrite d3/d35/d86/dae/fd8 [0,4194304] 0 2024-04-22T23:45:47.893 INFO:tasks.workunit.client.0.smithi031.stdout:8/562: rmdir d5/da/d12/d37 39 2024-04-22T23:45:47.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/639: unlink d9/d14/d1f/d47/fb1 0 2024-04-22T23:45:47.899 INFO:tasks.workunit.client.0.smithi031.stdout:4/803: mkdir db/d14/d2d/d111 0 2024-04-22T23:45:47.901 INFO:tasks.workunit.client.0.smithi031.stdout:9/638: truncate d2/dd/d75/d64/f55 149803 0 2024-04-22T23:45:47.906 INFO:tasks.workunit.client.0.smithi031.stdout:2/641: sync 2024-04-22T23:45:47.912 INFO:tasks.workunit.client.0.smithi031.stdout:8/563: creat d5/da/d83/d8d/fa4 x:0 0 0 2024-04-22T23:45:47.912 INFO:tasks.workunit.client.0.smithi031.stdout:3/573: rmdir d4/d6/d11/db3 0 2024-04-22T23:45:47.913 INFO:tasks.workunit.client.0.smithi031.stdout:9/639: mknod d2/dc/d8b/d3d/d40/da4/db1/cd2 0 2024-04-22T23:45:47.913 INFO:tasks.workunit.client.0.smithi031.stdout:7/640: creat d9/d14/d1f/d45/da6/daa/fda x:0 0 0 2024-04-22T23:45:47.915 INFO:tasks.workunit.client.0.smithi031.stdout:9/640: chown d2/d9a/lcf 1 1 2024-04-22T23:45:47.920 INFO:tasks.workunit.client.0.smithi031.stdout:6/725: dread d3/d2a/d85/d96/fbf [0,4194304] 0 2024-04-22T23:45:47.921 INFO:tasks.workunit.client.0.smithi031.stdout:6/726: write d3/d1f/da3/fd5 [983224,15909] 0 2024-04-22T23:45:47.927 INFO:tasks.workunit.client.0.smithi031.stdout:2/642: dread d6/dc/d49/d83/d85/da9/fad [0,4194304] 0 2024-04-22T23:45:47.927 INFO:tasks.workunit.client.0.smithi031.stdout:2/643: chown d6/dc/d9a/d6c/db2 2087407341 1 2024-04-22T23:45:47.930 INFO:tasks.workunit.client.0.smithi031.stdout:1/659: rmdir d1/d21 39 2024-04-22T23:45:47.948 INFO:tasks.workunit.client.0.smithi031.stdout:0/599: sync 2024-04-22T23:45:47.949 INFO:tasks.workunit.client.0.smithi031.stdout:0/600: readlink d0/d13/d18/l89 0 2024-04-22T23:45:47.949 INFO:tasks.workunit.client.0.smithi031.stdout:0/601: dread - d0/d1/d3f/fa1 zero size 2024-04-22T23:45:47.950 INFO:tasks.workunit.client.0.smithi031.stdout:0/602: chown d0/d1 2765 1 2024-04-22T23:45:47.950 INFO:tasks.workunit.client.0.smithi031.stdout:0/603: fsync d0/d1/d2/d36/d11/d1a/d56/fa2 0 2024-04-22T23:45:47.967 INFO:tasks.workunit.client.0.smithi031.stdout:8/564: write d5/da/d12/d37/d34/d6f/f4b [4197844,42669] 0 2024-04-22T23:45:47.968 INFO:tasks.workunit.client.0.smithi031.stdout:3/574: mknod d4/d54/d2c/d89/d2e/d8d/cc6 0 2024-04-22T23:45:47.970 INFO:tasks.workunit.client.0.smithi031.stdout:3/575: fdatasync d4/d6/d11/d25/d44/d7f/f8f 0 2024-04-22T23:45:47.975 INFO:tasks.workunit.client.0.smithi031.stdout:5/649: stat db/d16/d2a/d59/da8/fac 0 2024-04-22T23:45:47.979 INFO:tasks.workunit.client.0.smithi031.stdout:1/660: write d1/db/d16/d1f/fe2 [3948927,98416] 0 2024-04-22T23:45:47.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/727: dwrite d3/d2a/d3c/db6/ff2 [4194304,4194304] 0 2024-04-22T23:45:47.989 INFO:tasks.workunit.client.0.smithi031.stdout:6/728: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:45:47.990 INFO:tasks.workunit.client.0.smithi031.stdout:6/729: chown d3/d35/db1/de6/ff4 4984 1 2024-04-22T23:45:47.991 INFO:tasks.workunit.client.0.smithi031.stdout:8/565: mknod d5/da/d12/d2e/d4d/ca5 0 2024-04-22T23:45:47.997 INFO:tasks.workunit.client.0.smithi031.stdout:9/641: mkdir d2/dd3 0 2024-04-22T23:45:48.000 INFO:tasks.workunit.client.0.smithi031.stdout:3/576: mkdir d4/d83/dc7 0 2024-04-22T23:45:48.002 INFO:tasks.workunit.client.0.smithi031.stdout:7/641: getdents d9/d14/d1f/d45/da6/dae 0 2024-04-22T23:45:48.009 INFO:tasks.workunit.client.0.smithi031.stdout:4/804: link db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62/fba db/d14/d1d/d7b/d7f/d2a/d47/d80/deb/f112 0 2024-04-22T23:45:48.009 INFO:tasks.workunit.client.0.smithi031.stdout:7/642: dwrite d9/d14/d22/d5c/f80 [0,4194304] 0 2024-04-22T23:45:48.010 INFO:tasks.workunit.client.0.smithi031.stdout:7/643: readlink d9/l4e 0 2024-04-22T23:45:48.013 INFO:tasks.workunit.client.0.smithi031.stdout:1/661: symlink d1/d99/d7d/le5 0 2024-04-22T23:45:48.017 INFO:tasks.workunit.client.0.smithi031.stdout:6/730: mknod d3/d2a/d3c/d40/d7a/c101 0 2024-04-22T23:45:48.019 INFO:tasks.workunit.client.0.smithi031.stdout:8/566: creat d5/da/d12/d37/fa6 x:0 0 0 2024-04-22T23:45:48.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/577: dread d4/d54/d2c/d89/d2e/d45/f92 [0,4194304] 0 2024-04-22T23:45:48.021 INFO:tasks.workunit.client.0.smithi031.stdout:9/642: mkdir d2/dc/d8b/d3d/d40/da4/dd4 0 2024-04-22T23:45:48.022 INFO:tasks.workunit.client.0.smithi031.stdout:3/578: dread d4/d6/f80 [0,4194304] 0 2024-04-22T23:45:48.023 INFO:tasks.workunit.client.0.smithi031.stdout:9/643: write d2/dc/fb7 [193735,86040] 0 2024-04-22T23:45:48.025 INFO:tasks.workunit.client.0.smithi031.stdout:9/644: fdatasync d2/dd/d75/d4a/fba 0 2024-04-22T23:45:48.027 INFO:tasks.workunit.client.0.smithi031.stdout:5/650: mknod db/d16/cd2 0 2024-04-22T23:45:48.029 INFO:tasks.workunit.client.0.smithi031.stdout:5/651: truncate db/d16/d1d/d24/d27/d9e/fa1 22686 0 2024-04-22T23:45:48.032 INFO:tasks.workunit.client.0.smithi031.stdout:4/805: creat db/d14/d1d/d7b/d7f/d2a/d48/f113 x:0 0 0 2024-04-22T23:45:48.036 INFO:tasks.workunit.client.0.smithi031.stdout:7/644: rename d9/d15/d6d/lc3 to d9/d14/d22/dbe/ldb 0 2024-04-22T23:45:48.037 INFO:tasks.workunit.client.0.smithi031.stdout:0/604: dwrite d0/f4f [0,4194304] 0 2024-04-22T23:45:48.040 INFO:tasks.workunit.client.0.smithi031.stdout:6/731: mkdir d3/d35/d75/d7d/dc4/d102 0 2024-04-22T23:45:48.040 INFO:tasks.workunit.client.0.smithi031.stdout:0/605: read d0/f7f [2449179,79919] 0 2024-04-22T23:45:48.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/567: mknod d5/da/d83/d8d/ca7 0 2024-04-22T23:45:48.052 INFO:tasks.workunit.client.0.smithi031.stdout:2/644: link d6/dc/d57/d52/db5/lb8 d6/d45/ld5 0 2024-04-22T23:45:48.052 INFO:tasks.workunit.client.0.smithi031.stdout:2/645: chown d6/dc/d42/fa0 17 1 2024-04-22T23:45:48.053 INFO:tasks.workunit.client.0.smithi031.stdout:2/646: write d6/fc6 [1512878,34084] 0 2024-04-22T23:45:48.053 INFO:tasks.workunit.client.0.smithi031.stdout:5/652: mknod db/d16/d2a/d36/d74/d82/cd3 0 2024-04-22T23:45:48.060 INFO:tasks.workunit.client.0.smithi031.stdout:7/645: rename d9/d14/d1f/d47 to d9/d15/d3f/ddc 0 2024-04-22T23:45:48.064 INFO:tasks.workunit.client.0.smithi031.stdout:6/732: chown d3/d35/d86/l9f 24083 1 2024-04-22T23:45:48.077 INFO:tasks.workunit.client.0.smithi031.stdout:5/653: mknod db/d16/d2a/d36/cd4 0 2024-04-22T23:45:48.079 INFO:tasks.workunit.client.0.smithi031.stdout:5/654: readlink db/d16/d1d/d24/d27/l34 0 2024-04-22T23:45:48.080 INFO:tasks.workunit.client.0.smithi031.stdout:7/646: creat d9/d15/d3f/ddc/fdd x:0 0 0 2024-04-22T23:45:48.082 INFO:tasks.workunit.client.0.smithi031.stdout:1/662: dwrite d1/de/d1e/d34/d41/f62 [0,4194304] 0 2024-04-22T23:45:48.083 INFO:tasks.workunit.client.0.smithi031.stdout:7/647: chown d9/d14/d22/d28/d7a/f95 135 1 2024-04-22T23:45:48.086 INFO:tasks.workunit.client.0.smithi031.stdout:8/568: write d5/da/d12/d37/f78 [827260,15109] 0 2024-04-22T23:45:48.086 INFO:tasks.workunit.client.0.smithi031.stdout:3/579: write d4/d6/d11/d25/d44/f7e [2255008,114706] 0 2024-04-22T23:45:48.089 INFO:tasks.workunit.client.0.smithi031.stdout:2/647: creat d6/dc/d9a/d6c/db2/fd6 x:0 0 0 2024-04-22T23:45:48.090 INFO:tasks.workunit.client.0.smithi031.stdout:7/648: stat d9/d14/d22/d28/d7a/l7b 0 2024-04-22T23:45:48.091 INFO:tasks.workunit.client.0.smithi031.stdout:3/580: dread - d4/d54/d2c/fa8 zero size 2024-04-22T23:45:48.091 INFO:tasks.workunit.client.0.smithi031.stdout:2/648: chown d6/dc/d42/d24/c31 394336306 1 2024-04-22T23:45:48.092 INFO:tasks.workunit.client.0.smithi031.stdout:0/606: link d0/d13/d18/d1e/l97 d0/d13/d18/d6c/dad/lcb 0 2024-04-22T23:45:48.097 INFO:tasks.workunit.client.0.smithi031.stdout:2/649: unlink d6/f1c 0 2024-04-22T23:45:48.097 INFO:tasks.workunit.client.0.smithi031.stdout:7/649: symlink d9/d14/d22/d28/d7a/lde 0 2024-04-22T23:45:48.099 INFO:tasks.workunit.client.0.smithi031.stdout:7/650: chown d9/d15/d6d/fc8 8118171 1 2024-04-22T23:45:48.104 INFO:tasks.workunit.client.0.smithi031.stdout:3/581: mknod d4/d6/d11/cc8 0 2024-04-22T23:45:48.107 INFO:tasks.workunit.client.0.smithi031.stdout:3/582: stat d4/d54/d88/da2/ca7 0 2024-04-22T23:45:48.107 INFO:tasks.workunit.client.0.smithi031.stdout:5/655: getdents db/d16 0 2024-04-22T23:45:48.109 INFO:tasks.workunit.client.0.smithi031.stdout:6/733: dread d3/f17 [0,4194304] 0 2024-04-22T23:45:48.109 INFO:tasks.workunit.client.0.smithi031.stdout:3/583: write d4/d54/d56/f75 [952342,91009] 0 2024-04-22T23:45:48.111 INFO:tasks.workunit.client.0.smithi031.stdout:3/584: fsync d4/d54/d56/f87 0 2024-04-22T23:45:48.111 INFO:tasks.workunit.client.0.smithi031.stdout:8/569: link d5/da/d12/d37/d34/d6f/f6d d5/da/d12/d1d/d38/fa8 0 2024-04-22T23:45:48.112 INFO:tasks.workunit.client.0.smithi031.stdout:3/585: stat d4/f1e 0 2024-04-22T23:45:48.112 INFO:tasks.workunit.client.0.smithi031.stdout:6/734: truncate d3/d2a/d3c/d40/d5b/fc5 777169 0 2024-04-22T23:45:48.113 INFO:tasks.workunit.client.0.smithi031.stdout:9/645: truncate d2/dc/faa 225923 0 2024-04-22T23:45:48.116 INFO:tasks.workunit.client.0.smithi031.stdout:1/663: dread d1/de/d1e/d34/d41/f42 [0,4194304] 0 2024-04-22T23:45:48.118 INFO:tasks.workunit.client.0.smithi031.stdout:4/806: write db/d14/d1d/d7b/d7f/d2a/d47/d80/deb/f112 [123359,73951] 0 2024-04-22T23:45:48.122 INFO:tasks.workunit.client.0.smithi031.stdout:7/651: getdents d9/d14/d1f/d7f 0 2024-04-22T23:45:48.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/586: symlink d4/d54/d2c/d89/lc9 0 2024-04-22T23:45:48.132 INFO:tasks.workunit.client.0.smithi031.stdout:1/664: dwrite d1/d21/d4e/f71 [0,4194304] 0 2024-04-22T23:45:48.133 INFO:tasks.workunit.client.0.smithi031.stdout:2/650: write d6/dc/d42/f2c [3854751,88495] 0 2024-04-22T23:45:48.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/652: creat d9/d14/d22/d5c/d5f/fdf x:0 0 0 2024-04-22T23:45:48.136 INFO:tasks.workunit.client.0.smithi031.stdout:6/735: link d3/f11 d3/d1f/d4c/f103 0 2024-04-22T23:45:48.138 INFO:tasks.workunit.client.0.smithi031.stdout:0/607: dwrite d0/d13/f5b [0,4194304] 0 2024-04-22T23:45:48.141 INFO:tasks.workunit.client.0.smithi031.stdout:9/646: creat d2/dc/d3e/fd5 x:0 0 0 2024-04-22T23:45:48.143 INFO:tasks.workunit.client.0.smithi031.stdout:6/736: chown d3/d2a/d3c/c47 756660 1 2024-04-22T23:45:48.144 INFO:tasks.workunit.client.0.smithi031.stdout:3/587: rmdir d4/d54/d2c/d89 39 2024-04-22T23:45:48.144 INFO:tasks.workunit.client.0.smithi031.stdout:2/651: mknod d6/dc/d48/dc1/cd7 0 2024-04-22T23:45:48.146 INFO:tasks.workunit.client.0.smithi031.stdout:9/647: fdatasync d2/d15/fcc 0 2024-04-22T23:45:48.146 INFO:tasks.workunit.client.0.smithi031.stdout:4/807: dwrite db/d14/d2d/db7/fb8 [0,4194304] 0 2024-04-22T23:45:48.152 INFO:tasks.workunit.client.0.smithi031.stdout:2/652: dread d6/dc/d57/d52/db5/fc0 [4194304,4194304] 0 2024-04-22T23:45:48.155 INFO:tasks.workunit.client.0.smithi031.stdout:6/737: dread d3/d35/fc0 [0,4194304] 0 2024-04-22T23:45:48.156 INFO:tasks.workunit.client.0.smithi031.stdout:3/588: rmdir d4/d83/dc7 0 2024-04-22T23:45:48.156 INFO:tasks.workunit.client.0.smithi031.stdout:0/608: creat d0/d13/d18/d1e/d25/d4c/fcc x:0 0 0 2024-04-22T23:45:48.160 INFO:tasks.workunit.client.0.smithi031.stdout:4/808: dread - db/d14/d2d/d49/d59/fd6 zero size 2024-04-22T23:45:48.162 INFO:tasks.workunit.client.0.smithi031.stdout:2/653: write d6/dc/d17/f3c [2561828,100510] 0 2024-04-22T23:45:48.162 INFO:tasks.workunit.client.0.smithi031.stdout:8/570: truncate d5/da/d12/d37/d34/f5e 1853610 0 2024-04-22T23:45:48.163 INFO:tasks.workunit.client.0.smithi031.stdout:7/653: dread d9/f37 [0,4194304] 0 2024-04-22T23:45:48.163 INFO:tasks.workunit.client.0.smithi031.stdout:1/665: dwrite d1/de/d1e/d34/d41/f42 [0,4194304] 0 2024-04-22T23:45:48.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/666: chown d1/db 77001 1 2024-04-22T23:45:48.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/667: chown d1/db/d16/cd3 41009 1 2024-04-22T23:45:48.167 INFO:tasks.workunit.client.0.smithi031.stdout:3/589: rmdir d4/d6/d11 39 2024-04-22T23:45:48.169 INFO:tasks.workunit.client.0.smithi031.stdout:1/668: chown d1/de/d1e/d34/d66/f92 7480698 1 2024-04-22T23:45:48.171 INFO:tasks.workunit.client.0.smithi031.stdout:9/648: mknod d2/dc/d8b/d3d/d40/da4/cd6 0 2024-04-22T23:45:48.173 INFO:tasks.workunit.client.0.smithi031.stdout:9/649: stat d2/dc/f43 0 2024-04-22T23:45:48.176 INFO:tasks.workunit.client.0.smithi031.stdout:2/654: creat d6/dc/d42/d28/fd8 x:0 0 0 2024-04-22T23:45:48.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/571: creat d5/da/d12/d2e/d4d/d9a/fa9 x:0 0 0 2024-04-22T23:45:48.177 INFO:tasks.workunit.client.0.smithi031.stdout:1/669: dwrite d1/d3f/d54/dba/d28/f2a [0,4194304] 0 2024-04-22T23:45:48.179 INFO:tasks.workunit.client.0.smithi031.stdout:1/670: chown d1/f6 2143095 1 2024-04-22T23:45:48.180 INFO:tasks.workunit.client.0.smithi031.stdout:1/671: stat d1/de/l68 0 2024-04-22T23:45:48.186 INFO:tasks.workunit.client.0.smithi031.stdout:1/672: dwrite d1/db/d16/fcb [0,4194304] 0 2024-04-22T23:45:48.188 INFO:tasks.workunit.client.0.smithi031.stdout:6/738: link d3/l1c d3/d1f/da3/l104 0 2024-04-22T23:45:48.192 INFO:tasks.workunit.client.0.smithi031.stdout:6/739: read d3/f45 [3569045,82310] 0 2024-04-22T23:45:48.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/655: symlink d6/dc/d42/d28/ld9 0 2024-04-22T23:45:48.195 INFO:tasks.workunit.client.0.smithi031.stdout:8/572: mkdir d5/d48/d61/d6b/d6c/daa 0 2024-04-22T23:45:48.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/573: chown d5/da/d12/d2e/d4d/d56 576520 1 2024-04-22T23:45:48.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/650: rename d2/dd/d75/c71 to d2/d15/daf/cd7 0 2024-04-22T23:45:48.204 INFO:tasks.workunit.client.0.smithi031.stdout:6/740: symlink d3/d2a/d3c/db6/l105 0 2024-04-22T23:45:48.208 INFO:tasks.workunit.client.0.smithi031.stdout:0/609: link d0/d1/d3f/l70 d0/d1/d2/d36/d11/lcd 0 2024-04-22T23:45:48.211 INFO:tasks.workunit.client.0.smithi031.stdout:8/574: symlink d5/da/d12/d37/d34/d6f/d85/lab 0 2024-04-22T23:45:48.212 INFO:tasks.workunit.client.0.smithi031.stdout:3/590: creat d4/d6/fca x:0 0 0 2024-04-22T23:45:48.216 INFO:tasks.workunit.client.0.smithi031.stdout:1/673: rename d1/d21/cab to d1/d3f/da1/ce6 0 2024-04-22T23:45:48.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/651: mknod d2/dd/d65/cd8 0 2024-04-22T23:45:48.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/652: readlink d2/l4 0 2024-04-22T23:45:48.220 INFO:tasks.workunit.client.0.smithi031.stdout:6/741: creat d3/d35/db1/f106 x:0 0 0 2024-04-22T23:45:48.222 INFO:tasks.workunit.client.0.smithi031.stdout:2/656: creat d6/dce/fda x:0 0 0 2024-04-22T23:45:48.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/654: getdents d9/d14/d1f/d45/da6 0 2024-04-22T23:45:48.224 INFO:tasks.workunit.client.0.smithi031.stdout:2/657: readlink d6/dc/d48/d4b/d6b/l75 0 2024-04-22T23:45:48.227 INFO:tasks.workunit.client.0.smithi031.stdout:6/742: dwrite d3/d2a/d3c/d65/fb4 [0,4194304] 0 2024-04-22T23:45:48.229 INFO:tasks.workunit.client.0.smithi031.stdout:7/655: read d9/d15/d3a/d41/f83 [885427,106389] 0 2024-04-22T23:45:48.230 INFO:tasks.workunit.client.0.smithi031.stdout:2/658: dread d6/dc/d48/d4b/fb0 [0,4194304] 0 2024-04-22T23:45:48.232 INFO:tasks.workunit.client.0.smithi031.stdout:2/659: write d6/dc/d42/f43 [105345,18763] 0 2024-04-22T23:45:48.233 INFO:tasks.workunit.client.0.smithi031.stdout:0/610: rename d0/d13/d18/l24 to d0/d1/d3f/lce 0 2024-04-22T23:45:48.234 INFO:tasks.workunit.client.0.smithi031.stdout:1/674: rmdir d1/db/d16/d1f/d52/db0 39 2024-04-22T23:45:48.236 INFO:tasks.workunit.client.0.smithi031.stdout:2/660: chown d6/dc/d49/d83/d85/fa1 23017 1 2024-04-22T23:45:48.238 INFO:tasks.workunit.client.0.smithi031.stdout:7/656: dwrite d9/d15/d6d/dcc/fcd [0,4194304] 0 2024-04-22T23:45:48.243 INFO:tasks.workunit.client.0.smithi031.stdout:6/743: readlink d3/d35/d86/ld1 0 2024-04-22T23:45:48.248 INFO:tasks.workunit.client.0.smithi031.stdout:8/575: write d5/d48/f77 [561857,96872] 0 2024-04-22T23:45:48.249 INFO:tasks.workunit.client.0.smithi031.stdout:8/576: fdatasync d5/da/df/d33/f65 0 2024-04-22T23:45:48.250 INFO:tasks.workunit.client.0.smithi031.stdout:0/611: unlink d0/d1/d2/d36/c66 0 2024-04-22T23:45:48.253 INFO:tasks.workunit.client.0.smithi031.stdout:9/653: getdents d2/dd3 0 2024-04-22T23:45:48.253 INFO:tasks.workunit.client.0.smithi031.stdout:0/612: dread d0/f4f [0,4194304] 0 2024-04-22T23:45:48.253 INFO:tasks.workunit.client.0.smithi031.stdout:9/654: stat d2/dd/d75/d4a/c31 0 2024-04-22T23:45:48.255 INFO:tasks.workunit.client.0.smithi031.stdout:9/655: truncate d2/dc/d3e/f99 252718 0 2024-04-22T23:45:48.255 INFO:tasks.workunit.client.0.smithi031.stdout:9/656: dread - d2/d15/d5a/f8d zero size 2024-04-22T23:45:48.255 INFO:tasks.workunit.client.0.smithi031.stdout:7/657: unlink d9/d15/d3a/l42 0 2024-04-22T23:45:48.265 INFO:tasks.workunit.client.0.smithi031.stdout:0/613: creat d0/db5/fcf x:0 0 0 2024-04-22T23:45:48.269 INFO:tasks.workunit.client.0.smithi031.stdout:9/657: mknod d2/dd/d75/d80/cd9 0 2024-04-22T23:45:48.271 INFO:tasks.workunit.client.0.smithi031.stdout:9/658: read d2/dc/fb7 [90793,4722] 0 2024-04-22T23:45:48.275 INFO:tasks.workunit.client.0.smithi031.stdout:0/614: readlink d0/d1/d3f/lce 0 2024-04-22T23:45:48.276 INFO:tasks.workunit.client.0.smithi031.stdout:0/615: readlink d0/d1/d2/d36/d11/d1a/l29 0 2024-04-22T23:45:48.277 INFO:tasks.workunit.client.0.smithi031.stdout:7/658: mkdir d9/d14/d22/d5c/de0 0 2024-04-22T23:45:48.278 INFO:tasks.workunit.client.0.smithi031.stdout:7/659: write d9/d14/d1f/d45/da6/fd4 [246029,81832] 0 2024-04-22T23:45:48.281 INFO:tasks.workunit.client.0.smithi031.stdout:2/661: truncate d6/dc/d57/d52/db5/fc0 4861393 0 2024-04-22T23:45:48.282 INFO:tasks.workunit.client.0.smithi031.stdout:8/577: creat d5/fac x:0 0 0 2024-04-22T23:45:48.285 INFO:tasks.workunit.client.0.smithi031.stdout:6/744: getdents d3/d2a/d3c/d5e 0 2024-04-22T23:45:48.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/660: dwrite d9/d15/dcf/fd2 [0,4194304] 0 2024-04-22T23:45:48.287 INFO:tasks.workunit.client.0.smithi031.stdout:9/659: link d2/d15/f2c d2/dc/d8b/d3d/d40/da4/db1/dc5/fda 0 2024-04-22T23:45:48.288 INFO:tasks.workunit.client.0.smithi031.stdout:2/662: creat d6/dc/d57/fdb x:0 0 0 2024-04-22T23:45:48.289 INFO:tasks.workunit.client.0.smithi031.stdout:9/660: chown d2/dd/d75/d64/d5d/d59/d8e/f9d 0 1 2024-04-22T23:45:48.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/663: write d6/dc/d57/fdb [330282,87449] 0 2024-04-22T23:45:48.291 INFO:tasks.workunit.client.0.smithi031.stdout:9/661: fdatasync d2/dd/d75/d64/fa7 0 2024-04-22T23:45:48.291 INFO:tasks.workunit.client.0.smithi031.stdout:2/664: chown d6/dc/d9a/d6c/f76 5066916 1 2024-04-22T23:45:48.296 INFO:tasks.workunit.client.0.smithi031.stdout:6/745: rename d3/d35/d86/dae to d3/d2a/d3c/d40/d107 0 2024-04-22T23:45:48.300 INFO:tasks.workunit.client.0.smithi031.stdout:0/616: mknod d0/d13/d18/d1e/d25/d8e/da7/dc9/cd0 0 2024-04-22T23:45:48.301 INFO:tasks.workunit.client.0.smithi031.stdout:9/662: fsync d2/dc/f83 0 2024-04-22T23:45:48.303 INFO:tasks.workunit.client.0.smithi031.stdout:9/663: dread - d2/dd/d65/fbf zero size 2024-04-22T23:45:48.304 INFO:tasks.workunit.client.0.smithi031.stdout:6/746: dwrite d3/d77/d8d/fe0 [0,4194304] 0 2024-04-22T23:45:48.307 INFO:tasks.workunit.client.0.smithi031.stdout:7/661: creat d9/d15/fe1 x:0 0 0 2024-04-22T23:45:48.308 INFO:tasks.workunit.client.0.smithi031.stdout:8/578: write d5/da/d12/d37/d34/d6f/d5b/f74 [1638877,24016] 0 2024-04-22T23:45:48.308 INFO:tasks.workunit.client.0.smithi031.stdout:7/662: write d9/d15/d6d/d65/fb7 [1432151,102415] 0 2024-04-22T23:45:48.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/664: rename d2/dc/d8b/d3d/d40/f76 to d2/d15/fdb 0 2024-04-22T23:45:48.351 INFO:tasks.workunit.client.0.smithi031.stdout:6/747: mkdir d3/d1f/d108 0 2024-04-22T23:45:48.351 INFO:tasks.workunit.client.0.smithi031.stdout:5/656: sync 2024-04-22T23:45:48.352 INFO:tasks.workunit.client.0.smithi031.stdout:7/663: mknod d9/d14/d1f/d45/da6/daa/ce2 0 2024-04-22T23:45:48.353 INFO:tasks.workunit.client.0.smithi031.stdout:0/617: link d0/d1/d2/d36/d11/d1a/l29 d0/d13/d18/d1e/d25/d8e/dac/ld1 0 2024-04-22T23:45:48.353 INFO:tasks.workunit.client.0.smithi031.stdout:7/664: fsync d9/d15/d6d/dcc/fd0 0 2024-04-22T23:45:48.355 INFO:tasks.workunit.client.0.smithi031.stdout:9/665: creat d2/d15/d5a/fdc x:0 0 0 2024-04-22T23:45:48.357 INFO:tasks.workunit.client.0.smithi031.stdout:7/665: mkdir d9/d14/d1f/d45/de3 0 2024-04-22T23:45:48.358 INFO:tasks.workunit.client.0.smithi031.stdout:8/579: creat d5/da/d12/d37/d34/d6f/fad x:0 0 0 2024-04-22T23:45:48.359 INFO:tasks.workunit.client.0.smithi031.stdout:8/580: write d5/f9 [47549,83122] 0 2024-04-22T23:45:48.359 INFO:tasks.workunit.client.0.smithi031.stdout:6/748: mknod d3/d2a/d85/d96/c109 0 2024-04-22T23:45:48.361 INFO:tasks.workunit.client.0.smithi031.stdout:9/666: creat d2/dc/d8b/d3d/fdd x:0 0 0 2024-04-22T23:45:48.362 INFO:tasks.workunit.client.0.smithi031.stdout:2/665: dread d6/dc/d42/d28/f2a [4194304,4194304] 0 2024-04-22T23:45:48.363 INFO:tasks.workunit.client.0.smithi031.stdout:9/667: write d2/d15/d5a/f8d [238624,3164] 0 2024-04-22T23:45:48.367 INFO:tasks.workunit.client.0.smithi031.stdout:6/749: mknod d3/d1f/d4c/de2/c10a 0 2024-04-22T23:45:48.370 INFO:tasks.workunit.client.0.smithi031.stdout:7/666: link d9/d14/d1f/d45/da6/daa/fda d9/d15/fe4 0 2024-04-22T23:45:48.371 INFO:tasks.workunit.client.0.smithi031.stdout:2/666: symlink d6/dc/d49/d83/dc2/ldc 0 2024-04-22T23:45:48.372 INFO:tasks.workunit.client.0.smithi031.stdout:9/668: dwrite d2/dc/d8b/d3d/d40/da4/fab [0,4194304] 0 2024-04-22T23:45:48.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/750: creat d3/d2a/d3c/d40/f10b x:0 0 0 2024-04-22T23:45:48.374 INFO:tasks.workunit.client.0.smithi031.stdout:9/669: chown d2/dc/d8b/d3d 192705811 1 2024-04-22T23:45:48.375 INFO:tasks.workunit.client.0.smithi031.stdout:6/751: stat d3/d35/d75 0 2024-04-22T23:45:48.375 INFO:tasks.workunit.client.0.smithi031.stdout:2/667: creat d6/dc/d57/d52/fdd x:0 0 0 2024-04-22T23:45:48.377 INFO:tasks.workunit.client.0.smithi031.stdout:7/667: rename d9/d14/d22/d28/f36 to d9/d15/d3f/ddc/fe5 0 2024-04-22T23:45:48.380 INFO:tasks.workunit.client.0.smithi031.stdout:9/670: write d2/dd/d75/d64/d5d/f89 [3651814,41400] 0 2024-04-22T23:45:48.383 INFO:tasks.workunit.client.0.smithi031.stdout:9/671: symlink d2/dc/d8b/d3d/lde 0 2024-04-22T23:45:48.384 INFO:tasks.workunit.client.0.smithi031.stdout:9/672: chown d2/dd/l6b 366538 1 2024-04-22T23:45:48.385 INFO:tasks.workunit.client.0.smithi031.stdout:2/668: getdents d6/dc/d42/d24/d9c 0 2024-04-22T23:45:48.388 INFO:tasks.workunit.client.0.smithi031.stdout:9/673: creat d2/dc/d8b/fdf x:0 0 0 2024-04-22T23:45:48.389 INFO:tasks.workunit.client.0.smithi031.stdout:0/618: truncate d0/d1/d2/f23 2511282 0 2024-04-22T23:45:48.390 INFO:tasks.workunit.client.0.smithi031.stdout:0/619: write d0/db5/fcf [476507,23411] 0 2024-04-22T23:45:48.403 INFO:tasks.workunit.client.0.smithi031.stdout:9/674: dread d2/dd/d75/d64/d5d/f57 [0,4194304] 0 2024-04-22T23:45:48.404 INFO:tasks.workunit.client.0.smithi031.stdout:9/675: fsync d2/dc/d8b/d3d/d40/f44 0 2024-04-22T23:45:48.406 INFO:tasks.workunit.client.0.smithi031.stdout:9/676: symlink d2/dc/d8b/d3d/d40/da4/le0 0 2024-04-22T23:45:48.407 INFO:tasks.workunit.client.0.smithi031.stdout:9/677: chown d2/d15/d5a/fa5 154 1 2024-04-22T23:45:48.409 INFO:tasks.workunit.client.0.smithi031.stdout:9/678: symlink d2/dd/d75/d64/le1 0 2024-04-22T23:45:48.414 INFO:tasks.workunit.client.0.smithi031.stdout:9/679: link d2/dc/d8b/d3d/d40/l93 d2/dd/le2 0 2024-04-22T23:45:48.415 INFO:tasks.workunit.client.0.smithi031.stdout:9/680: fdatasync d2/d15/d5a/fa5 0 2024-04-22T23:45:48.417 INFO:tasks.workunit.client.0.smithi031.stdout:9/681: mknod d2/dd3/ce3 0 2024-04-22T23:45:48.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:48 smithi031 ceph-mon[26721]: pgmap v30: 65 pgs: 65 active+clean; 1.2 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 147 MiB/s wr, 367 op/s 2024-04-22T23:45:48.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:48 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:48.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:48 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:48.444 INFO:tasks.workunit.client.0.smithi031.stdout:8/581: dread d5/da/d12/d37/d34/f5e [0,4194304] 0 2024-04-22T23:45:48.445 INFO:tasks.workunit.client.0.smithi031.stdout:8/582: fdatasync d5/da/d12/d37/d34/d6f/fad 0 2024-04-22T23:45:48.448 INFO:tasks.workunit.client.0.smithi031.stdout:0/620: write d0/f4f [4600201,53986] 0 2024-04-22T23:45:48.453 INFO:tasks.workunit.client.0.smithi031.stdout:6/752: dwrite d3/f17 [0,4194304] 0 2024-04-22T23:45:48.454 INFO:tasks.workunit.client.0.smithi031.stdout:8/583: rename d5/da/d12/d1d/d38/l7b to d5/da/d83/d8d/lae 0 2024-04-22T23:45:48.455 INFO:tasks.workunit.client.0.smithi031.stdout:6/753: readlink d3/d2a/d3c/d40/d5b/l7c 0 2024-04-22T23:45:48.457 INFO:tasks.workunit.client.0.smithi031.stdout:9/682: fdatasync d2/dd/d75/d64/d5d/d59/f78 0 2024-04-22T23:45:48.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:48 smithi044 ceph-mon[34463]: pgmap v30: 65 pgs: 65 active+clean; 1.2 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 147 MiB/s wr, 367 op/s 2024-04-22T23:45:48.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:48 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:48.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:48 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:48.466 INFO:tasks.workunit.client.0.smithi031.stdout:6/754: getdents d3/da5 0 2024-04-22T23:45:48.467 INFO:tasks.workunit.client.0.smithi031.stdout:7/668: dread d9/d15/f35 [0,4194304] 0 2024-04-22T23:45:48.469 INFO:tasks.workunit.client.0.smithi031.stdout:8/584: dwrite d5/da/fa2 [0,4194304] 0 2024-04-22T23:45:48.470 INFO:tasks.workunit.client.0.smithi031.stdout:7/669: chown d9/d15/d6d/d65/faf 91 1 2024-04-22T23:45:48.471 INFO:tasks.workunit.client.0.smithi031.stdout:6/755: mkdir d3/d2a/d85/d96/d10c 0 2024-04-22T23:45:48.473 INFO:tasks.workunit.client.0.smithi031.stdout:8/585: read d5/da/d12/d37/d34/d6f/f6d [2710071,7278] 0 2024-04-22T23:45:48.473 INFO:tasks.workunit.client.0.smithi031.stdout:6/756: write d3/f1a [2627395,108072] 0 2024-04-22T23:45:48.474 INFO:tasks.workunit.client.0.smithi031.stdout:7/670: symlink d9/le6 0 2024-04-22T23:45:48.475 INFO:tasks.workunit.client.0.smithi031.stdout:6/757: chown d3/d35/d86/ld1 11606223 1 2024-04-22T23:45:48.482 INFO:tasks.workunit.client.0.smithi031.stdout:0/621: truncate d0/d13/d18/d1e/f6e 1124901 0 2024-04-22T23:45:48.484 INFO:tasks.workunit.client.0.smithi031.stdout:6/758: dwrite d3/d2a/d3c/d5e/fda [0,4194304] 0 2024-04-22T23:45:48.484 INFO:tasks.workunit.client.0.smithi031.stdout:9/683: write d2/dd/d75/f5e [82428,48032] 0 2024-04-22T23:45:48.485 INFO:tasks.workunit.client.0.smithi031.stdout:3/591: sync 2024-04-22T23:45:48.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/809: sync 2024-04-22T23:45:48.485 INFO:tasks.workunit.client.0.smithi031.stdout:0/622: write d0/d13/d18/d1e/f6f [3632037,100597] 0 2024-04-22T23:45:48.489 INFO:tasks.workunit.client.0.smithi031.stdout:0/623: creat d0/fd2 x:0 0 0 2024-04-22T23:45:48.490 INFO:tasks.workunit.client.0.smithi031.stdout:6/759: symlink d3/d35/l10d 0 2024-04-22T23:45:48.493 INFO:tasks.workunit.client.0.smithi031.stdout:8/586: truncate d5/da/d83/d8d/f91 3535007 0 2024-04-22T23:45:48.494 INFO:tasks.workunit.client.0.smithi031.stdout:8/587: stat d5/da/d12/d2e/d4d/d56/f88 0 2024-04-22T23:45:48.495 INFO:tasks.workunit.client.0.smithi031.stdout:8/588: write d5/fac [727920,76580] 0 2024-04-22T23:45:48.495 INFO:tasks.workunit.client.0.smithi031.stdout:7/671: unlink d9/d14/d22/d28/d7a/fa8 0 2024-04-22T23:45:48.499 INFO:tasks.workunit.client.0.smithi031.stdout:9/684: rename d2/dc/d8b/d3d/d40/da4/dd4 to d2/dd/d75/d64/d5d/d59/de4 0 2024-04-22T23:45:48.504 INFO:tasks.workunit.client.0.smithi031.stdout:8/589: dwrite d5/da/f59 [4194304,4194304] 0 2024-04-22T23:45:48.510 INFO:tasks.workunit.client.0.smithi031.stdout:9/685: unlink d2/dd/d75/d80/l81 0 2024-04-22T23:45:48.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/686: dwrite d2/dd/d75/d4a/f61 [0,4194304] 0 2024-04-22T23:45:48.522 INFO:tasks.workunit.client.0.smithi031.stdout:4/810: truncate db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/f51 573699 0 2024-04-22T23:45:48.524 INFO:tasks.workunit.client.0.smithi031.stdout:7/672: dread d9/d15/d3f/ddc/f69 [0,4194304] 0 2024-04-22T23:45:48.525 INFO:tasks.workunit.client.0.smithi031.stdout:7/673: chown d9/d14/f75 6 1 2024-04-22T23:45:48.526 INFO:tasks.workunit.client.0.smithi031.stdout:7/674: write d9/d15/f56 [804821,101466] 0 2024-04-22T23:45:48.527 INFO:tasks.workunit.client.0.smithi031.stdout:7/675: dread d9/d14/d22/d28/f82 [0,4194304] 0 2024-04-22T23:45:48.528 INFO:tasks.workunit.client.0.smithi031.stdout:9/687: mkdir d2/d15/daf/de5 0 2024-04-22T23:45:48.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/688: fdatasync d2/d15/d5a/f9f 0 2024-04-22T23:45:48.530 INFO:tasks.workunit.client.0.smithi031.stdout:6/760: rename d3/d2a/d85/d96/f9e to d3/d35/d86/f10e 0 2024-04-22T23:45:48.533 INFO:tasks.workunit.client.0.smithi031.stdout:4/811: unlink db/d14/d2d/d49/d59/d6a/d63/c93 0 2024-04-22T23:45:48.535 INFO:tasks.workunit.client.0.smithi031.stdout:4/812: truncate db/d14/d2d/d49/d59/d6a/d41/fc3 653322 0 2024-04-22T23:45:48.537 INFO:tasks.workunit.client.0.smithi031.stdout:4/813: truncate db/d14/d2d/d49/d59/d6a/fbb 35493 0 2024-04-22T23:45:48.538 INFO:tasks.workunit.client.0.smithi031.stdout:8/590: creat d5/da/d12/d37/faf x:0 0 0 2024-04-22T23:45:48.545 INFO:tasks.workunit.client.0.smithi031.stdout:3/592: rename d4/d54/d88/da2/ca7 to d4/d6/d11/ccb 0 2024-04-22T23:45:48.545 INFO:tasks.workunit.client.0.smithi031.stdout:6/761: rmdir d3/d35/d75/d7d/dc4 39 2024-04-22T23:45:48.545 INFO:tasks.workunit.client.0.smithi031.stdout:3/593: write d4/d54/d6d/faa [731772,108886] 0 2024-04-22T23:45:48.550 INFO:tasks.workunit.client.0.smithi031.stdout:8/591: unlink d5/da/d12/d1d/fa0 0 2024-04-22T23:45:48.550 INFO:tasks.workunit.client.0.smithi031.stdout:6/762: dread d3/d2a/d3c/d40/d107/fd8 [0,4194304] 0 2024-04-22T23:45:48.551 INFO:tasks.workunit.client.0.smithi031.stdout:0/624: write d0/d13/d18/d1e/f6e [112151,115658] 0 2024-04-22T23:45:48.552 INFO:tasks.workunit.client.0.smithi031.stdout:6/763: read d3/f39 [1088505,123360] 0 2024-04-22T23:45:48.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/594: symlink d4/d49/lcc 0 2024-04-22T23:45:48.554 INFO:tasks.workunit.client.0.smithi031.stdout:3/595: rename d4/d6/d11/d25/d44/d7f to d4/d6/d11/d25/d44/d7f/dcd 22 2024-04-22T23:45:48.556 INFO:tasks.workunit.client.0.smithi031.stdout:4/814: unlink db/l97 0 2024-04-22T23:45:48.561 INFO:tasks.workunit.client.0.smithi031.stdout:3/596: chown d4/d54/d2c/d89/d2e 1 1 2024-04-22T23:45:48.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/625: fdatasync d0/d1/d2/d36/d11/d1c/f88 0 2024-04-22T23:45:48.564 INFO:tasks.workunit.client.0.smithi031.stdout:6/764: link d3/d1f/d4c/d8e/fb2 d3/d77/db5/f10f 0 2024-04-22T23:45:48.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/765: readlink d3/d1f/l27 0 2024-04-22T23:45:48.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/766: chown d3/d1f/d4c/d8e/l9c 205455 1 2024-04-22T23:45:48.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/597: dread d4/d54/d2c/d89/d2e/d45/f60 [0,4194304] 0 2024-04-22T23:45:48.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/815: truncate db/d14/fad 2774901 0 2024-04-22T23:45:48.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/598: dread - d4/d54/d6d/f82 zero size 2024-04-22T23:45:48.567 INFO:tasks.workunit.client.0.smithi031.stdout:4/816: stat db/d14/d2d/f35 0 2024-04-22T23:45:48.571 INFO:tasks.workunit.client.0.smithi031.stdout:0/626: mkdir d0/d13/d18/d1e/dd3 0 2024-04-22T23:45:48.572 INFO:tasks.workunit.client.0.smithi031.stdout:0/627: write d0/fbe [497680,3763] 0 2024-04-22T23:45:48.573 INFO:tasks.workunit.client.0.smithi031.stdout:0/628: read - d0/d13/d18/d1e/d25/d8e/fbd zero size 2024-04-22T23:45:48.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/592: getdents d5/da/d12/d2e/d7e 0 2024-04-22T23:45:48.576 INFO:tasks.workunit.client.0.smithi031.stdout:4/817: mkdir db/d14/d2d/d49/d59/d6a/d63/d86/d114 0 2024-04-22T23:45:48.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/599: creat d4/d6/d11/d25/d44/d7f/da5/fce x:0 0 0 2024-04-22T23:45:48.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/600: chown d4/d54/d56/daf 19875889 1 2024-04-22T23:45:48.581 INFO:tasks.workunit.client.0.smithi031.stdout:0/629: chown d0/d13/d18/d1e/f30 4075 1 2024-04-22T23:45:48.582 INFO:tasks.workunit.client.0.smithi031.stdout:4/818: truncate db/d14/d1d/d7b/d7f/d2a/d30/f60 1118284 0 2024-04-22T23:45:48.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/593: mkdir d5/da/d12/d2e/db0 0 2024-04-22T23:45:48.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/594: fdatasync d5/da/d12/f1c 0 2024-04-22T23:45:48.590 INFO:tasks.workunit.client.0.smithi031.stdout:4/819: mknod db/d14/d2d/d49/d59/d83/d9c/d72/dde/c115 0 2024-04-22T23:45:48.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/820: readlink db/d14/d1d/d7b/d7f/l17 0 2024-04-22T23:45:48.593 INFO:tasks.workunit.client.0.smithi031.stdout:3/601: creat d4/d6/d11/dc0/fcf x:0 0 0 2024-04-22T23:45:48.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/821: write db/d14/d1d/d7b/d7f/d2a/d47/d80/fcf [1205526,58060] 0 2024-04-22T23:45:48.596 INFO:tasks.workunit.client.0.smithi031.stdout:8/595: mkdir d5/da/d12/d2e/d4d/d9a/db1 0 2024-04-22T23:45:48.598 INFO:tasks.workunit.client.0.smithi031.stdout:6/767: getdents d3/d1f/d60 0 2024-04-22T23:45:48.598 INFO:tasks.workunit.client.0.smithi031.stdout:6/768: stat d3/d35/fcf 0 2024-04-22T23:45:48.599 INFO:tasks.workunit.client.0.smithi031.stdout:6/769: stat d3/d35/db1/de6/cef 0 2024-04-22T23:45:48.600 INFO:tasks.workunit.client.0.smithi031.stdout:6/770: write d3/d1f/f4f [1485442,123301] 0 2024-04-22T23:45:48.603 INFO:tasks.workunit.client.0.smithi031.stdout:3/602: unlink d4/d6/d22/l24 0 2024-04-22T23:45:48.603 INFO:tasks.workunit.client.0.smithi031.stdout:0/630: dread d0/f7f [0,4194304] 0 2024-04-22T23:45:48.606 INFO:tasks.workunit.client.0.smithi031.stdout:6/771: dread d3/d1f/da3/fd5 [0,4194304] 0 2024-04-22T23:45:48.608 INFO:tasks.workunit.client.0.smithi031.stdout:6/772: dread - d3/d1f/d60/f6a zero size 2024-04-22T23:45:48.610 INFO:tasks.workunit.client.0.smithi031.stdout:4/822: mkdir db/d14/d2d/d111/d116 0 2024-04-22T23:45:48.611 INFO:tasks.workunit.client.0.smithi031.stdout:6/773: write d3/d2a/d3c/d40/d7a/fec [253732,117359] 0 2024-04-22T23:45:48.611 INFO:tasks.workunit.client.0.smithi031.stdout:3/603: dwrite d4/d6/d11/f1c [4194304,4194304] 0 2024-04-22T23:45:48.613 INFO:tasks.workunit.client.0.smithi031.stdout:0/631: creat d0/d1/d2/d36/d11/d1a/d3c/fd4 x:0 0 0 2024-04-22T23:45:48.616 INFO:tasks.workunit.client.0.smithi031.stdout:4/823: write db/d14/d2d/f35 [3802473,109116] 0 2024-04-22T23:45:48.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/604: mknod d4/d6/d11/d25/d44/d7f/cd0 0 2024-04-22T23:45:48.618 INFO:tasks.workunit.client.0.smithi031.stdout:1/675: sync 2024-04-22T23:45:48.618 INFO:tasks.workunit.client.0.smithi031.stdout:5/657: sync 2024-04-22T23:45:48.618 INFO:tasks.workunit.client.0.smithi031.stdout:7/676: sync 2024-04-22T23:45:48.619 INFO:tasks.workunit.client.0.smithi031.stdout:3/605: chown d4/l1a 69461131 1 2024-04-22T23:45:48.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/669: sync 2024-04-22T23:45:48.619 INFO:tasks.workunit.client.0.smithi031.stdout:5/658: chown db/d16 2737158 1 2024-04-22T23:45:48.620 INFO:tasks.workunit.client.0.smithi031.stdout:1/676: dread - d1/d3f/fd4 zero size 2024-04-22T23:45:48.623 INFO:tasks.workunit.client.0.smithi031.stdout:8/596: getdents d5/da/d12/d2e/d4d/d9a 0 2024-04-22T23:45:48.624 INFO:tasks.workunit.client.0.smithi031.stdout:2/670: fdatasync d6/dc/f4e 0 2024-04-22T23:45:48.625 INFO:tasks.workunit.client.0.smithi031.stdout:3/606: rename d4/d54/c46 to d4/d6/d11/d3e/cd1 0 2024-04-22T23:45:48.628 INFO:tasks.workunit.client.0.smithi031.stdout:0/632: symlink d0/d13/d18/d1e/d25/d8e/dac/ld5 0 2024-04-22T23:45:48.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/659: symlink db/d16/d2a/d36/d74/ld5 0 2024-04-22T23:45:48.634 INFO:tasks.workunit.client.0.smithi031.stdout:7/677: dwrite d9/d15/d3a/d41/f8f [0,4194304] 0 2024-04-22T23:45:48.634 INFO:tasks.workunit.client.0.smithi031.stdout:3/607: write d4/d6/d11/d25/d44/d7f/da5/fce [30278,41305] 0 2024-04-22T23:45:48.637 INFO:tasks.workunit.client.0.smithi031.stdout:1/677: dwrite d1/de/d1e/dcf/fa3 [0,4194304] 0 2024-04-22T23:45:48.640 INFO:tasks.workunit.client.0.smithi031.stdout:3/608: write d4/d49/f6f [2732069,47850] 0 2024-04-22T23:45:48.645 INFO:tasks.workunit.client.0.smithi031.stdout:2/671: dwrite d6/dc/d17/d69/d78/fd3 [0,4194304] 0 2024-04-22T23:45:48.647 INFO:tasks.workunit.client.0.smithi031.stdout:6/774: truncate d3/f1a 3235738 0 2024-04-22T23:45:48.647 INFO:tasks.workunit.client.0.smithi031.stdout:2/672: stat d6/dc/d42/d28/fd8 0 2024-04-22T23:45:48.648 INFO:tasks.workunit.client.0.smithi031.stdout:6/775: chown d3/d1f/d4c/df7 0 1 2024-04-22T23:45:48.650 INFO:tasks.workunit.client.0.smithi031.stdout:8/597: mkdir d5/da/d12/d2e/db2 0 2024-04-22T23:45:48.650 INFO:tasks.workunit.client.0.smithi031.stdout:2/673: chown d6/dc/d42/d24/c44 82 1 2024-04-22T23:45:48.655 INFO:tasks.workunit.client.0.smithi031.stdout:5/660: mkdir db/d16/d1d/d24/d58/d5d/dd6 0 2024-04-22T23:45:48.655 INFO:tasks.workunit.client.0.smithi031.stdout:4/824: link db/l29 db/d14/d2d/d111/l117 0 2024-04-22T23:45:48.656 INFO:tasks.workunit.client.0.smithi031.stdout:4/825: chown db/d14/d2d/d49/d59/d6a/d63/d86 936050812 1 2024-04-22T23:45:48.660 INFO:tasks.workunit.client.0.smithi031.stdout:7/678: fsync d9/d15/d6d/d65/f72 0 2024-04-22T23:45:48.662 INFO:tasks.workunit.client.0.smithi031.stdout:1/678: symlink d1/db/le7 0 2024-04-22T23:45:48.665 INFO:tasks.workunit.client.0.smithi031.stdout:0/633: write d0/f7f [723382,67808] 0 2024-04-22T23:45:48.670 INFO:tasks.workunit.client.0.smithi031.stdout:6/776: mkdir d3/d77/db5/d110 0 2024-04-22T23:45:48.671 INFO:tasks.workunit.client.0.smithi031.stdout:6/777: truncate d3/d2a/d3c/db6/fdb 775134 0 2024-04-22T23:45:48.672 INFO:tasks.workunit.client.0.smithi031.stdout:2/674: chown d6/dc/d17/d69/d78/l93 577472043 1 2024-04-22T23:45:48.673 INFO:tasks.workunit.client.0.smithi031.stdout:9/689: sync 2024-04-22T23:45:48.674 INFO:tasks.workunit.client.0.smithi031.stdout:5/661: mknod db/d16/d1d/d24/d27/cd7 0 2024-04-22T23:45:48.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/679: creat d9/d14/fe7 x:0 0 0 2024-04-22T23:45:48.678 INFO:tasks.workunit.client.0.smithi031.stdout:2/675: dwrite d6/dc/d57/f4d [0,4194304] 0 2024-04-22T23:45:48.679 INFO:tasks.workunit.client.0.smithi031.stdout:7/680: chown d9/d14/d22/d5c/d5f/c93 1878 1 2024-04-22T23:45:48.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/609: write d4/d54/d2c/d89/d2e/d45/f6c [4626853,130712] 0 2024-04-22T23:45:48.688 INFO:tasks.workunit.client.0.smithi031.stdout:4/826: unlink db/lac 0 2024-04-22T23:45:48.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/676: unlink d6/dc/d42/d28/fb6 0 2024-04-22T23:45:48.695 INFO:tasks.workunit.client.0.smithi031.stdout:8/598: truncate d5/f9 7573379 0 2024-04-22T23:45:48.699 INFO:tasks.workunit.client.0.smithi031.stdout:6/778: mknod d3/d35/d75/d7d/c111 0 2024-04-22T23:45:48.699 INFO:tasks.workunit.client.0.smithi031.stdout:6/779: chown d3/c3f 65544 1 2024-04-22T23:45:48.699 INFO:tasks.workunit.client.0.smithi031.stdout:9/690: mknod d2/dd/d65/ce6 0 2024-04-22T23:45:48.700 INFO:tasks.workunit.client.0.smithi031.stdout:9/691: chown d2/d9a/fc1 1972 1 2024-04-22T23:45:48.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/662: symlink db/d8d/d9b/ld8 0 2024-04-22T23:45:48.708 INFO:tasks.workunit.client.0.smithi031.stdout:3/610: mknod d4/d54/cd2 0 2024-04-22T23:45:48.710 INFO:tasks.workunit.client.0.smithi031.stdout:0/634: truncate d0/d13/d18/d1e/d25/d8e/da7/f55 2431851 0 2024-04-22T23:45:48.712 INFO:tasks.workunit.client.0.smithi031.stdout:7/681: dwrite d9/d15/d3a/f51 [0,4194304] 0 2024-04-22T23:45:48.713 INFO:tasks.workunit.client.0.smithi031.stdout:1/679: getdents d1/d21/d4b/d6f/da6 0 2024-04-22T23:45:48.716 INFO:tasks.workunit.client.0.smithi031.stdout:2/677: symlink d6/dc/lde 0 2024-04-22T23:45:48.718 INFO:tasks.workunit.client.0.smithi031.stdout:8/599: symlink d5/lb3 0 2024-04-22T23:45:48.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/678: write d6/dc/f1a [527227,79871] 0 2024-04-22T23:45:48.720 INFO:tasks.workunit.client.0.smithi031.stdout:8/600: chown d5/da/d12/d37/faf 303385 1 2024-04-22T23:45:48.720 INFO:tasks.workunit.client.0.smithi031.stdout:0/635: dread d0/d1/d2/d36/d11/d1c/d54/f72 [0,4194304] 0 2024-04-22T23:45:48.721 INFO:tasks.workunit.client.0.smithi031.stdout:2/679: truncate d6/dc/d9a/d6c/f7e 4636756 0 2024-04-22T23:45:48.723 INFO:tasks.workunit.client.0.smithi031.stdout:0/636: fdatasync d0/d13/d18/d1e/d25/d4c/fcc 0 2024-04-22T23:45:48.724 INFO:tasks.workunit.client.0.smithi031.stdout:0/637: truncate d0/db5/fcf 699930 0 2024-04-22T23:45:48.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/680: creat d1/de/d1e/fe8 x:0 0 0 2024-04-22T23:45:48.725 INFO:tasks.workunit.client.0.smithi031.stdout:7/682: mknod d9/d15/d3f/ce8 0 2024-04-22T23:45:48.729 INFO:tasks.workunit.client.0.smithi031.stdout:3/611: symlink d4/d6/ld3 0 2024-04-22T23:45:48.731 INFO:tasks.workunit.client.0.smithi031.stdout:8/601: dwrite d5/da/d12/d2e/d4d/d9a/fa9 [0,4194304] 0 2024-04-22T23:45:48.734 INFO:tasks.workunit.client.0.smithi031.stdout:4/827: getdents db/d14/d2d/d49/d59/d6a/d63/d107/d4f 0 2024-04-22T23:45:48.735 INFO:tasks.workunit.client.0.smithi031.stdout:9/692: rename d2/dd/d75/d4a/l87 to d2/dd/le7 0 2024-04-22T23:45:48.739 INFO:tasks.workunit.client.0.smithi031.stdout:6/780: write d3/d1f/d4c/f103 [4454173,37565] 0 2024-04-22T23:45:48.740 INFO:tasks.workunit.client.0.smithi031.stdout:2/680: rmdir d6/dc/d42/d24 39 2024-04-22T23:45:48.743 INFO:tasks.workunit.client.0.smithi031.stdout:5/663: truncate db/d16/d1d/d24/f5a 3520045 0 2024-04-22T23:45:48.754 INFO:tasks.workunit.client.0.smithi031.stdout:0/638: mkdir d0/d1/d2/d36/d11/d1c/dd6 0 2024-04-22T23:45:48.755 INFO:tasks.workunit.client.0.smithi031.stdout:5/664: readlink db/d16/d2a/d36/d74/lc7 0 2024-04-22T23:45:48.755 INFO:tasks.workunit.client.0.smithi031.stdout:8/602: creat d5/da/d12/d2e/d7e/fb4 x:0 0 0 2024-04-22T23:45:48.755 INFO:tasks.workunit.client.0.smithi031.stdout:7/683: chown d9/d14/d22/d28/d7a/c87 65410 1 2024-04-22T23:45:48.756 INFO:tasks.workunit.client.0.smithi031.stdout:6/781: symlink d3/d1f/d4c/d8e/l112 0 2024-04-22T23:45:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:7/684: chown d9/d14/d1f/d7f/fc7 3 1 2024-04-22T23:45:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:4/828: dread db/d14/d2d/d49/d59/d6a/d63/d107/d4f/f78 [0,4194304] 0 2024-04-22T23:45:48.759 INFO:tasks.workunit.client.0.smithi031.stdout:0/639: symlink d0/ld7 0 2024-04-22T23:45:48.760 INFO:tasks.workunit.client.0.smithi031.stdout:2/681: write d6/f9 [5345750,124327] 0 2024-04-22T23:45:48.762 INFO:tasks.workunit.client.0.smithi031.stdout:5/665: mkdir db/d8d/dd9 0 2024-04-22T23:45:48.763 INFO:tasks.workunit.client.0.smithi031.stdout:2/682: fdatasync d6/dc/d17/d69/f7f 0 2024-04-22T23:45:48.763 INFO:tasks.workunit.client.0.smithi031.stdout:1/681: creat d1/db/d16/fe9 x:0 0 0 2024-04-22T23:45:48.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/640: dwrite d0/d13/d18/d1e/f6e [0,4194304] 0 2024-04-22T23:45:48.770 INFO:tasks.workunit.client.0.smithi031.stdout:0/641: stat d0/d1/d2/d36/d11/d1a 0 2024-04-22T23:45:48.771 INFO:tasks.workunit.client.0.smithi031.stdout:4/829: creat db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/f118 x:0 0 0 2024-04-22T23:45:48.771 INFO:tasks.workunit.client.0.smithi031.stdout:0/642: truncate d0/d13/f81 176981 0 2024-04-22T23:45:48.775 INFO:tasks.workunit.client.0.smithi031.stdout:8/603: creat d5/da/d12/d37/d4e/fb5 x:0 0 0 2024-04-22T23:45:48.778 INFO:tasks.workunit.client.0.smithi031.stdout:6/782: creat d3/d77/db5/d110/f113 x:0 0 0 2024-04-22T23:45:48.780 INFO:tasks.workunit.client.0.smithi031.stdout:9/693: write d2/dd/d75/f7e [909896,92629] 0 2024-04-22T23:45:48.782 INFO:tasks.workunit.client.0.smithi031.stdout:5/666: write db/d16/d2a/f7f [2525367,61303] 0 2024-04-22T23:45:48.785 INFO:tasks.workunit.client.0.smithi031.stdout:6/783: mknod d3/d2a/d3c/db6/c114 0 2024-04-22T23:45:48.786 INFO:tasks.workunit.client.0.smithi031.stdout:1/682: creat d1/db/d16/d1f/d52/db0/fea x:0 0 0 2024-04-22T23:45:48.786 INFO:tasks.workunit.client.0.smithi031.stdout:2/683: mknod d6/dc/d42/d28/d94/dab/cdf 0 2024-04-22T23:45:48.787 INFO:tasks.workunit.client.0.smithi031.stdout:9/694: symlink d2/dc/d8b/d3d/d40/da4/db1/le8 0 2024-04-22T23:45:48.787 INFO:tasks.workunit.client.0.smithi031.stdout:1/683: write d1/d21/fb5 [3389489,88708] 0 2024-04-22T23:45:48.791 INFO:tasks.workunit.client.0.smithi031.stdout:9/695: read d2/dc/f25 [540020,77927] 0 2024-04-22T23:45:48.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/643: rename d0/d1/d2/d36/d11/d1c/dd6 to d0/d1/d2/dd8 0 2024-04-22T23:45:48.793 INFO:tasks.workunit.client.0.smithi031.stdout:5/667: creat db/d16/d2a/dca/fda x:0 0 0 2024-04-22T23:45:48.794 INFO:tasks.workunit.client.0.smithi031.stdout:8/604: dwrite d5/da/d12/d37/d34/d6f/d5b/f8a [0,4194304] 0 2024-04-22T23:45:48.795 INFO:tasks.workunit.client.0.smithi031.stdout:9/696: chown d2/dc/d8b/d3d/d40/da4/db1/dc5 24910771 1 2024-04-22T23:45:48.795 INFO:tasks.workunit.client.0.smithi031.stdout:2/684: creat d6/dc/d42/d28/d94/fe0 x:0 0 0 2024-04-22T23:45:48.797 INFO:tasks.workunit.client.0.smithi031.stdout:8/605: fsync d5/da/d12/d1d/d38/f3f 0 2024-04-22T23:45:48.797 INFO:tasks.workunit.client.0.smithi031.stdout:5/668: chown db/d16/d2a/d36/l69 709 1 2024-04-22T23:45:48.798 INFO:tasks.workunit.client.0.smithi031.stdout:1/684: write d1/db/d16/d1f/faa [676649,37238] 0 2024-04-22T23:45:48.798 INFO:tasks.workunit.client.0.smithi031.stdout:4/830: getdents db/d14/d1d/d7b/d7f/d2a/da1/da7 0 2024-04-22T23:45:48.799 INFO:tasks.workunit.client.0.smithi031.stdout:5/669: stat db/d16/f2b 0 2024-04-22T23:45:48.806 INFO:tasks.workunit.client.0.smithi031.stdout:2/685: creat d6/dc/d48/fe1 x:0 0 0 2024-04-22T23:45:48.809 INFO:tasks.workunit.client.0.smithi031.stdout:5/670: creat db/d16/d2a/d36/fdb x:0 0 0 2024-04-22T23:45:48.809 INFO:tasks.workunit.client.0.smithi031.stdout:4/831: dwrite db/d14/d1d/d7b/d7f/d2a/d48/fec [0,4194304] 0 2024-04-22T23:45:48.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/685: read d1/d3f/d54/d84/f3e [4009324,21699] 0 2024-04-22T23:45:48.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/671: chown db/d16/c71 26360 1 2024-04-22T23:45:48.812 INFO:tasks.workunit.client.0.smithi031.stdout:9/697: fdatasync d2/dd/d75/d64/f55 0 2024-04-22T23:45:48.813 INFO:tasks.workunit.client.0.smithi031.stdout:0/644: rmdir d0/d13/d18/d1e/d25/d8e/dc7 0 2024-04-22T23:45:48.816 INFO:tasks.workunit.client.0.smithi031.stdout:6/784: dwrite d3/da4/fb9 [0,4194304] 0 2024-04-22T23:45:48.820 INFO:tasks.workunit.client.0.smithi031.stdout:6/785: write d3/d1f/fd3 [944035,104172] 0 2024-04-22T23:45:48.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/606: creat d5/da/d12/fb6 x:0 0 0 2024-04-22T23:45:48.825 INFO:tasks.workunit.client.0.smithi031.stdout:2/686: creat d6/dc/d42/d24/fe2 x:0 0 0 2024-04-22T23:45:48.827 INFO:tasks.workunit.client.0.smithi031.stdout:4/832: dwrite db/d14/d1d/dd8/ffe [0,4194304] 0 2024-04-22T23:45:48.830 INFO:tasks.workunit.client.0.smithi031.stdout:0/645: dread d0/d13/d18/d1e/f6e [0,4194304] 0 2024-04-22T23:45:48.831 INFO:tasks.workunit.client.0.smithi031.stdout:5/672: creat db/d16/d1d/d24/d58/d5d/dd6/fdc x:0 0 0 2024-04-22T23:45:48.834 INFO:tasks.workunit.client.0.smithi031.stdout:4/833: stat db/l29 0 2024-04-22T23:45:48.836 INFO:tasks.workunit.client.0.smithi031.stdout:9/698: dwrite d2/dc/d8b/d3d/d40/da4/fab [0,4194304] 0 2024-04-22T23:45:48.837 INFO:tasks.workunit.client.0.smithi031.stdout:5/673: unlink db/d16/d1d/d24/d27/f5b 0 2024-04-22T23:45:48.838 INFO:tasks.workunit.client.0.smithi031.stdout:8/607: symlink d5/da/d12/d37/d34/d6f/lb7 0 2024-04-22T23:45:48.840 INFO:tasks.workunit.client.0.smithi031.stdout:4/834: fdatasync db/d14/d2d/d49/d59/d83/d9c/f88 0 2024-04-22T23:45:48.841 INFO:tasks.workunit.client.0.smithi031.stdout:2/687: creat d6/dc/d48/d4b/fe3 x:0 0 0 2024-04-22T23:45:48.843 INFO:tasks.workunit.client.0.smithi031.stdout:9/699: chown d2/dd/d75/d64/d5d/d59/fb0 207383502 1 2024-04-22T23:45:48.844 INFO:tasks.workunit.client.0.smithi031.stdout:8/608: unlink d5/da/l25 0 2024-04-22T23:45:48.846 INFO:tasks.workunit.client.0.smithi031.stdout:2/688: truncate d6/dc/d9a/d9e/da4/fd0 4931692 0 2024-04-22T23:45:48.855 INFO:tasks.workunit.client.0.smithi031.stdout:2/689: fdatasync d6/f16 0 2024-04-22T23:45:48.858 INFO:tasks.workunit.client.0.smithi031.stdout:4/835: rename db/d14/d2d/d49/d59/d8a/d89/l106 to db/d14/d1d/d7b/d7f/d2a/da1/l119 0 2024-04-22T23:45:48.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/609: creat d5/d48/d61/d6b/d6c/daa/fb8 x:0 0 0 2024-04-22T23:45:48.864 INFO:tasks.workunit.client.0.smithi031.stdout:2/690: truncate d6/dc/d48/d4b/f7d 204630 0 2024-04-22T23:45:48.865 INFO:tasks.workunit.client.0.smithi031.stdout:8/610: unlink d5/da/d12/d37/d34/d6f/fad 0 2024-04-22T23:45:48.866 INFO:tasks.workunit.client.0.smithi031.stdout:0/646: dwrite d0/d13/d18/d1e/f30 [0,4194304] 0 2024-04-22T23:45:48.866 INFO:tasks.workunit.client.0.smithi031.stdout:4/836: unlink db/d14/c1a 0 2024-04-22T23:45:48.867 INFO:tasks.workunit.client.0.smithi031.stdout:8/611: fsync d5/da/d12/d1d/d38/f3f 0 2024-04-22T23:45:48.867 INFO:tasks.workunit.client.0.smithi031.stdout:0/647: read d0/d13/f81 [105159,7799] 0 2024-04-22T23:45:48.870 INFO:tasks.workunit.client.0.smithi031.stdout:8/612: chown d5/da/d12/d37/d34/d6f/d5b/f67 177547934 1 2024-04-22T23:45:48.870 INFO:tasks.workunit.client.0.smithi031.stdout:0/648: dread - d0/d13/d18/d1e/d25/d4c/fc1 zero size 2024-04-22T23:45:48.870 INFO:tasks.workunit.client.0.smithi031.stdout:2/691: mkdir d6/dc/d9a/d6c/de4 0 2024-04-22T23:45:48.871 INFO:tasks.workunit.client.0.smithi031.stdout:4/837: truncate db/d14/d1d/d7b/d7f/d2a/d47/d80/deb/f112 686724 0 2024-04-22T23:45:48.872 INFO:tasks.workunit.client.0.smithi031.stdout:2/692: write d6/dc/d57/f4d [3551612,94526] 0 2024-04-22T23:45:48.874 INFO:tasks.workunit.client.0.smithi031.stdout:4/838: truncate db/d14/d2d/d49/d59/d8a/f10c 181040 0 2024-04-22T23:45:48.874 INFO:tasks.workunit.client.0.smithi031.stdout:8/613: mkdir d5/da/d12/d1d/d38/db9 0 2024-04-22T23:45:48.876 INFO:tasks.workunit.client.0.smithi031.stdout:0/649: rename d0/d13/f5b to d0/d1/d2/dd8/fd9 0 2024-04-22T23:45:48.877 INFO:tasks.workunit.client.0.smithi031.stdout:2/693: chown d6/d45/d50 2058267 1 2024-04-22T23:45:48.878 INFO:tasks.workunit.client.0.smithi031.stdout:4/839: symlink db/d14/d1d/d7b/l11a 0 2024-04-22T23:45:48.883 INFO:tasks.workunit.client.0.smithi031.stdout:4/840: mknod db/d14/c11b 0 2024-04-22T23:45:48.886 INFO:tasks.workunit.client.0.smithi031.stdout:0/650: fdatasync d0/d1/d2/d36/d11/f9e 0 2024-04-22T23:45:48.892 INFO:tasks.workunit.client.0.smithi031.stdout:0/651: rename d0/d13/d18/d1e/d25/d4c to d0/d13/d18/d1e/d25/d8e/dac/dda 0 2024-04-22T23:45:48.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/652: dread - d0/d13/d18/d1e/d25/d8e/da7/d2b/f90 zero size 2024-04-22T23:45:48.897 INFO:tasks.workunit.client.0.smithi031.stdout:4/841: getdents db/d14/d2d/d49/d59/d83/d9c/d72/dde 0 2024-04-22T23:45:48.904 INFO:tasks.workunit.client.0.smithi031.stdout:4/842: rename db/d14/d1d/fd0 to db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/f11c 0 2024-04-22T23:45:48.906 INFO:tasks.workunit.client.0.smithi031.stdout:0/653: getdents d0/db5 0 2024-04-22T23:45:48.916 INFO:tasks.workunit.client.0.smithi031.stdout:4/843: creat db/d14/d2d/d49/d59/d83/d9c/d72/dde/f11d x:0 0 0 2024-04-22T23:45:48.918 INFO:tasks.workunit.client.0.smithi031.stdout:0/654: dwrite d0/d1/d2/d36/d11/d1a/d3c/fd4 [0,4194304] 0 2024-04-22T23:45:48.920 INFO:tasks.workunit.client.0.smithi031.stdout:4/844: fsync db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62/fba 0 2024-04-22T23:45:48.923 INFO:tasks.workunit.client.0.smithi031.stdout:9/700: dwrite d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:48.924 INFO:tasks.workunit.client.0.smithi031.stdout:9/701: chown d2/dc/l94 79 1 2024-04-22T23:45:48.951 INFO:tasks.workunit.client.0.smithi031.stdout:8/614: write d5/d48/d61/d6b/d6c/f73 [202861,83511] 0 2024-04-22T23:45:48.955 INFO:tasks.workunit.client.0.smithi031.stdout:2/694: truncate d6/ff 2275347 0 2024-04-22T23:45:48.958 INFO:tasks.workunit.client.0.smithi031.stdout:3/612: sync 2024-04-22T23:45:48.958 INFO:tasks.workunit.client.0.smithi031.stdout:7/685: sync 2024-04-22T23:45:48.968 INFO:tasks.workunit.client.0.smithi031.stdout:0/655: creat d0/d13/d18/dbf/fdb x:0 0 0 2024-04-22T23:45:48.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/845: symlink db/d14/d2d/d49/d59/d6a/d41/l11e 0 2024-04-22T23:45:48.971 INFO:tasks.workunit.client.0.smithi031.stdout:9/702: creat d2/dd/d75/d4a/da0/fe9 x:0 0 0 2024-04-22T23:45:48.973 INFO:tasks.workunit.client.0.smithi031.stdout:8/615: creat d5/d48/fba x:0 0 0 2024-04-22T23:45:48.977 INFO:tasks.workunit.client.0.smithi031.stdout:7/686: read d9/f4b [616646,33545] 0 2024-04-22T23:45:48.982 INFO:tasks.workunit.client.0.smithi031.stdout:9/703: creat d2/dc/d6d/fea x:0 0 0 2024-04-22T23:45:48.986 INFO:tasks.workunit.client.0.smithi031.stdout:3/613: symlink d4/d6/d4c/d8c/d93/ld4 0 2024-04-22T23:45:48.990 INFO:tasks.workunit.client.0.smithi031.stdout:0/656: fdatasync d0/d13/d18/d1e/d25/f85 0 2024-04-22T23:45:48.991 INFO:tasks.workunit.client.0.smithi031.stdout:0/657: stat d0/d13/d18/d1e/d25/d8e/da7/d2b/l7b 0 2024-04-22T23:45:48.993 INFO:tasks.workunit.client.0.smithi031.stdout:9/704: fsync d2/dc/f48 0 2024-04-22T23:45:48.994 INFO:tasks.workunit.client.0.smithi031.stdout:8/616: mkdir d5/da/d12/d2e/d4d/d9a/db1/dbb 0 2024-04-22T23:45:48.995 INFO:tasks.workunit.client.0.smithi031.stdout:8/617: write d5/da/df/d33/f65 [8356974,98392] 0 2024-04-22T23:45:48.999 INFO:tasks.workunit.client.0.smithi031.stdout:8/618: dread d5/da/f59 [4194304,4194304] 0 2024-04-22T23:45:49.002 INFO:tasks.workunit.client.0.smithi031.stdout:4/846: creat db/d14/d2d/f11f x:0 0 0 2024-04-22T23:45:49.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/614: unlink d4/d54/d2c/d89/fad 0 2024-04-22T23:45:49.012 INFO:tasks.workunit.client.0.smithi031.stdout:8/619: rename d5/da/df/d33/f93 to d5/da/d12/d2e/d4d/fbc 0 2024-04-22T23:45:49.013 INFO:tasks.workunit.client.0.smithi031.stdout:8/620: write d5/da/d12/d37/d34/d6f/d5b/f84 [387798,106014] 0 2024-04-22T23:45:49.014 INFO:tasks.workunit.client.0.smithi031.stdout:8/621: dread - d5/da/d12/d37/d4e/fb5 zero size 2024-04-22T23:45:49.015 INFO:tasks.workunit.client.0.smithi031.stdout:8/622: chown d5/da/d12/d2e/f7d 198438450 1 2024-04-22T23:45:49.016 INFO:tasks.workunit.client.0.smithi031.stdout:0/658: getdents d0/d13/d18/dbf/dc3 0 2024-04-22T23:45:49.017 INFO:tasks.workunit.client.0.smithi031.stdout:0/659: chown d0/d13/d18/d1e/f6f 4486 1 2024-04-22T23:45:49.024 INFO:tasks.workunit.client.0.smithi031.stdout:6/786: sync 2024-04-22T23:45:49.025 INFO:tasks.workunit.client.0.smithi031.stdout:1/686: sync 2024-04-22T23:45:49.025 INFO:tasks.workunit.client.0.smithi031.stdout:6/787: chown d3/d1f/d4c/d8e/c100 422 1 2024-04-22T23:45:49.026 INFO:tasks.workunit.client.0.smithi031.stdout:1/687: write d1/db/d16/d1f/fe2 [4027911,18161] 0 2024-04-22T23:45:49.027 INFO:tasks.workunit.client.0.smithi031.stdout:1/688: fdatasync d1/d21/d4e/fcd 0 2024-04-22T23:45:49.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/695: write d6/dc/d42/d24/f34 [1368087,113702] 0 2024-04-22T23:45:49.037 INFO:tasks.workunit.client.0.smithi031.stdout:4/847: mknod db/d14/d1d/d7b/d7f/d2a/d47/c120 0 2024-04-22T23:45:49.039 INFO:tasks.workunit.client.0.smithi031.stdout:6/788: symlink d3/d2a/l115 0 2024-04-22T23:45:49.042 INFO:tasks.workunit.client.0.smithi031.stdout:1/689: rename d1/d99/d7d/f8a to d1/d21/feb 0 2024-04-22T23:45:49.043 INFO:tasks.workunit.client.0.smithi031.stdout:0/660: mknod d0/d13/d18/d1e/dd3/cdc 0 2024-04-22T23:45:49.047 INFO:tasks.workunit.client.0.smithi031.stdout:6/789: rmdir d3/d2a/d3c/db6 39 2024-04-22T23:45:49.048 INFO:tasks.workunit.client.0.smithi031.stdout:2/696: dread d6/dc/f9f [0,4194304] 0 2024-04-22T23:45:49.048 INFO:tasks.workunit.client.0.smithi031.stdout:6/790: read - d3/d35/db1/de6/ffd zero size 2024-04-22T23:45:49.051 INFO:tasks.workunit.client.0.smithi031.stdout:4/848: rename l0 to db/d14/d1d/d7b/d7f/d2a/d30/l121 0 2024-04-22T23:45:49.053 INFO:tasks.workunit.client.0.smithi031.stdout:7/687: dwrite d9/d14/d22/d28/f82 [0,4194304] 0 2024-04-22T23:45:49.053 INFO:tasks.workunit.client.0.smithi031.stdout:1/690: mkdir d1/d21/d4b/d6f/da6/dec 0 2024-04-22T23:45:49.056 INFO:tasks.workunit.client.0.smithi031.stdout:6/791: symlink d3/d77/l116 0 2024-04-22T23:45:49.074 INFO:tasks.workunit.client.0.smithi031.stdout:8/623: rmdir d5/da/df/d33 39 2024-04-22T23:45:49.074 INFO:tasks.workunit.client.0.smithi031.stdout:5/674: sync 2024-04-22T23:45:49.074 INFO:tasks.workunit.client.0.smithi031.stdout:7/688: dread d9/d14/d1f/d7f/f8c [0,4194304] 0 2024-04-22T23:45:49.075 INFO:tasks.workunit.client.0.smithi031.stdout:1/691: dread d1/de/f11 [0,4194304] 0 2024-04-22T23:45:49.076 INFO:tasks.workunit.client.0.smithi031.stdout:6/792: creat d3/d2a/d3c/d5e/f117 x:0 0 0 2024-04-22T23:45:49.076 INFO:tasks.workunit.client.0.smithi031.stdout:6/793: stat d3/f39 0 2024-04-22T23:45:49.078 INFO:tasks.workunit.client.0.smithi031.stdout:9/705: write d2/dc/f4c [790718,71928] 0 2024-04-22T23:45:49.085 INFO:tasks.workunit.client.0.smithi031.stdout:8/624: rmdir d5/d48/d61/d6b/d6c/daa 39 2024-04-22T23:45:49.085 INFO:tasks.workunit.client.0.smithi031.stdout:8/625: chown d5/c7 155409805 1 2024-04-22T23:45:49.086 INFO:tasks.workunit.client.0.smithi031.stdout:5/675: mkdir db/d16/d2a/d59/d90/ddd 0 2024-04-22T23:45:49.090 INFO:tasks.workunit.client.0.smithi031.stdout:3/615: truncate d4/d54/d6d/f9b 321666 0 2024-04-22T23:45:49.093 INFO:tasks.workunit.client.0.smithi031.stdout:6/794: write d3/d1f/d60/f8f [868862,96611] 0 2024-04-22T23:45:49.094 INFO:tasks.workunit.client.0.smithi031.stdout:9/706: dread d2/dc/f43 [0,4194304] 0 2024-04-22T23:45:49.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/661: rename d0/d1/d2/d36/d11/d1a/l34 to d0/d13/d18/d1e/d25/ldd 0 2024-04-22T23:45:49.098 INFO:tasks.workunit.client.0.smithi031.stdout:4/849: rmdir db/d14/d2d/d111/d116 0 2024-04-22T23:45:49.102 INFO:tasks.workunit.client.0.smithi031.stdout:7/689: mknod d9/d14/d22/d5c/ce9 0 2024-04-22T23:45:49.106 INFO:tasks.workunit.client.0.smithi031.stdout:6/795: mknod d3/d2a/c118 0 2024-04-22T23:45:49.114 INFO:tasks.workunit.client.0.smithi031.stdout:8/626: truncate d5/da/df/f79 1218591 0 2024-04-22T23:45:49.115 INFO:tasks.workunit.client.0.smithi031.stdout:1/692: mknod d1/de/d1e/d34/ced 0 2024-04-22T23:45:49.116 INFO:tasks.workunit.client.0.smithi031.stdout:2/697: write d6/dc/d17/d69/f97 [87568,101532] 0 2024-04-22T23:45:49.118 INFO:tasks.workunit.client.0.smithi031.stdout:1/693: truncate d1/d21/d4b/d6f/da6/fc7 4859039 0 2024-04-22T23:45:49.120 INFO:tasks.workunit.client.0.smithi031.stdout:9/707: creat d2/dd/d65/feb x:0 0 0 2024-04-22T23:45:49.121 INFO:tasks.workunit.client.0.smithi031.stdout:0/662: symlink d0/d1/lde 0 2024-04-22T23:45:49.122 INFO:tasks.workunit.client.0.smithi031.stdout:0/663: chown d0/d1/d2 0 1 2024-04-22T23:45:49.123 INFO:tasks.workunit.client.0.smithi031.stdout:4/850: rmdir db/d14/d1d/d7b/dc1 39 2024-04-22T23:45:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:8/627: unlink d5/da/d12/d1d/d38/f47 0 2024-04-22T23:45:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:7/690: getdents d9/d14/d1f/d45/da6/dae 0 2024-04-22T23:45:49.129 INFO:tasks.workunit.client.0.smithi031.stdout:8/628: chown d5/f87 27 1 2024-04-22T23:45:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:3/616: creat d4/d54/d2c/d89/d2e/d45/fd5 x:0 0 0 2024-04-22T23:45:49.131 INFO:tasks.workunit.client.0.smithi031.stdout:4/851: dwrite db/d14/d2d/d49/dcd/fd3 [0,4194304] 0 2024-04-22T23:45:49.133 INFO:tasks.workunit.client.0.smithi031.stdout:2/698: creat d6/dc/d49/d83/fe5 x:0 0 0 2024-04-22T23:45:49.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/796: dread d3/d2a/d3c/f73 [0,4194304] 0 2024-04-22T23:45:49.141 INFO:tasks.workunit.client.0.smithi031.stdout:7/691: mkdir d9/d15/d6d/d65/dea 0 2024-04-22T23:45:49.142 INFO:tasks.workunit.client.0.smithi031.stdout:2/699: dread d6/dc/d57/f4d [0,4194304] 0 2024-04-22T23:45:49.145 INFO:tasks.workunit.client.0.smithi031.stdout:0/664: dwrite d0/d13/d18/d1e/f6e [4194304,4194304] 0 2024-04-22T23:45:49.147 INFO:tasks.workunit.client.0.smithi031.stdout:3/617: truncate d4/d6/f7 1213399 0 2024-04-22T23:45:49.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/618: chown d4/d6/d4c/c58 1098 1 2024-04-22T23:45:49.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/619: chown d4/d54 1510226 1 2024-04-22T23:45:49.149 INFO:tasks.workunit.client.0.smithi031.stdout:3/620: write d4/d6/fca [892550,68531] 0 2024-04-22T23:45:49.154 INFO:tasks.workunit.client.0.smithi031.stdout:9/708: mknod d2/dd/d75/cec 0 2024-04-22T23:45:49.156 INFO:tasks.workunit.client.0.smithi031.stdout:3/621: dwrite d4/d49/f6f [4194304,4194304] 0 2024-04-22T23:45:49.160 INFO:tasks.workunit.client.0.smithi031.stdout:8/629: creat d5/da/df/d33/fbd x:0 0 0 2024-04-22T23:45:49.162 INFO:tasks.workunit.client.0.smithi031.stdout:2/700: chown d6/dc/d42/d24/d32/fc4 20516 1 2024-04-22T23:45:49.167 INFO:tasks.workunit.client.0.smithi031.stdout:0/665: mknod d0/d13/d18/d1e/d25/d8e/dac/dda/cdf 0 2024-04-22T23:45:49.167 INFO:tasks.workunit.client.0.smithi031.stdout:5/676: dwrite db/f25 [0,4194304] 0 2024-04-22T23:45:49.174 INFO:tasks.workunit.client.0.smithi031.stdout:9/709: symlink d2/d15/daf/led 0 2024-04-22T23:45:49.177 INFO:tasks.workunit.client.0.smithi031.stdout:3/622: rename d4/d6/d11/d3e/f43 to d4/d6/d4c/d8c/d93/fd6 0 2024-04-22T23:45:49.181 INFO:tasks.workunit.client.0.smithi031.stdout:2/701: mkdir d6/dc/d9a/d9e/de6 0 2024-04-22T23:45:49.185 INFO:tasks.workunit.client.0.smithi031.stdout:3/623: dread d4/d54/d6d/faa [0,4194304] 0 2024-04-22T23:45:49.190 INFO:tasks.workunit.client.0.smithi031.stdout:1/694: link d1/d3f/da1/ce6 d1/d21/cee 0 2024-04-22T23:45:49.191 INFO:tasks.workunit.client.0.smithi031.stdout:9/710: mknod d2/d15/daf/cee 0 2024-04-22T23:45:49.194 INFO:tasks.workunit.client.0.smithi031.stdout:6/797: creat d3/d35/d75/d7d/dc4/d102/f119 x:0 0 0 2024-04-22T23:45:49.195 INFO:tasks.workunit.client.0.smithi031.stdout:3/624: dwrite d4/d6/da/fb9 [0,4194304] 0 2024-04-22T23:45:49.196 INFO:tasks.workunit.client.0.smithi031.stdout:2/702: truncate d6/f8b 648915 0 2024-04-22T23:45:49.197 INFO:tasks.workunit.client.0.smithi031.stdout:6/798: chown d3/d35/d75/d7d/dc4/d102 364 1 2024-04-22T23:45:49.207 INFO:tasks.workunit.client.0.smithi031.stdout:3/625: mknod d4/d49/cd7 0 2024-04-22T23:45:49.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/703: dwrite d6/dc/d9a/d6c/f7e [0,4194304] 0 2024-04-22T23:45:49.211 INFO:tasks.workunit.client.0.smithi031.stdout:1/695: write d1/de/d1e/f64 [4601645,12072] 0 2024-04-22T23:45:49.211 INFO:tasks.workunit.client.0.smithi031.stdout:9/711: rename d2/d9a to d2/dc/d8b/d3d/d40/da4/db1/dc5/def 0 2024-04-22T23:45:49.213 INFO:tasks.workunit.client.0.smithi031.stdout:7/692: link d9/d14/d22/d28/c9b d9/ceb 0 2024-04-22T23:45:49.214 INFO:tasks.workunit.client.0.smithi031.stdout:4/852: getdents db/d14/d1d/d7b/dc1 0 2024-04-22T23:45:49.222 INFO:tasks.workunit.client.0.smithi031.stdout:8/630: dwrite d5/da/d12/d2e/d7e/f51 [0,4194304] 0 2024-04-22T23:45:49.224 INFO:tasks.workunit.client.0.smithi031.stdout:3/626: mkdir d4/d83/dd8 0 2024-04-22T23:45:49.226 INFO:tasks.workunit.client.0.smithi031.stdout:3/627: fdatasync d4/d6/d4c/d8c/fa3 0 2024-04-22T23:45:49.226 INFO:tasks.workunit.client.0.smithi031.stdout:2/704: dwrite d6/dc/d17/d69/d78/fd3 [0,4194304] 0 2024-04-22T23:45:49.227 INFO:tasks.workunit.client.0.smithi031.stdout:9/712: symlink d2/dc/d8b/d3d/lf0 0 2024-04-22T23:45:49.228 INFO:tasks.workunit.client.0.smithi031.stdout:3/628: fsync d4/d54/d59/f6e 0 2024-04-22T23:45:49.230 INFO:tasks.workunit.client.0.smithi031.stdout:0/666: truncate d0/d13/d18/d1e/f6a 5217980 0 2024-04-22T23:45:49.230 INFO:tasks.workunit.client.0.smithi031.stdout:6/799: mknod d3/d1f/d108/c11a 0 2024-04-22T23:45:49.230 INFO:tasks.workunit.client.0.smithi031.stdout:1/696: readlink d1/d3f/d54/dba/d28/l78 0 2024-04-22T23:45:49.234 INFO:tasks.workunit.client.0.smithi031.stdout:5/677: getdents db/d8d 0 2024-04-22T23:45:49.235 INFO:tasks.workunit.client.0.smithi031.stdout:1/697: fsync d1/d3f/d54/dba/d28/f2a 0 2024-04-22T23:45:49.240 INFO:tasks.workunit.client.0.smithi031.stdout:8/631: mknod d5/da/d12/d37/d4e/cbe 0 2024-04-22T23:45:49.240 INFO:tasks.workunit.client.0.smithi031.stdout:0/667: readlink d0/d13/d18/d1e/d25/d8e/dac/dda/l5d 0 2024-04-22T23:45:49.242 INFO:tasks.workunit.client.0.smithi031.stdout:6/800: mkdir d3/d2a/d3c/d5e/d11b 0 2024-04-22T23:45:49.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/629: mknod d4/d54/d2c/cd9 0 2024-04-22T23:45:49.252 INFO:tasks.workunit.client.0.smithi031.stdout:5/678: dwrite db/d16/d2a/d59/f75 [0,4194304] 0 2024-04-22T23:45:49.252 INFO:tasks.workunit.client.0.smithi031.stdout:0/668: chown d0/d13/d18/d1e/d25/c53 2695 1 2024-04-22T23:45:49.253 INFO:tasks.workunit.client.0.smithi031.stdout:4/853: creat db/d14/d2d/d49/d59/d83/d9c/d72/f122 x:0 0 0 2024-04-22T23:45:49.255 INFO:tasks.workunit.client.0.smithi031.stdout:1/698: dwrite d1/d3f/d54/dba/f1b [0,4194304] 0 2024-04-22T23:45:49.256 INFO:tasks.workunit.client.0.smithi031.stdout:7/693: link d9/d14/d1f/d45/l62 d9/d15/d3a/d41/lec 0 2024-04-22T23:45:49.257 INFO:tasks.workunit.client.0.smithi031.stdout:0/669: mkdir d0/de0 0 2024-04-22T23:45:49.306 INFO:tasks.workunit.client.0.smithi031.stdout:4/854: chown db/d14/d2d/d49/d59/d6a/d63 137806 1 2024-04-22T23:45:49.306 INFO:tasks.workunit.client.0.smithi031.stdout:2/705: getdents d6/dc/d49/d83/dc2 0 2024-04-22T23:45:49.308 INFO:tasks.workunit.client.0.smithi031.stdout:9/713: write d2/dc/f50 [784989,50786] 0 2024-04-22T23:45:49.309 INFO:tasks.workunit.client.0.smithi031.stdout:8/632: write d5/f76 [3071899,77940] 0 2024-04-22T23:45:49.311 INFO:tasks.workunit.client.0.smithi031.stdout:2/706: fdatasync d6/dc/d48/fe1 0 2024-04-22T23:45:49.311 INFO:tasks.workunit.client.0.smithi031.stdout:0/670: readlink d0/d1/d2/d36/d11/d1a/l29 0 2024-04-22T23:45:49.314 INFO:tasks.workunit.client.0.smithi031.stdout:8/633: write d5/f8 [3510898,64294] 0 2024-04-22T23:45:49.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/801: dread d3/f1a [0,4194304] 0 2024-04-22T23:45:49.321 INFO:tasks.workunit.client.0.smithi031.stdout:2/707: dwrite d6/dc/d42/d24/f39 [4194304,4194304] 0 2024-04-22T23:45:49.321 INFO:tasks.workunit.client.0.smithi031.stdout:1/699: dread d1/db/d16/d2f/f50 [0,4194304] 0 2024-04-22T23:45:49.322 INFO:tasks.workunit.client.0.smithi031.stdout:5/679: link db/d16/d1d/c8c db/d16/d1d/d24/cde 0 2024-04-22T23:45:49.328 INFO:tasks.workunit.client.0.smithi031.stdout:8/634: read d5/da/d12/d2e/d4d/d56/f71 [1392655,44123] 0 2024-04-22T23:45:49.329 INFO:tasks.workunit.client.0.smithi031.stdout:2/708: symlink d6/dc/d49/d83/dc2/le7 0 2024-04-22T23:45:49.331 INFO:tasks.workunit.client.0.smithi031.stdout:4/855: rmdir db/d14/d2d/d49/d59/d6a/d63/d86/d114 0 2024-04-22T23:45:49.332 INFO:tasks.workunit.client.0.smithi031.stdout:5/680: write db/d16/d1d/d24/d27/d9e/fa1 [605003,107832] 0 2024-04-22T23:45:49.333 INFO:tasks.workunit.client.0.smithi031.stdout:1/700: dwrite d1/d3f/d54/dba/f98 [0,4194304] 0 2024-04-22T23:45:49.337 INFO:tasks.workunit.client.0.smithi031.stdout:0/671: getdents d0/d13/d18 0 2024-04-22T23:45:49.340 INFO:tasks.workunit.client.0.smithi031.stdout:6/802: creat d3/d35/d86/f11c x:0 0 0 2024-04-22T23:45:49.342 INFO:tasks.workunit.client.0.smithi031.stdout:7/694: write d9/d14/d1f/d45/da6/daa/db0/fb6 [587020,13622] 0 2024-04-22T23:45:49.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/856: creat db/d14/d2d/d49/d59/d6a/f123 x:0 0 0 2024-04-22T23:45:49.345 INFO:tasks.workunit.client.0.smithi031.stdout:5/681: dwrite db/d16/d1d/d24/fc6 [0,4194304] 0 2024-04-22T23:45:49.346 INFO:tasks.workunit.client.0.smithi031.stdout:0/672: chown d0/d1/d2/l65 28734112 1 2024-04-22T23:45:49.355 INFO:tasks.workunit.client.0.smithi031.stdout:8/635: write d5/da/d12/d2e/d4d/d56/f71 [1885137,94904] 0 2024-04-22T23:45:49.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/673: write d0/d1/d2/dd8/fd9 [3580324,94463] 0 2024-04-22T23:45:49.361 INFO:tasks.workunit.client.0.smithi031.stdout:6/803: write d3/d2a/d3c/d40/d5b/f63 [1078325,17359] 0 2024-04-22T23:45:49.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/857: fsync db/d14/d1d/d7b/d7f/d2a/d30/f60 0 2024-04-22T23:45:49.367 INFO:tasks.workunit.client.0.smithi031.stdout:6/804: mknod d3/d2a/d3c/d40/d5b/c11d 0 2024-04-22T23:45:49.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/858: creat db/d14/d2d/db7/f124 x:0 0 0 2024-04-22T23:45:49.370 INFO:tasks.workunit.client.0.smithi031.stdout:2/709: truncate d6/dc/d17/d69/d78/fd3 598372 0 2024-04-22T23:45:49.371 INFO:tasks.workunit.client.0.smithi031.stdout:2/710: readlink d6/dc/d57/d52/db5/lc8 0 2024-04-22T23:45:49.371 INFO:tasks.workunit.client.0.smithi031.stdout:7/695: getdents d9/d14/d22/dbe 0 2024-04-22T23:45:49.373 INFO:tasks.workunit.client.0.smithi031.stdout:7/696: chown d9/d14/d22/d5c/f9a 355 1 2024-04-22T23:45:49.375 INFO:tasks.workunit.client.0.smithi031.stdout:6/805: rename d3/d35/f59 to d3/d2a/d3c/d65/f11e 0 2024-04-22T23:45:49.378 INFO:tasks.workunit.client.0.smithi031.stdout:2/711: unlink f3 0 2024-04-22T23:45:49.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/674: dwrite d0/d1/d2/d36/d11/d1a/d3c/f59 [0,4194304] 0 2024-04-22T23:45:49.379 INFO:tasks.workunit.client.0.smithi031.stdout:5/682: dread db/d16/d2a/d59/d90/f8e [0,4194304] 0 2024-04-22T23:45:49.382 INFO:tasks.workunit.client.0.smithi031.stdout:0/675: chown d0/d13 549 1 2024-04-22T23:45:49.383 INFO:tasks.workunit.client.0.smithi031.stdout:6/806: creat d3/d35/db1/f11f x:0 0 0 2024-04-22T23:45:49.384 INFO:tasks.workunit.client.0.smithi031.stdout:5/683: chown db/d16/f65 940774025 1 2024-04-22T23:45:49.386 INFO:tasks.workunit.client.0.smithi031.stdout:4/859: dwrite db/d14/d2d/d49/d59/d6a/d63/fa5 [0,4194304] 0 2024-04-22T23:45:49.389 INFO:tasks.workunit.client.0.smithi031.stdout:4/860: dread db/d14/d2d/d49/d59/d6a/fbb [0,4194304] 0 2024-04-22T23:45:49.390 INFO:tasks.workunit.client.0.smithi031.stdout:2/712: rename d6/dc/d9a/d9e to d6/dc/d48/d4b/d6b/de8 0 2024-04-22T23:45:49.394 INFO:tasks.workunit.client.0.smithi031.stdout:6/807: mkdir d3/d1f/d4c/df7/d120 0 2024-04-22T23:45:49.397 INFO:tasks.workunit.client.0.smithi031.stdout:5/684: mkdir db/d16/d1d/d24/ddf 0 2024-04-22T23:45:49.402 INFO:tasks.workunit.client.0.smithi031.stdout:4/861: unlink db/d14/d1d/d7b/d7f/d2a/da1/fdd 0 2024-04-22T23:45:49.404 INFO:tasks.workunit.client.0.smithi031.stdout:4/862: write db/d14/d1d/d7b/d7f/d2a/d47/d80/f95 [1382228,51425] 0 2024-04-22T23:45:49.404 INFO:tasks.workunit.client.0.smithi031.stdout:7/697: truncate d9/d15/d6d/d65/f6b 7945602 0 2024-04-22T23:45:49.406 INFO:tasks.workunit.client.0.smithi031.stdout:2/713: rename d6/dc/lde to d6/dc/d17/d69/d90/le9 0 2024-04-22T23:45:49.407 INFO:tasks.workunit.client.0.smithi031.stdout:2/714: chown d6/dc/d48 2 1 2024-04-22T23:45:49.407 INFO:tasks.workunit.client.0.smithi031.stdout:6/808: write d3/f39 [1788847,44453] 0 2024-04-22T23:45:49.414 INFO:tasks.workunit.client.0.smithi031.stdout:0/676: write d0/d1/d2/d36/d11/d1c/d54/f72 [188904,14193] 0 2024-04-22T23:45:49.416 INFO:tasks.workunit.client.0.smithi031.stdout:6/809: rmdir d3/d1f/d4c 39 2024-04-22T23:45:49.418 INFO:tasks.workunit.client.0.smithi031.stdout:5/685: symlink db/d16/d1d/le0 0 2024-04-22T23:45:49.420 INFO:tasks.workunit.client.0.smithi031.stdout:0/677: dwrite d0/d1/d2/d36/f1d [0,4194304] 0 2024-04-22T23:45:49.422 INFO:tasks.workunit.client.0.smithi031.stdout:2/715: creat d6/dc/d9a/d6c/de4/fea x:0 0 0 2024-04-22T23:45:49.423 INFO:tasks.workunit.client.0.smithi031.stdout:2/716: chown d6/dc/d57/d52/cc5 492624 1 2024-04-22T23:45:49.424 INFO:tasks.workunit.client.0.smithi031.stdout:6/810: write d3/d2a/d3c/db6/fdb [1245032,13450] 0 2024-04-22T23:45:49.425 INFO:tasks.workunit.client.0.smithi031.stdout:5/686: rmdir db/d16/d2a/d59 39 2024-04-22T23:45:49.428 INFO:tasks.workunit.client.0.smithi031.stdout:4/863: write db/d14/d2d/d49/d59/d6a/d63/d107/d40/f68 [1230611,90355] 0 2024-04-22T23:45:49.431 INFO:tasks.workunit.client.0.smithi031.stdout:5/687: write db/d16/d1d/d24/d58/d5d/fa0 [917719,117963] 0 2024-04-22T23:45:49.435 INFO:tasks.workunit.client.0.smithi031.stdout:2/717: link d6/dc/d17/d69/f7f d6/dc/d49/d83/dc2/feb 0 2024-04-22T23:45:49.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/678: dwrite d0/d1/d2/d36/d11/d1a/d56/fae [0,4194304] 0 2024-04-22T23:45:49.451 INFO:tasks.workunit.client.0.smithi031.stdout:5/688: dwrite db/d16/d1d/d24/d58/d5d/f77 [0,4194304] 0 2024-04-22T23:45:49.452 INFO:tasks.workunit.client.0.smithi031.stdout:5/689: fsync db/f29 0 2024-04-22T23:45:49.453 INFO:tasks.workunit.client.0.smithi031.stdout:5/690: stat db/d16/c6c 0 2024-04-22T23:45:49.455 INFO:tasks.workunit.client.0.smithi031.stdout:6/811: dread d3/f15 [4194304,4194304] 0 2024-04-22T23:45:49.458 INFO:tasks.workunit.client.0.smithi031.stdout:5/691: creat db/d16/fe1 x:0 0 0 2024-04-22T23:45:49.462 INFO:tasks.workunit.client.0.smithi031.stdout:3/630: sync 2024-04-22T23:45:49.464 INFO:tasks.workunit.client.0.smithi031.stdout:0/679: write d0/d1/d2/d36/d11/f9e [573657,111499] 0 2024-04-22T23:45:49.467 INFO:tasks.workunit.client.0.smithi031.stdout:3/631: mkdir d4/d6/d11/dda 0 2024-04-22T23:45:49.467 INFO:tasks.workunit.client.0.smithi031.stdout:0/680: stat d0/d13/d18/d1e/f30 0 2024-04-22T23:45:49.469 INFO:tasks.workunit.client.0.smithi031.stdout:4/864: dwrite db/d14/d2d/d49/d59/d6a/d41/f7d [0,4194304] 0 2024-04-22T23:45:49.470 INFO:tasks.workunit.client.0.smithi031.stdout:0/681: fdatasync d0/d1/d3f/fa1 0 2024-04-22T23:45:49.472 INFO:tasks.workunit.client.0.smithi031.stdout:3/632: write d4/d6/d11/f19 [936446,18450] 0 2024-04-22T23:45:49.475 INFO:tasks.workunit.client.0.smithi031.stdout:2/718: dwrite d6/dc/d42/d24/d32/fc4 [0,4194304] 0 2024-04-22T23:45:49.475 INFO:tasks.workunit.client.0.smithi031.stdout:0/682: readlink d0/d1/d2/d36/d11/d1c/d54/la9 0 2024-04-22T23:45:49.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/633: mkdir d4/d6/d4c/ddb 0 2024-04-22T23:45:49.480 INFO:tasks.workunit.client.0.smithi031.stdout:3/634: write d4/d6/d11/f3c [4384573,5001] 0 2024-04-22T23:45:49.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/812: dwrite d3/f45 [4194304,4194304] 0 2024-04-22T23:45:49.483 INFO:tasks.workunit.client.0.smithi031.stdout:6/813: chown d3/d2a/d85/d96/fbf 240 1 2024-04-22T23:45:49.491 INFO:tasks.workunit.client.0.smithi031.stdout:4/865: link db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/c108 db/d14/d2d/d49/d59/d8a/d89/c125 0 2024-04-22T23:45:49.493 INFO:tasks.workunit.client.0.smithi031.stdout:5/692: creat db/d16/d2a/fe2 x:0 0 0 2024-04-22T23:45:49.494 INFO:tasks.workunit.client.0.smithi031.stdout:3/635: link d4/d6/d11/f19 d4/d54/fdc 0 2024-04-22T23:45:49.497 INFO:tasks.workunit.client.0.smithi031.stdout:4/866: rmdir db/d14/d2d/d49/d59/d83/d9c/d72/dde 39 2024-04-22T23:45:49.499 INFO:tasks.workunit.client.0.smithi031.stdout:3/636: creat d4/d6/d11/d3e/fdd x:0 0 0 2024-04-22T23:45:49.500 INFO:tasks.workunit.client.0.smithi031.stdout:6/814: getdents d3/d35/d75 0 2024-04-22T23:45:49.501 INFO:tasks.workunit.client.0.smithi031.stdout:5/693: truncate db/f29 145490 0 2024-04-22T23:45:49.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/694: fdatasync db/d16/d2a/d59/f75 0 2024-04-22T23:45:49.509 INFO:tasks.workunit.client.0.smithi031.stdout:4/867: link db/d14/d2d/d49/d59/d83/d9c/d72/f122 db/d14/d2d/f126 0 2024-04-22T23:45:49.513 INFO:tasks.workunit.client.0.smithi031.stdout:0/683: dread d0/d13/d18/d1e/d25/f82 [0,4194304] 0 2024-04-22T23:45:49.514 INFO:tasks.workunit.client.0.smithi031.stdout:3/637: rmdir d4/d6/d11/d25/dbd 0 2024-04-22T23:45:49.514 INFO:tasks.workunit.client.0.smithi031.stdout:5/695: dwrite db/d16/d2a/d36/d74/d82/f9f [0,4194304] 0 2024-04-22T23:45:49.517 INFO:tasks.workunit.client.0.smithi031.stdout:4/868: mknod db/d14/d2d/d49/d59/d6a/d63/c127 0 2024-04-22T23:45:49.520 INFO:tasks.workunit.client.0.smithi031.stdout:5/696: write db/d16/d1d/d24/d58/d5d/fbd [473531,33118] 0 2024-04-22T23:45:49.523 INFO:tasks.workunit.client.0.smithi031.stdout:6/815: dread d3/d2a/d85/d96/fbf [4194304,4194304] 0 2024-04-22T23:45:49.524 INFO:tasks.workunit.client.0.smithi031.stdout:0/684: creat d0/d13/d18/d1e/d25/d8e/dac/dda/fe1 x:0 0 0 2024-04-22T23:45:49.524 INFO:tasks.workunit.client.0.smithi031.stdout:3/638: dwrite d4/d6/d11/d25/d44/d7f/f69 [0,4194304] 0 2024-04-22T23:45:49.525 INFO:tasks.workunit.client.0.smithi031.stdout:6/816: readlink d3/d1f/l1e 0 2024-04-22T23:45:49.528 INFO:tasks.workunit.client.0.smithi031.stdout:0/685: read d0/d1/d2/d36/d11/d1a/d56/fae [2029343,43384] 0 2024-04-22T23:45:49.530 INFO:tasks.workunit.client.0.smithi031.stdout:5/697: creat db/d16/d1d/d24/d32/fe3 x:0 0 0 2024-04-22T23:45:49.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/686: fdatasync d0/d1/d2/d36/d11/d1a/d56/fa2 0 2024-04-22T23:45:49.532 INFO:tasks.workunit.client.0.smithi031.stdout:0/687: read d0/d1/d2/d36/d11/f43 [3739980,58028] 0 2024-04-22T23:45:49.540 INFO:tasks.workunit.client.0.smithi031.stdout:3/639: mkdir d4/d54/d2c/d89/d2e/d45/dde 0 2024-04-22T23:45:49.545 INFO:tasks.workunit.client.0.smithi031.stdout:0/688: rename d0/d1/d2/d36/d11/f9e to d0/d1/d2/dd8/fe2 0 2024-04-22T23:45:49.549 INFO:tasks.workunit.client.0.smithi031.stdout:4/869: dread db/d14/d1d/fdc [0,4194304] 0 2024-04-22T23:45:49.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/640: rename d4/d54/d2c/d89/d2e/d45/d96 to d4/d6/d11/ddf 0 2024-04-22T23:45:49.557 INFO:tasks.workunit.client.0.smithi031.stdout:6/817: getdents d3/d35/d86 0 2024-04-22T23:45:49.558 INFO:tasks.workunit.client.0.smithi031.stdout:5/698: dread db/d16/d2a/d7c/d89/fb4 [0,4194304] 0 2024-04-22T23:45:49.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/699: fdatasync db/d16/d2a/d7c/fbe 0 2024-04-22T23:45:49.560 INFO:tasks.workunit.client.0.smithi031.stdout:3/641: unlink d4/d54/d59/fb8 0 2024-04-22T23:45:49.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/700: fsync db/d16/d2a/f7f 0 2024-04-22T23:45:49.561 INFO:tasks.workunit.client.0.smithi031.stdout:4/870: creat db/d14/d1d/d7b/d7f/dbf/f128 x:0 0 0 2024-04-22T23:45:49.567 INFO:tasks.workunit.client.0.smithi031.stdout:5/701: rmdir db/d16/d2a/d36/d74 39 2024-04-22T23:45:49.568 INFO:tasks.workunit.client.0.smithi031.stdout:5/702: chown db/d8d/dd9 1 1 2024-04-22T23:45:49.568 INFO:tasks.workunit.client.0.smithi031.stdout:3/642: truncate d4/d6/d11/d25/d44/f71 851875 0 2024-04-22T23:45:49.571 INFO:tasks.workunit.client.0.smithi031.stdout:4/871: rename db/d14/d2d/d49/d59/d83/d9c/d72 to db/d14/d2d/d49/d59/d6a/d63/d107/d129 0 2024-04-22T23:45:49.572 INFO:tasks.workunit.client.0.smithi031.stdout:6/818: truncate d3/d1f/d4c/d8e/f93 1743933 0 2024-04-22T23:45:49.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/819: write d3/d1f/d4c/de2/ffa [173777,7083] 0 2024-04-22T23:45:49.581 INFO:tasks.workunit.client.0.smithi031.stdout:3/643: truncate d4/d54/d59/f6e 1106599 0 2024-04-22T23:45:49.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/820: creat d3/d1f/d4c/df9/f121 x:0 0 0 2024-04-22T23:45:49.584 INFO:tasks.workunit.client.0.smithi031.stdout:4/872: dread db/d14/d1d/f76 [0,4194304] 0 2024-04-22T23:45:49.592 INFO:tasks.workunit.client.0.smithi031.stdout:6/821: rename d3/fe to d3/d2a/d3c/f122 0 2024-04-22T23:45:49.594 INFO:tasks.workunit.client.0.smithi031.stdout:5/703: dwrite db/d16/d1d/f47 [0,4194304] 0 2024-04-22T23:45:49.597 INFO:tasks.workunit.client.0.smithi031.stdout:3/644: write d4/d6/d11/d25/d44/f71 [731474,78453] 0 2024-04-22T23:45:49.605 INFO:tasks.workunit.client.0.smithi031.stdout:6/822: dwrite d3/d2a/d3c/d40/d5b/fc5 [0,4194304] 0 2024-04-22T23:45:49.608 INFO:tasks.workunit.client.0.smithi031.stdout:5/704: creat db/d16/d2a/d36/d74/d82/fe4 x:0 0 0 2024-04-22T23:45:49.608 INFO:tasks.workunit.client.0.smithi031.stdout:9/714: sync 2024-04-22T23:45:49.611 INFO:tasks.workunit.client.0.smithi031.stdout:4/873: dwrite db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/f51 [0,4194304] 0 2024-04-22T23:45:49.613 INFO:tasks.workunit.client.0.smithi031.stdout:3/645: dread d4/d6/da/fb9 [0,4194304] 0 2024-04-22T23:45:49.613 INFO:tasks.workunit.client.0.smithi031.stdout:3/646: write d4/d6/da/f67 [1315937,35269] 0 2024-04-22T23:45:49.614 INFO:tasks.workunit.client.0.smithi031.stdout:9/715: mkdir d2/dc/d6d/df1 0 2024-04-22T23:45:49.617 INFO:tasks.workunit.client.0.smithi031.stdout:9/716: chown d2/dc/d3e/c67 8364 1 2024-04-22T23:45:49.619 INFO:tasks.workunit.client.0.smithi031.stdout:3/647: mkdir d4/d54/d88/da2/de0 0 2024-04-22T23:45:49.619 INFO:tasks.workunit.client.0.smithi031.stdout:5/705: creat db/d16/d1d/fe5 x:0 0 0 2024-04-22T23:45:49.620 INFO:tasks.workunit.client.0.smithi031.stdout:9/717: truncate d2/dc/f1e 2075668 0 2024-04-22T23:45:49.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/648: fsync d4/d54/d2c/d89/d2e/f39 0 2024-04-22T23:45:49.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/649: chown d4/d83 13866106 1 2024-04-22T23:45:49.625 INFO:tasks.workunit.client.0.smithi031.stdout:9/718: mknod d2/d15/cf2 0 2024-04-22T23:45:49.631 INFO:tasks.workunit.client.0.smithi031.stdout:9/719: dread d2/d15/d5a/f8d [0,4194304] 0 2024-04-22T23:45:49.635 INFO:tasks.workunit.client.0.smithi031.stdout:9/720: creat d2/dd/d75/d4a/da0/dd0/ff3 x:0 0 0 2024-04-22T23:45:49.635 INFO:tasks.workunit.client.0.smithi031.stdout:3/650: write d4/d54/fdc [1562702,38465] 0 2024-04-22T23:45:49.636 INFO:tasks.workunit.client.0.smithi031.stdout:5/706: write db/d16/d1d/d24/d58/f4f [468733,118084] 0 2024-04-22T23:45:49.638 INFO:tasks.workunit.client.0.smithi031.stdout:6/823: dread d3/d1f/f64 [0,4194304] 0 2024-04-22T23:45:49.640 INFO:tasks.workunit.client.0.smithi031.stdout:9/721: creat d2/dc/d8b/d3d/d40/da4/ff4 x:0 0 0 2024-04-22T23:45:49.641 INFO:tasks.workunit.client.0.smithi031.stdout:5/707: creat db/d16/d1d/d24/d32/fe6 x:0 0 0 2024-04-22T23:45:49.641 INFO:tasks.workunit.client.0.smithi031.stdout:9/722: dread - d2/d15/d5a/f9f zero size 2024-04-22T23:45:49.641 INFO:tasks.workunit.client.0.smithi031.stdout:6/824: read d3/d2a/d3c/db6/ff2 [984808,70412] 0 2024-04-22T23:45:49.642 INFO:tasks.workunit.client.0.smithi031.stdout:3/651: write d4/d6/d11/d25/f36 [286514,110391] 0 2024-04-22T23:45:49.646 INFO:tasks.workunit.client.0.smithi031.stdout:6/825: rename d3/f1a to d3/d35/d75/d7d/dc4/f123 0 2024-04-22T23:45:49.647 INFO:tasks.workunit.client.0.smithi031.stdout:9/723: truncate d2/dc/d8b/f58 439575 0 2024-04-22T23:45:49.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/724: chown d2/dc/d3e/f56 148 1 2024-04-22T23:45:49.649 INFO:tasks.workunit.client.0.smithi031.stdout:3/652: rename d4/d6/d11/d25/d44/d7f/d5f/db4 to d4/d54/d88/de1 0 2024-04-22T23:45:49.649 INFO:tasks.workunit.client.0.smithi031.stdout:6/826: rmdir d3/d35/d75 39 2024-04-22T23:45:49.650 INFO:tasks.workunit.client.0.smithi031.stdout:5/708: getdents db/d16/d2a/d59/d90/da4 0 2024-04-22T23:45:49.651 INFO:tasks.workunit.client.0.smithi031.stdout:3/653: read d4/d54/fdc [2998534,58399] 0 2024-04-22T23:45:49.651 INFO:tasks.workunit.client.0.smithi031.stdout:6/827: fdatasync d3/d1f/d4c/de2/ffa 0 2024-04-22T23:45:49.656 INFO:tasks.workunit.client.0.smithi031.stdout:5/709: creat db/d16/d1d/d24/d58/d5d/fe7 x:0 0 0 2024-04-22T23:45:49.660 INFO:tasks.workunit.client.0.smithi031.stdout:6/828: rename d3/d35/d75/d7d/dc4/d102 to d3/d35/d75/d7d/dc4/d124 0 2024-04-22T23:45:49.661 INFO:tasks.workunit.client.0.smithi031.stdout:3/654: unlink d4/d54/d2c/d89/l99 0 2024-04-22T23:45:49.662 INFO:tasks.workunit.client.0.smithi031.stdout:3/655: truncate d4/d54/d56/f75 4435165 0 2024-04-22T23:45:49.666 INFO:tasks.workunit.client.0.smithi031.stdout:6/829: dwrite d3/d1f/d60/f8f [0,4194304] 0 2024-04-22T23:45:49.668 INFO:tasks.workunit.client.0.smithi031.stdout:9/725: link d2/dc/l23 d2/dc/d8b/d3d/d40/da4/db1/lf5 0 2024-04-22T23:45:49.670 INFO:tasks.workunit.client.0.smithi031.stdout:6/830: dread d3/d35/d86/f10e [4194304,4194304] 0 2024-04-22T23:45:49.676 INFO:tasks.workunit.client.0.smithi031.stdout:5/710: rename db/d16/d1d/d24/d27/d9e/fa5 to db/d16/d2a/d36/d74/fe8 0 2024-04-22T23:45:49.680 INFO:tasks.workunit.client.0.smithi031.stdout:6/831: dread - d3/d2a/d3c/d65/f11e zero size 2024-04-22T23:45:49.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/656: creat d4/d54/d2c/d89/d2e/d45/fe2 x:0 0 0 2024-04-22T23:45:49.685 INFO:tasks.workunit.client.0.smithi031.stdout:6/832: rename d3/d2a/d3c/d65/l99 to d3/d2a/d3c/d40/d5b/dfe/l125 0 2024-04-22T23:45:49.691 INFO:tasks.workunit.client.0.smithi031.stdout:6/833: rename d3/d2a/d3c/d40/d5b/c9b to d3/d1f/d4c/c126 0 2024-04-22T23:45:49.696 INFO:tasks.workunit.client.0.smithi031.stdout:1/701: sync 2024-04-22T23:45:49.696 INFO:tasks.workunit.client.0.smithi031.stdout:7/698: sync 2024-04-22T23:45:49.696 INFO:tasks.workunit.client.0.smithi031.stdout:8/636: sync 2024-04-22T23:45:49.696 INFO:tasks.workunit.client.0.smithi031.stdout:2/719: sync 2024-04-22T23:45:49.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/834: unlink d3/d35/d75/d7d/dc4/cd6 0 2024-04-22T23:45:49.698 INFO:tasks.workunit.client.0.smithi031.stdout:9/726: truncate d2/dd/d75/d64/d5d/d59/fb0 1918116 0 2024-04-22T23:45:49.700 INFO:tasks.workunit.client.0.smithi031.stdout:1/702: rmdir d1/d99/d63 39 2024-04-22T23:45:49.701 INFO:tasks.workunit.client.0.smithi031.stdout:6/835: fdatasync d3/d35/d75/d7d/dc4/d124/f119 0 2024-04-22T23:45:49.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/699: mknod d9/d14/d1f/d45/da6/daa/dbb/ced 0 2024-04-22T23:45:49.703 INFO:tasks.workunit.client.0.smithi031.stdout:3/657: dread d4/d54/d2c/d89/f62 [0,4194304] 0 2024-04-22T23:45:49.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/700: stat d9/d15/dbf/lce 0 2024-04-22T23:45:49.704 INFO:tasks.workunit.client.0.smithi031.stdout:8/637: rename d5/da/f1e to d5/da/d12/d37/d34/d6f/d5b/fbf 0 2024-04-22T23:45:49.706 INFO:tasks.workunit.client.0.smithi031.stdout:9/727: symlink d2/dd/d65/lf6 0 2024-04-22T23:45:49.707 INFO:tasks.workunit.client.0.smithi031.stdout:9/728: chown d2/dd/d75/d64/f5f 824553610 1 2024-04-22T23:45:49.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/729: fsync d2/dc/d8b/d3d/d40/da4/db1/dc5/def/fc1 0 2024-04-22T23:45:49.711 INFO:tasks.workunit.client.0.smithi031.stdout:3/658: mkdir d4/d6/d11/de3 0 2024-04-22T23:45:49.714 INFO:tasks.workunit.client.0.smithi031.stdout:6/836: dread d3/d35/d75/fe3 [0,4194304] 0 2024-04-22T23:45:49.714 INFO:tasks.workunit.client.0.smithi031.stdout:7/701: dread d9/d14/d22/d28/d7a/f95 [0,4194304] 0 2024-04-22T23:45:49.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/703: creat d1/d3f/d54/dc6/fef x:0 0 0 2024-04-22T23:45:49.719 INFO:tasks.workunit.client.0.smithi031.stdout:3/659: mknod d4/d6/d22/ce4 0 2024-04-22T23:45:49.722 INFO:tasks.workunit.client.0.smithi031.stdout:2/720: dwrite d6/dc/d49/f70 [0,4194304] 0 2024-04-22T23:45:49.732 INFO:tasks.workunit.client.0.smithi031.stdout:9/730: rename d2/dd/d75/d64/d5d/d59/de4 to d2/dd/d75/df7 0 2024-04-22T23:45:49.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/660: dread d4/d6/d4c/d8c/d93/fd6 [0,4194304] 0 2024-04-22T23:45:49.734 INFO:tasks.workunit.client.0.smithi031.stdout:6/837: dread d3/d77/d8d/fe0 [0,4194304] 0 2024-04-22T23:45:49.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/721: fdatasync d6/dc/d42/d28/f37 0 2024-04-22T23:45:49.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/731: mkdir d2/dd/d75/d80/df8 0 2024-04-22T23:45:49.742 INFO:tasks.workunit.client.0.smithi031.stdout:3/661: rmdir d4/d54/d56 39 2024-04-22T23:45:49.745 INFO:tasks.workunit.client.0.smithi031.stdout:6/838: write d3/d77/db5/fd2 [419327,122943] 0 2024-04-22T23:45:49.745 INFO:tasks.workunit.client.0.smithi031.stdout:7/702: dwrite d9/f4b [0,4194304] 0 2024-04-22T23:45:49.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/722: symlink d6/dc/d42/d24/dd4/lec 0 2024-04-22T23:45:49.755 INFO:tasks.workunit.client.0.smithi031.stdout:9/732: dwrite d2/dc/d8b/d3d/d40/da4/ff4 [0,4194304] 0 2024-04-22T23:45:49.755 INFO:tasks.workunit.client.0.smithi031.stdout:3/662: fsync d4/d6/f7 0 2024-04-22T23:45:49.756 INFO:tasks.workunit.client.0.smithi031.stdout:2/723: stat d6/dc/d42/d24/dd4/lec 0 2024-04-22T23:45:49.757 INFO:tasks.workunit.client.0.smithi031.stdout:6/839: dread d3/d1f/d60/f8f [0,4194304] 0 2024-04-22T23:45:49.758 INFO:tasks.workunit.client.0.smithi031.stdout:7/703: mkdir d9/d14/d22/d5c/dee 0 2024-04-22T23:45:49.759 INFO:tasks.workunit.client.0.smithi031.stdout:2/724: write d6/dc/d48/d4b/d6b/f6f [3994051,78373] 0 2024-04-22T23:45:49.761 INFO:tasks.workunit.client.0.smithi031.stdout:3/663: creat d4/d6/d11/d25/d44/d7f/d5f/fe5 x:0 0 0 2024-04-22T23:45:49.765 INFO:tasks.workunit.client.0.smithi031.stdout:2/725: fsync d6/dc/d9a/d6c/f76 0 2024-04-22T23:45:49.768 INFO:tasks.workunit.client.0.smithi031.stdout:3/664: mkdir d4/d83/dd8/de6 0 2024-04-22T23:45:49.775 INFO:tasks.workunit.client.0.smithi031.stdout:2/726: creat d6/d45/d50/fed x:0 0 0 2024-04-22T23:45:49.775 INFO:tasks.workunit.client.0.smithi031.stdout:7/704: dwrite d9/d14/d1f/d7f/fc7 [0,4194304] 0 2024-04-22T23:45:49.778 INFO:tasks.workunit.client.0.smithi031.stdout:9/733: dwrite d2/dc/d8b/d3d/d40/da4/fab [0,4194304] 0 2024-04-22T23:45:49.781 INFO:tasks.workunit.client.0.smithi031.stdout:6/840: dwrite d3/d77/db5/d110/f113 [0,4194304] 0 2024-04-22T23:45:49.782 INFO:tasks.workunit.client.0.smithi031.stdout:2/727: symlink d6/dc/d17/d69/d90/lee 0 2024-04-22T23:45:49.783 INFO:tasks.workunit.client.0.smithi031.stdout:3/665: link d4/d6/d22/l70 d4/d6/d22/le7 0 2024-04-22T23:45:49.786 INFO:tasks.workunit.client.0.smithi031.stdout:3/666: chown d4/d6/d11/d25/d44/d7f/d5f/c95 6404971 1 2024-04-22T23:45:49.787 INFO:tasks.workunit.client.0.smithi031.stdout:6/841: mknod d3/d35/d86/c127 0 2024-04-22T23:45:49.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/728: creat d6/dc/d48/d8a/fef x:0 0 0 2024-04-22T23:45:49.791 INFO:tasks.workunit.client.0.smithi031.stdout:6/842: write d3/d2a/d3c/d40/d5b/fc5 [4874452,44188] 0 2024-04-22T23:45:49.791 INFO:tasks.workunit.client.0.smithi031.stdout:9/734: mkdir d2/dc/d8b/d3d/df9 0 2024-04-22T23:45:49.794 INFO:tasks.workunit.client.0.smithi031.stdout:7/705: mknod d9/d14/d22/cef 0 2024-04-22T23:45:49.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/735: read d2/f20 [2684120,106677] 0 2024-04-22T23:45:49.801 INFO:tasks.workunit.client.0.smithi031.stdout:3/667: dwrite d4/d6/f23 [0,4194304] 0 2024-04-22T23:45:49.806 INFO:tasks.workunit.client.0.smithi031.stdout:2/729: read d6/ff [1423979,44745] 0 2024-04-22T23:45:49.806 INFO:tasks.workunit.client.0.smithi031.stdout:7/706: rename d9/d14/d1f/d45/da6/daa/fda to d9/d15/d6d/da9/ff0 0 2024-04-22T23:45:49.806 INFO:tasks.workunit.client.0.smithi031.stdout:6/843: dwrite d3/d77/d8d/fa0 [0,4194304] 0 2024-04-22T23:45:49.810 INFO:tasks.workunit.client.0.smithi031.stdout:6/844: readlink d3/d2a/d3c/db6/lbe 0 2024-04-22T23:45:49.817 INFO:tasks.workunit.client.0.smithi031.stdout:2/730: rename d6/dc/d42/d28 to d6/dc/d9a/d6c/db2/df0 0 2024-04-22T23:45:49.818 INFO:tasks.workunit.client.0.smithi031.stdout:2/731: readlink d6/dc/d9a/d6c/db2/df0/ld9 0 2024-04-22T23:45:49.819 INFO:tasks.workunit.client.0.smithi031.stdout:2/732: fdatasync d6/dc/d17/d69/fa8 0 2024-04-22T23:45:49.820 INFO:tasks.workunit.client.0.smithi031.stdout:0/689: sync 2024-04-22T23:45:49.821 INFO:tasks.workunit.client.0.smithi031.stdout:4/874: sync 2024-04-22T23:45:49.822 INFO:tasks.workunit.client.0.smithi031.stdout:3/668: creat d4/d54/fe8 x:0 0 0 2024-04-22T23:45:49.822 INFO:tasks.workunit.client.0.smithi031.stdout:0/690: dread - d0/d13/d18/d1e/d25/d8e/dac/dda/fe1 zero size 2024-04-22T23:45:49.824 INFO:tasks.workunit.client.0.smithi031.stdout:9/736: rmdir d2/dd/d75/d64/d5d/dc3 0 2024-04-22T23:45:49.825 INFO:tasks.workunit.client.0.smithi031.stdout:7/707: mknod d9/d14/d1f/d45/da6/daa/dbb/dd8/cf1 0 2024-04-22T23:45:49.827 INFO:tasks.workunit.client.0.smithi031.stdout:4/875: dwrite db/d14/d2d/f35 [0,4194304] 0 2024-04-22T23:45:49.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/733: fdatasync d6/dc/d57/d52/f6a 0 2024-04-22T23:45:49.830 INFO:tasks.workunit.client.0.smithi031.stdout:2/734: chown d6/dc/d48/d8a 487270 1 2024-04-22T23:45:49.830 INFO:tasks.workunit.client.0.smithi031.stdout:3/669: creat d4/d54/d88/da2/fe9 x:0 0 0 2024-04-22T23:45:49.832 INFO:tasks.workunit.client.0.smithi031.stdout:0/691: fdatasync d0/d1/d2/dd8/fe2 0 2024-04-22T23:45:49.839 INFO:tasks.workunit.client.0.smithi031.stdout:3/670: creat d4/d54/d6d/d98/fea x:0 0 0 2024-04-22T23:45:49.842 INFO:tasks.workunit.client.0.smithi031.stdout:3/671: chown d4/d6/d11/d25/d44/d7f/d5f/c95 1 1 2024-04-22T23:45:49.842 INFO:tasks.workunit.client.0.smithi031.stdout:6/845: getdents d3/d35/d75/d7d/dc4 0 2024-04-22T23:45:49.843 INFO:tasks.workunit.client.0.smithi031.stdout:6/846: readlink d3/d1f/l27 0 2024-04-22T23:45:49.844 INFO:tasks.workunit.client.0.smithi031.stdout:3/672: chown d4/d6/d11/d25/d44/f8e 2412030 1 2024-04-22T23:45:49.845 INFO:tasks.workunit.client.0.smithi031.stdout:6/847: chown d3/d2a/d3c/db6/ff2 487276391 1 2024-04-22T23:45:49.850 INFO:tasks.workunit.client.0.smithi031.stdout:7/708: link d9/d15/d3f/ddc/f69 d9/d14/d1f/d7f/ff2 0 2024-04-22T23:45:49.850 INFO:tasks.workunit.client.0.smithi031.stdout:0/692: write d0/d1/d2/d36/d11/f14 [3581160,1168] 0 2024-04-22T23:45:49.850 INFO:tasks.workunit.client.0.smithi031.stdout:2/735: creat d6/dc/ff1 x:0 0 0 2024-04-22T23:45:49.852 INFO:tasks.workunit.client.0.smithi031.stdout:0/693: stat d0/d1/d2/dd8/fe2 0 2024-04-22T23:45:49.853 INFO:tasks.workunit.client.0.smithi031.stdout:3/673: rename d4/d54/d2c/d89/d2e/d45/f92 to d4/d6/d22/feb 0 2024-04-22T23:45:49.854 INFO:tasks.workunit.client.0.smithi031.stdout:7/709: chown d9/d14/d1f/d45/da6/daa/lbd 45 1 2024-04-22T23:45:49.854 INFO:tasks.workunit.client.0.smithi031.stdout:9/737: truncate d2/dd/fa2 3907141 0 2024-04-22T23:45:49.856 INFO:tasks.workunit.client.0.smithi031.stdout:6/848: dwrite d3/d1f/d60/f84 [0,4194304] 0 2024-04-22T23:45:49.857 INFO:tasks.workunit.client.0.smithi031.stdout:2/736: mknod d6/dc/cf2 0 2024-04-22T23:45:49.858 INFO:tasks.workunit.client.0.smithi031.stdout:4/876: dread db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62/f77 [0,4194304] 0 2024-04-22T23:45:49.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/694: rmdir d0/d1/d2/d36/d11/d1c 39 2024-04-22T23:45:49.860 INFO:tasks.workunit.client.0.smithi031.stdout:3/674: dread d4/d6/d11/d25/d44/d7f/d5f/f9d [0,4194304] 0 2024-04-22T23:45:49.860 INFO:tasks.workunit.client.0.smithi031.stdout:7/710: mkdir d9/d15/df3 0 2024-04-22T23:45:49.863 INFO:tasks.workunit.client.0.smithi031.stdout:9/738: mkdir d2/d15/daf/dfa 0 2024-04-22T23:45:49.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/737: symlink d6/dc/d49/d83/d85/da9/lf3 0 2024-04-22T23:45:49.868 INFO:tasks.workunit.client.0.smithi031.stdout:0/695: stat d0/d1/d2/d36/d11/d1c/f88 0 2024-04-22T23:45:49.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/849: truncate d3/d35/d75/d7d/dc4/f123 2471075 0 2024-04-22T23:45:49.872 INFO:tasks.workunit.client.0.smithi031.stdout:4/877: rmdir db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62 39 2024-04-22T23:45:49.877 INFO:tasks.workunit.client.0.smithi031.stdout:7/711: mkdir d9/d14/d1f/dc6/df4 0 2024-04-22T23:45:49.879 INFO:tasks.workunit.client.0.smithi031.stdout:6/850: rename d3/d2a/d3c/d5e/d11b to d3/d128 0 2024-04-22T23:45:49.879 INFO:tasks.workunit.client.0.smithi031.stdout:7/712: fdatasync d9/d15/d3a/d41/f8f 0 2024-04-22T23:45:49.879 INFO:tasks.workunit.client.0.smithi031.stdout:2/738: dread d6/dc/d57/f4d [0,4194304] 0 2024-04-22T23:45:49.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/675: mknod d4/d54/d85/dc1/cec 0 2024-04-22T23:45:49.883 INFO:tasks.workunit.client.0.smithi031.stdout:9/739: truncate d2/dc/f10 2543134 0 2024-04-22T23:45:49.883 INFO:tasks.workunit.client.0.smithi031.stdout:2/739: chown d6/dc/d42/d24/d32 493430180 1 2024-04-22T23:45:49.884 INFO:tasks.workunit.client.0.smithi031.stdout:5/711: sync 2024-04-22T23:45:49.886 INFO:tasks.workunit.client.0.smithi031.stdout:0/696: dwrite d0/d13/d18/d6c/dad/fc8 [0,4194304] 0 2024-04-22T23:45:49.888 INFO:tasks.workunit.client.0.smithi031.stdout:2/740: write d6/dc/d9a/fb9 [395313,70736] 0 2024-04-22T23:45:49.892 INFO:tasks.workunit.client.0.smithi031.stdout:9/740: rename d2/dd/d75/d64 to d2/dd/d75/d64/d5d/d59/d8e/dfb 22 2024-04-22T23:45:49.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/697: read d0/d1/d2/d36/f9c [520371,47468] 0 2024-04-22T23:45:49.895 INFO:tasks.workunit.client.0.smithi031.stdout:3/676: dwrite d4/d6/f23 [0,4194304] 0 2024-04-22T23:45:49.899 INFO:tasks.workunit.client.0.smithi031.stdout:3/677: fdatasync d4/d6/d11/f3c 0 2024-04-22T23:45:49.904 INFO:tasks.workunit.client.0.smithi031.stdout:7/713: symlink d9/d14/d22/d5c/lf5 0 2024-04-22T23:45:49.906 INFO:tasks.workunit.client.0.smithi031.stdout:9/741: dwrite d2/dc/d8b/d3d/d40/f5c [0,4194304] 0 2024-04-22T23:45:49.906 INFO:tasks.workunit.client.0.smithi031.stdout:0/698: dread d0/d1/d2/d36/fb1 [0,4194304] 0 2024-04-22T23:45:49.907 INFO:tasks.workunit.client.0.smithi031.stdout:2/741: dwrite d6/dc/d9a/d6c/de4/fea [0,4194304] 0 2024-04-22T23:45:49.913 INFO:tasks.workunit.client.0.smithi031.stdout:6/851: dread d3/f74 [0,4194304] 0 2024-04-22T23:45:49.915 INFO:tasks.workunit.client.0.smithi031.stdout:2/742: chown d6/dc/d9a/d6c/db2/df0/d94 84418696 1 2024-04-22T23:45:49.922 INFO:tasks.workunit.client.0.smithi031.stdout:5/712: truncate db/d16/d1d/f6a 3696963 0 2024-04-22T23:45:49.923 INFO:tasks.workunit.client.0.smithi031.stdout:2/743: chown d6/dc/d57/d52/cc5 6 1 2024-04-22T23:45:49.923 INFO:tasks.workunit.client.0.smithi031.stdout:7/714: chown d9/d14/c68 12 1 2024-04-22T23:45:49.928 INFO:tasks.workunit.client.0.smithi031.stdout:4/878: dwrite db/d14/d2d/d49/d59/d6a/d63/dc9/fe6 [4194304,4194304] 0 2024-04-22T23:45:49.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/638: sync 2024-04-22T23:45:49.937 INFO:tasks.workunit.client.0.smithi031.stdout:1/704: sync 2024-04-22T23:45:49.939 INFO:tasks.workunit.client.0.smithi031.stdout:9/742: dread d2/f8 [0,4194304] 0 2024-04-22T23:45:49.940 INFO:tasks.workunit.client.0.smithi031.stdout:8/639: truncate d5/d48/d61/d6b/d6c/f70 4416351 0 2024-04-22T23:45:49.940 INFO:tasks.workunit.client.0.smithi031.stdout:9/743: readlink d2/l4 0 2024-04-22T23:45:49.941 INFO:tasks.workunit.client.0.smithi031.stdout:9/744: read d2/dc/d8b/d3d/d40/f39 [60920,81562] 0 2024-04-22T23:45:49.942 INFO:tasks.workunit.client.0.smithi031.stdout:9/745: chown d2/d15/l52 728016994 1 2024-04-22T23:45:49.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/852: write d3/f78 [291906,64124] 0 2024-04-22T23:45:49.962 INFO:tasks.workunit.client.0.smithi031.stdout:3/678: write d4/d6/d11/f37 [1092316,86177] 0 2024-04-22T23:45:49.980 INFO:tasks.workunit.client.0.smithi031.stdout:7/715: creat d9/d14/d22/d28/ff6 x:0 0 0 2024-04-22T23:45:49.983 INFO:tasks.workunit.client.0.smithi031.stdout:4/879: mknod db/d14/d2d/db7/c12a 0 2024-04-22T23:45:49.993 INFO:tasks.workunit.client.0.smithi031.stdout:9/746: rename d2/d15/daf/led to d2/dd/d65/lfc 0 2024-04-22T23:45:49.997 INFO:tasks.workunit.client.0.smithi031.stdout:6/853: creat d3/d35/d75/d7d/dc4/d124/f129 x:0 0 0 2024-04-22T23:45:49.998 INFO:tasks.workunit.client.0.smithi031.stdout:1/705: dread d1/d21/fc4 [0,4194304] 0 2024-04-22T23:45:50.000 INFO:tasks.workunit.client.0.smithi031.stdout:0/699: link d0/f4f d0/d13/fe3 0 2024-04-22T23:45:50.003 INFO:tasks.workunit.client.0.smithi031.stdout:2/744: unlink d6/dc/d9a/d6c/lcf 0 2024-04-22T23:45:50.006 INFO:tasks.workunit.client.0.smithi031.stdout:4/880: write db/d14/d2d/d49/d59/d6a/d63/d107/f33 [7713396,114358] 0 2024-04-22T23:45:50.007 INFO:tasks.workunit.client.0.smithi031.stdout:4/881: chown db/d14/d1d/d7b/d7f/d2a/d30/f42 2 1 2024-04-22T23:45:50.010 INFO:tasks.workunit.client.0.smithi031.stdout:3/679: rename d4/d6/d11/d25/d44/f7e to d4/d54/d6d/fed 0 2024-04-22T23:45:50.010 INFO:tasks.workunit.client.0.smithi031.stdout:3/680: chown d4/d54/d88/cbf 1966 1 2024-04-22T23:45:50.011 INFO:tasks.workunit.client.0.smithi031.stdout:3/681: fdatasync d4/d54/d2c/d89/d2e/d45/fd5 0 2024-04-22T23:45:50.014 INFO:tasks.workunit.client.0.smithi031.stdout:6/854: creat d3/d1f/d60/f12a x:0 0 0 2024-04-22T23:45:50.015 INFO:tasks.workunit.client.0.smithi031.stdout:6/855: write d3/d1f/f4f [1261600,17626] 0 2024-04-22T23:45:50.022 INFO:tasks.workunit.client.0.smithi031.stdout:0/700: truncate d0/d1/d2/d36/d11/d1a/d3c/f4d 5114526 0 2024-04-22T23:45:50.028 INFO:tasks.workunit.client.0.smithi031.stdout:8/640: rename d5/da/d12/l24 to d5/da/d12/lc0 0 2024-04-22T23:45:50.031 INFO:tasks.workunit.client.0.smithi031.stdout:9/747: mkdir d2/dc/d8b/d3d/d40/dfd 0 2024-04-22T23:45:50.033 INFO:tasks.workunit.client.0.smithi031.stdout:3/682: mkdir d4/d6/d11/d25/d44/d7f/d5f/dee 0 2024-04-22T23:45:50.037 INFO:tasks.workunit.client.0.smithi031.stdout:1/706: symlink d1/db/d16/d1f/d52/lf0 0 2024-04-22T23:45:50.037 INFO:tasks.workunit.client.0.smithi031.stdout:9/748: dwrite d2/dd/d75/f7e [0,4194304] 0 2024-04-22T23:45:50.038 INFO:tasks.workunit.client.0.smithi031.stdout:9/749: read - d2/dc/d6d/fa6 zero size 2024-04-22T23:45:50.039 INFO:tasks.workunit.client.0.smithi031.stdout:5/713: link db/f3f db/d16/d1d/d24/ddf/fe9 0 2024-04-22T23:45:50.040 INFO:tasks.workunit.client.0.smithi031.stdout:2/745: creat d6/dc/d48/d8a/d92/ff4 x:0 0 0 2024-04-22T23:45:50.041 INFO:tasks.workunit.client.0.smithi031.stdout:5/714: write db/d16/d2a/f7f [1285856,107642] 0 2024-04-22T23:45:50.042 INFO:tasks.workunit.client.0.smithi031.stdout:2/746: chown d6/dc/d48/d4b/d6b/de8/da4/lae 1 1 2024-04-22T23:45:50.045 INFO:tasks.workunit.client.0.smithi031.stdout:8/641: rename d5/da/df/d33/f5c to d5/d48/d61/d6b/fc1 0 2024-04-22T23:45:50.053 INFO:tasks.workunit.client.0.smithi031.stdout:1/707: truncate d1/de/d1e/d34/d41/f58 255265 0 2024-04-22T23:45:50.054 INFO:tasks.workunit.client.0.smithi031.stdout:1/708: fsync d1/de/d1e/d34/fa2 0 2024-04-22T23:45:50.055 INFO:tasks.workunit.client.0.smithi031.stdout:0/701: mkdir d0/d1/d2/db4/de4 0 2024-04-22T23:45:50.057 INFO:tasks.workunit.client.0.smithi031.stdout:0/702: write d0/d1/d2/d36/d11/d1c/d54/fc0 [373563,78380] 0 2024-04-22T23:45:50.057 INFO:tasks.workunit.client.0.smithi031.stdout:5/715: symlink db/d16/d2a/d59/lea 0 2024-04-22T23:45:50.058 INFO:tasks.workunit.client.0.smithi031.stdout:2/747: mknod d6/dc/d17/d69/cf5 0 2024-04-22T23:45:50.060 INFO:tasks.workunit.client.0.smithi031.stdout:2/748: truncate d6/dc/d42/d24/f34 4211137 0 2024-04-22T23:45:50.061 INFO:tasks.workunit.client.0.smithi031.stdout:9/750: rename d2/dd/d75/d64/d5d/d59/d8e/f9d to d2/dc/d8b/d3d/d40/dfd/ffe 0 2024-04-22T23:45:50.063 INFO:tasks.workunit.client.0.smithi031.stdout:8/642: fdatasync d5/da/d12/d37/d34/d6f/f42 0 2024-04-22T23:45:50.093 INFO:tasks.workunit.client.0.smithi031.stdout:1/709: symlink d1/d21/d4b/d6f/dd8/lf1 0 2024-04-22T23:45:50.128 INFO:tasks.workunit.client.0.smithi031.stdout:2/749: fdatasync d6/dc/d9a/d6c/db2/df0/f2a 0 2024-04-22T23:45:50.135 INFO:tasks.workunit.client.0.smithi031.stdout:3/683: getdents d4/d54/d56/dae 0 2024-04-22T23:45:50.138 INFO:tasks.workunit.client.0.smithi031.stdout:1/710: truncate d1/de/d1e/d34/d66/f92 254063 0 2024-04-22T23:45:50.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/716: symlink db/d16/d1d/d24/d7d/leb 0 2024-04-22T23:45:50.141 INFO:tasks.workunit.client.0.smithi031.stdout:7/716: truncate d9/d15/d3a/f51 1800674 0 2024-04-22T23:45:50.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/684: mknod d4/d54/d6d/d98/cef 0 2024-04-22T23:45:50.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/711: rmdir d1/d21/d4b 39 2024-04-22T23:45:50.155 INFO:tasks.workunit.client.0.smithi031.stdout:0/703: creat d0/d1/d2/d36/d11/fe5 x:0 0 0 2024-04-22T23:45:50.157 INFO:tasks.workunit.client.0.smithi031.stdout:7/717: rename d9/d15/d3f/ddc/f7c to d9/d15/dcf/ff7 0 2024-04-22T23:45:50.164 INFO:tasks.workunit.client.0.smithi031.stdout:3/685: rmdir d4/d6/da 39 2024-04-22T23:45:50.166 INFO:tasks.workunit.client.0.smithi031.stdout:6/856: truncate d3/d1f/f2f 2771816 0 2024-04-22T23:45:50.166 INFO:tasks.workunit.client.0.smithi031.stdout:6/857: chown d3/d1f/f4f 12598087 1 2024-04-22T23:45:50.170 INFO:tasks.workunit.client.0.smithi031.stdout:4/882: sync 2024-04-22T23:45:50.174 INFO:tasks.workunit.client.0.smithi031.stdout:0/704: truncate d0/d13/d18/d1e/d25/f85 4252949 0 2024-04-22T23:45:50.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/705: truncate d0/d13/d18/d1e/d25/d8e/fbd 773489 0 2024-04-22T23:45:50.178 INFO:tasks.workunit.client.0.smithi031.stdout:2/750: link d6/cd d6/dc/d48/d4b/d6b/cf6 0 2024-04-22T23:45:50.183 INFO:tasks.workunit.client.0.smithi031.stdout:3/686: dread d4/d6/d11/f1c [0,4194304] 0 2024-04-22T23:45:50.186 INFO:tasks.workunit.client.0.smithi031.stdout:2/751: dwrite d6/dc/d49/d83/fe5 [0,4194304] 0 2024-04-22T23:45:50.194 INFO:tasks.workunit.client.0.smithi031.stdout:3/687: dwrite d4/d6/d11/d25/d44/d7f/d5f/f63 [0,4194304] 0 2024-04-22T23:45:50.197 INFO:tasks.workunit.client.0.smithi031.stdout:3/688: dread d4/d54/d2c/d89/d2e/d45/f60 [0,4194304] 0 2024-04-22T23:45:50.213 INFO:tasks.workunit.client.0.smithi031.stdout:3/689: chown d4/d54/d6d/ca0 1061 1 2024-04-22T23:45:50.214 INFO:tasks.workunit.client.0.smithi031.stdout:3/690: chown d4/d54/d2c/d89/d2e/l90 24880 1 2024-04-22T23:45:50.218 INFO:tasks.workunit.client.0.smithi031.stdout:6/858: getdents d3/d1f/d4c/d8e 0 2024-04-22T23:45:50.219 INFO:tasks.workunit.client.0.smithi031.stdout:3/691: read d4/d6/f80 [79221,13102] 0 2024-04-22T23:45:50.221 INFO:tasks.workunit.client.0.smithi031.stdout:3/692: write d4/d6/d11/dc0/fcf [272169,42761] 0 2024-04-22T23:45:50.221 INFO:tasks.workunit.client.0.smithi031.stdout:3/693: chown d4/d6/d11 20 1 2024-04-22T23:45:50.222 INFO:tasks.workunit.client.0.smithi031.stdout:6/859: write d3/d1f/d60/f8f [2595168,90839] 0 2024-04-22T23:45:50.226 INFO:tasks.workunit.client.0.smithi031.stdout:6/860: rmdir d3/d2a/d85 39 2024-04-22T23:45:50.233 INFO:tasks.workunit.client.0.smithi031.stdout:3/694: mknod d4/d54/d2c/d89/d2e/d45/dde/cf0 0 2024-04-22T23:45:50.245 INFO:tasks.workunit.client.0.smithi031.stdout:6/861: getdents d3/d2a/d3c 0 2024-04-22T23:45:50.245 INFO:tasks.workunit.client.0.smithi031.stdout:6/862: fdatasync d3/fba 0 2024-04-22T23:45:50.250 INFO:tasks.workunit.client.0.smithi031.stdout:6/863: fsync d3/d1f/d4c/d8e/fb2 0 2024-04-22T23:45:50.252 INFO:tasks.workunit.client.0.smithi031.stdout:6/864: read d3/f78 [207041,115380] 0 2024-04-22T23:45:50.261 INFO:tasks.workunit.client.0.smithi031.stdout:6/865: link d3/d2a/d85/f9d d3/d35/d75/d7d/dc4/d124/f12b 0 2024-04-22T23:45:50.261 INFO:tasks.workunit.client.0.smithi031.stdout:6/866: chown d3/d1f/d4c/d8e 11665 1 2024-04-22T23:45:50.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/867: read - d3/d1f/d60/f12a zero size 2024-04-22T23:45:50.277 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:50 smithi031 ceph-mon[26721]: pgmap v31: 65 pgs: 65 active+clean; 1.2 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 147 MiB/s wr, 367 op/s 2024-04-22T23:45:50.277 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:50 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:50.277 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:50 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:50.280 INFO:tasks.workunit.client.0.smithi031.stdout:9/751: write d2/dd/d75/f97 [522792,86732] 0 2024-04-22T23:45:50.291 INFO:tasks.workunit.client.0.smithi031.stdout:9/752: dread d2/dc/f1e [0,4194304] 0 2024-04-22T23:45:50.292 INFO:tasks.workunit.client.0.smithi031.stdout:9/753: dread - d2/d15/d5a/f9f zero size 2024-04-22T23:45:50.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/754: write d2/dd/d75/d64/d5d/f57 [1371426,119771] 0 2024-04-22T23:45:50.300 INFO:tasks.workunit.client.0.smithi031.stdout:9/755: mknod d2/dc/d3e/cff 0 2024-04-22T23:45:50.300 INFO:tasks.workunit.client.0.smithi031.stdout:9/756: stat d2/f6e 0 2024-04-22T23:45:50.301 INFO:tasks.workunit.client.0.smithi031.stdout:9/757: dread - d2/dd/d75/d4a/da0/dd0/ff3 zero size 2024-04-22T23:45:50.303 INFO:tasks.workunit.client.0.smithi031.stdout:9/758: read d2/dd/d75/d64/f55 [3323,82540] 0 2024-04-22T23:45:50.310 INFO:tasks.workunit.client.0.smithi031.stdout:5/717: link db/d16/f3e db/d16/d2a/d59/fec 0 2024-04-22T23:45:50.310 INFO:tasks.workunit.client.0.smithi031.stdout:7/718: write d9/d15/d6d/d65/f72 [1302950,78604] 0 2024-04-22T23:45:50.314 INFO:tasks.workunit.client.0.smithi031.stdout:2/752: mknod d6/dc/d17/d69/cf7 0 2024-04-22T23:45:50.315 INFO:tasks.workunit.client.0.smithi031.stdout:2/753: read d6/f16 [1501839,94690] 0 2024-04-22T23:45:50.316 INFO:tasks.workunit.client.0.smithi031.stdout:2/754: dread - d6/dc/d48/d4b/fe3 zero size 2024-04-22T23:45:50.318 INFO:tasks.workunit.client.0.smithi031.stdout:2/755: stat d6/c8 0 2024-04-22T23:45:50.319 INFO:tasks.workunit.client.0.smithi031.stdout:2/756: dread - d6/dce/fda zero size 2024-04-22T23:45:50.321 INFO:tasks.workunit.client.0.smithi031.stdout:2/757: creat d6/dc/d48/d8a/d92/ff8 x:0 0 0 2024-04-22T23:45:50.323 INFO:tasks.workunit.client.0.smithi031.stdout:4/883: write db/d14/d2d/d49/d59/fa2 [684551,15264] 0 2024-04-22T23:45:50.327 INFO:tasks.workunit.client.0.smithi031.stdout:4/884: rmdir db/d14/d2d/d49/d59/d6a/d63/d8b 39 2024-04-22T23:45:50.333 INFO:tasks.workunit.client.0.smithi031.stdout:4/885: rename db/d14/d2d/d49/d59/fff to db/d14/d2d/d49/d59/d6a/d63/d107/f12b 0 2024-04-22T23:45:50.334 INFO:tasks.workunit.client.0.smithi031.stdout:2/758: getdents d6/dc/d9a/d6c/db2/df0/d94/dab 0 2024-04-22T23:45:50.337 INFO:tasks.workunit.client.0.smithi031.stdout:2/759: truncate d6/dc/d42/fa0 293856 0 2024-04-22T23:45:50.337 INFO:tasks.workunit.client.0.smithi031.stdout:2/760: chown d6/dc/d17/d69/d78 3 1 2024-04-22T23:45:50.345 INFO:tasks.workunit.client.0.smithi031.stdout:7/719: symlink d9/d14/d1f/d45/da6/lf8 0 2024-04-22T23:45:50.347 INFO:tasks.workunit.client.0.smithi031.stdout:2/761: dwrite d6/dc/d42/d24/f34 [4194304,4194304] 0 2024-04-22T23:45:50.348 INFO:tasks.workunit.client.0.smithi031.stdout:0/706: dwrite d0/d1/d2/d36/d11/d1a/d3c/f4d [4194304,4194304] 0 2024-04-22T23:45:50.352 INFO:tasks.workunit.client.0.smithi031.stdout:0/707: read d0/d13/d18/d1e/f30 [493667,62252] 0 2024-04-22T23:45:50.353 INFO:tasks.workunit.client.0.smithi031.stdout:4/886: dread db/d14/d1d/d7b/d7f/d2a/da1/fbc [0,4194304] 0 2024-04-22T23:45:50.354 INFO:tasks.workunit.client.0.smithi031.stdout:2/762: creat d6/dc/d48/d4b/d6b/de8/ff9 x:0 0 0 2024-04-22T23:45:50.356 INFO:tasks.workunit.client.0.smithi031.stdout:0/708: read - d0/f86 zero size 2024-04-22T23:45:50.358 INFO:tasks.workunit.client.0.smithi031.stdout:0/709: chown d0/d13/fe3 16 1 2024-04-22T23:45:50.359 INFO:tasks.workunit.client.0.smithi031.stdout:1/712: unlink d1/d35/f7b 0 2024-04-22T23:45:50.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/887: dread db/d14/d1d/d7b/d7f/d2a/d30/f101 [4194304,4194304] 0 2024-04-22T23:45:50.365 INFO:tasks.workunit.client.0.smithi031.stdout:1/713: stat d1/de/d1e/d34/c53 0 2024-04-22T23:45:50.366 INFO:tasks.workunit.client.0.smithi031.stdout:0/710: mkdir d0/d13/d18/d6c/dad/dba/de6 0 2024-04-22T23:45:50.370 INFO:tasks.workunit.client.0.smithi031.stdout:1/714: mknod d1/db/d16/cf2 0 2024-04-22T23:45:50.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/711: dread - d0/d1/d2/f7a zero size 2024-04-22T23:45:50.371 INFO:tasks.workunit.client.0.smithi031.stdout:3/695: dwrite d4/d54/fdc [4194304,4194304] 0 2024-04-22T23:45:50.373 INFO:tasks.workunit.client.0.smithi031.stdout:4/888: link db/d14/d1d/d7b/d7f/d2a/d48/fec db/d14/d2d/d49/d59/d6a/d41/f12c 0 2024-04-22T23:45:50.375 INFO:tasks.workunit.client.0.smithi031.stdout:4/889: truncate db/d14/d2d/d49/dcd/fd3 5232304 0 2024-04-22T23:45:50.376 INFO:tasks.workunit.client.1.smithi044.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-22T23:45:50.378 INFO:tasks.workunit.client.0.smithi031.stdout:3/696: fsync d4/d6/d11/d25/d44/d7f/f7d 0 2024-04-22T23:45:50.382 INFO:tasks.workunit.client.0.smithi031.stdout:1/715: rename d1/de/d1e/dcf to d1/d3f/da1/df3 0 2024-04-22T23:45:50.384 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:45:50.384 INFO:tasks.workunit.client.1.smithi044.stderr:+ make 2024-04-22T23:45:50.385 INFO:tasks.workunit.client.0.smithi031.stdout:3/697: rename d4/d54/d2c/d89/d2e/f4f to d4/d6/d22/ff1 0 2024-04-22T23:45:50.388 INFO:tasks.workunit.client.0.smithi031.stdout:4/890: rmdir db/d14/d2d/d49/d59/d98 0 2024-04-22T23:45:50.392 INFO:tasks.workunit.client.0.smithi031.stdout:4/891: write db/d14/d1d/d7b/d7f/ff0 [1043814,106790] 0 2024-04-22T23:45:50.394 INFO:tasks.workunit.client.0.smithi031.stdout:6/868: write d3/d1f/d4c/d8e/f93 [395686,118923] 0 2024-04-22T23:45:50.399 INFO:tasks.workunit.client.0.smithi031.stdout:6/869: symlink d3/d77/db5/d110/l12c 0 2024-04-22T23:45:50.400 INFO:tasks.workunit.client.0.smithi031.stdout:6/870: dread - d3/d35/d75/d7d/dc4/d124/f129 zero size 2024-04-22T23:45:50.401 INFO:tasks.workunit.client.0.smithi031.stdout:6/871: read - d3/d1f/d4c/df9/f121 zero size 2024-04-22T23:45:50.403 INFO:tasks.workunit.client.0.smithi031.stdout:4/892: rename db/d14/d1d/d7b/d7f/d2a/d30/f42 to db/d14/d2d/d49/d59/d6a/d63/d107/d129/dde/f12d 0 2024-04-22T23:45:50.408 INFO:tasks.workunit.client.0.smithi031.stdout:6/872: dwrite d3/d2a/d3c/d40/fdc [0,4194304] 0 2024-04-22T23:45:50.410 INFO:tasks.workunit.client.0.smithi031.stdout:4/893: creat db/d14/d2d/d49/d59/d6a/d63/d86/f12e x:0 0 0 2024-04-22T23:45:50.411 INFO:tasks.workunit.client.0.smithi031.stdout:8/643: symlink d5/d48/d61/lc2 0 2024-04-22T23:45:50.416 INFO:tasks.workunit.client.0.smithi031.stdout:6/873: unlink d3/d1f/c41 0 2024-04-22T23:45:50.418 INFO:tasks.workunit.client.0.smithi031.stdout:4/894: symlink db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/l12f 0 2024-04-22T23:45:50.419 INFO:tasks.workunit.client.0.smithi031.stdout:4/895: dread - db/d14/d1d/d7b/d7f/d2a/d48/fe5 zero size 2024-04-22T23:45:50.421 INFO:tasks.workunit.client.0.smithi031.stdout:8/644: dread d5/da/df/f13 [0,4194304] 0 2024-04-22T23:45:50.425 INFO:tasks.workunit.client.0.smithi031.stdout:4/896: dwrite db/d14/d2d/d49/d59/d6a/d63/d107/f33 [4194304,4194304] 0 2024-04-22T23:45:50.427 INFO:tasks.workunit.client.0.smithi031.stdout:0/712: mknod d0/d1/d2/ce7 0 2024-04-22T23:45:50.428 INFO:tasks.workunit.client.0.smithi031.stdout:6/874: creat d3/d2a/f12d x:0 0 0 2024-04-22T23:45:50.428 INFO:tasks.workunit.client.0.smithi031.stdout:9/759: write d2/d15/f7c [322713,71854] 0 2024-04-22T23:45:50.432 INFO:tasks.workunit.client.0.smithi031.stdout:0/713: write d0/d13/d18/d1e/d25/d8e/da7/d2b/f90 [966931,68633] 0 2024-04-22T23:45:50.434 INFO:tasks.workunit.client.0.smithi031.stdout:6/875: creat d3/d2a/d85/f12e x:0 0 0 2024-04-22T23:45:50.435 INFO:tasks.workunit.client.0.smithi031.stdout:8/645: getdents d5/da/d12/d2e/d7e 0 2024-04-22T23:45:50.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/714: truncate d0/d13/d18/dbf/fdb 686429 0 2024-04-22T23:45:50.437 INFO:tasks.workunit.client.0.smithi031.stdout:4/897: mknod db/d14/d2d/d49/d59/d6a/d63/d8b/c130 0 2024-04-22T23:45:50.438 INFO:tasks.workunit.client.0.smithi031.stdout:6/876: symlink d3/l12f 0 2024-04-22T23:45:50.439 INFO:tasks.workunit.client.0.smithi031.stdout:5/718: dwrite db/d16/d1d/d24/d32/f79 [0,4194304] 0 2024-04-22T23:45:50.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:50 smithi044 ceph-mon[34463]: pgmap v31: 65 pgs: 65 active+clean; 1.2 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 31 MiB/s rd, 147 MiB/s wr, 367 op/s 2024-04-22T23:45:50.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:50 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:50.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:50 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:50.478 INFO:tasks.workunit.client.1.smithi044.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-22T23:45:50.526 INFO:tasks.workunit.client.0.smithi031.stdout:8/646: mkdir d5/da/d12/d2e/db0/dc3 0 2024-04-22T23:45:50.526 INFO:tasks.workunit.client.0.smithi031.stdout:5/719: dwrite db/d16/d2a/d36/d74/d82/d86/f91 [0,4194304] 0 2024-04-22T23:45:50.530 INFO:tasks.workunit.client.0.smithi031.stdout:2/763: write d6/dc/d49/d83/d85/da9/fad [1392030,55410] 0 2024-04-22T23:45:50.532 INFO:tasks.workunit.client.0.smithi031.stdout:5/720: readlink db/d16/d1d/d24/l43 0 2024-04-22T23:45:50.535 INFO:tasks.workunit.client.0.smithi031.stdout:3/698: stat d4/d6/d22/ff1 0 2024-04-22T23:45:50.537 INFO:tasks.workunit.client.0.smithi031.stdout:5/721: write db/d16/d2a/d59/f75 [3841000,24376] 0 2024-04-22T23:45:50.537 INFO:tasks.workunit.client.0.smithi031.stdout:7/720: dwrite d9/d15/f60 [0,4194304] 0 2024-04-22T23:45:50.538 INFO:tasks.workunit.client.0.smithi031.stdout:9/760: truncate d2/d15/f2c 651087 0 2024-04-22T23:45:50.539 INFO:tasks.workunit.client.0.smithi031.stdout:2/764: mknod d6/dc/d9a/d6c/d86/cfa 0 2024-04-22T23:45:50.542 INFO:tasks.workunit.client.0.smithi031.stdout:3/699: creat d4/d54/d59/ff2 x:0 0 0 2024-04-22T23:45:50.544 INFO:tasks.workunit.client.0.smithi031.stdout:5/722: chown db/d16/f65 1350 1 2024-04-22T23:45:50.545 INFO:tasks.workunit.client.0.smithi031.stdout:8/647: link d5/da/d12/d37/d34/d6f/f42 d5/da/d12/d37/d4e/fc4 0 2024-04-22T23:45:50.546 INFO:tasks.workunit.client.0.smithi031.stdout:2/765: dread - d6/dc/d9a/d6c/db2/df0/d94/fe0 zero size 2024-04-22T23:45:50.550 INFO:tasks.workunit.client.0.smithi031.stdout:2/766: fdatasync d6/d45/d50/f58 0 2024-04-22T23:45:50.551 INFO:tasks.workunit.client.0.smithi031.stdout:5/723: mkdir db/d16/d1d/d24/d32/ded 0 2024-04-22T23:45:50.551 INFO:tasks.workunit.client.0.smithi031.stdout:6/877: dwrite d3/f38 [0,4194304] 0 2024-04-22T23:45:50.557 INFO:tasks.workunit.client.0.smithi031.stdout:6/878: mkdir d3/d2a/d3c/d130 0 2024-04-22T23:45:50.557 INFO:tasks.workunit.client.0.smithi031.stdout:2/767: creat d6/dc/d9a/d6c/db2/df0/d94/dab/ffb x:0 0 0 2024-04-22T23:45:50.557 INFO:tasks.workunit.client.0.smithi031.stdout:2/768: fdatasync d6/fc6 0 2024-04-22T23:45:50.559 INFO:tasks.workunit.client.0.smithi031.stdout:2/769: write d6/dce/fda [903837,94002] 0 2024-04-22T23:45:50.559 INFO:tasks.workunit.client.0.smithi031.stdout:5/724: unlink db/d16/d1d/d24/d27/l6d 0 2024-04-22T23:45:50.561 INFO:tasks.workunit.client.0.smithi031.stdout:5/725: chown db/d16/d1d/d24/d32/lc0 31208 1 2024-04-22T23:45:50.563 INFO:tasks.workunit.client.0.smithi031.stdout:2/770: rmdir d6/dc/d17/d69 39 2024-04-22T23:45:50.563 INFO:tasks.workunit.client.0.smithi031.stdout:7/721: write d9/d15/d3a/f51 [1585659,7552] 0 2024-04-22T23:45:50.564 INFO:tasks.workunit.client.0.smithi031.stdout:3/700: write d4/d6/da/f3d [3129054,1472] 0 2024-04-22T23:45:50.568 INFO:tasks.workunit.client.0.smithi031.stdout:7/722: chown d9/d14/d1f/d7f/l81 107 1 2024-04-22T23:45:50.570 INFO:tasks.workunit.client.0.smithi031.stdout:3/701: symlink d4/d6/d4c/d8c/lf3 0 2024-04-22T23:45:50.575 INFO:tasks.workunit.client.0.smithi031.stdout:5/726: dwrite db/d16/fe1 [0,4194304] 0 2024-04-22T23:45:50.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/723: mknod d9/d14/d1f/dc6/cf9 0 2024-04-22T23:45:50.577 INFO:tasks.workunit.client.0.smithi031.stdout:9/761: dwrite d2/dd/d75/d64/d5d/d59/fb0 [0,4194304] 0 2024-04-22T23:45:50.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/879: read - d3/d2a/d3c/d5e/f117 zero size 2024-04-22T23:45:50.580 INFO:tasks.workunit.client.0.smithi031.stdout:8/648: dread d5/d48/d61/d6b/d6c/f70 [0,4194304] 0 2024-04-22T23:45:50.580 INFO:tasks.workunit.client.0.smithi031.stdout:1/716: dread d1/de/d1e/d34/d41/f44 [0,4194304] 0 2024-04-22T23:45:50.581 INFO:tasks.workunit.client.0.smithi031.stdout:7/724: write d9/d15/d3f/ddc/fdd [871517,78199] 0 2024-04-22T23:45:50.581 INFO:tasks.workunit.client.0.smithi031.stdout:3/702: rename d4/d6/d11/f1c to d4/d6/d22/ff4 0 2024-04-22T23:45:50.584 INFO:tasks.workunit.client.0.smithi031.stdout:9/762: symlink d2/dd3/l100 0 2024-04-22T23:45:50.584 INFO:tasks.workunit.client.0.smithi031.stdout:3/703: stat d4/d54/d2c/d89/d2e/f97 0 2024-04-22T23:45:50.586 INFO:tasks.workunit.client.0.smithi031.stdout:7/725: stat d9/ceb 0 2024-04-22T23:45:50.587 INFO:tasks.workunit.client.0.smithi031.stdout:5/727: truncate db/d16/d2a/d59/d90/f8e 3140159 0 2024-04-22T23:45:50.587 INFO:tasks.workunit.client.0.smithi031.stdout:1/717: fsync d1/db/d16/d1f/fe2 0 2024-04-22T23:45:50.591 INFO:tasks.workunit.client.0.smithi031.stdout:8/649: rename d5/da/d12/d37/d34/d6f/f6d to d5/da/d83/fc5 0 2024-04-22T23:45:50.593 INFO:tasks.workunit.client.0.smithi031.stdout:5/728: truncate db/d16/d1d/d24/f63 2368953 0 2024-04-22T23:45:50.594 INFO:tasks.workunit.client.0.smithi031.stdout:9/763: dwrite d2/dc/d8b/d3d/d40/f44 [0,4194304] 0 2024-04-22T23:45:50.595 INFO:tasks.workunit.client.0.smithi031.stdout:8/650: chown d5/da/d12/d37/d34/d6f/f4b 443229281 1 2024-04-22T23:45:50.597 INFO:tasks.workunit.client.0.smithi031.stdout:3/704: rename d4/d6/da/f29 to d4/d83/dd8/de6/ff5 0 2024-04-22T23:45:50.598 INFO:tasks.workunit.client.0.smithi031.stdout:2/771: dread d6/dc/d48/d4b/d6b/de8/fc9 [0,4194304] 0 2024-04-22T23:45:50.598 INFO:tasks.workunit.client.0.smithi031.stdout:6/880: link d3/d2a/d3c/f71 d3/d35/d75/d7d/f131 0 2024-04-22T23:45:50.599 INFO:tasks.workunit.client.0.smithi031.stdout:5/729: unlink db/d16/d2a/d36/f98 0 2024-04-22T23:45:50.600 INFO:tasks.workunit.client.0.smithi031.stdout:1/718: dwrite d1/d21/d4b/f5d [0,4194304] 0 2024-04-22T23:45:50.601 INFO:tasks.workunit.client.0.smithi031.stdout:8/651: creat d5/da/d12/d1d/d38/fc6 x:0 0 0 2024-04-22T23:45:50.603 INFO:tasks.workunit.client.0.smithi031.stdout:7/726: rename d9/d15/d6d/dcc/fcd to d9/d14/d1f/d7f/ffa 0 2024-04-22T23:45:50.606 INFO:tasks.workunit.client.0.smithi031.stdout:8/652: truncate d5/da/d12/f96 898510 0 2024-04-22T23:45:50.607 INFO:tasks.workunit.client.0.smithi031.stdout:9/764: dread d2/dc/f4c [0,4194304] 0 2024-04-22T23:45:50.608 INFO:tasks.workunit.client.0.smithi031.stdout:3/705: creat d4/d54/d2c/d89/d2e/ff6 x:0 0 0 2024-04-22T23:45:50.609 INFO:tasks.workunit.client.0.smithi031.stdout:8/653: readlink d5/da/d83/l9e 0 2024-04-22T23:45:50.611 INFO:tasks.workunit.client.0.smithi031.stdout:7/727: creat d9/d15/ffb x:0 0 0 2024-04-22T23:45:50.613 INFO:tasks.workunit.client.0.smithi031.stdout:5/730: dread f7 [0,4194304] 0 2024-04-22T23:45:50.613 INFO:tasks.workunit.client.0.smithi031.stdout:8/654: symlink d5/da/df/d33/lc7 0 2024-04-22T23:45:50.614 INFO:tasks.workunit.client.0.smithi031.stdout:7/728: stat d9/d15/d6d/d65/l67 0 2024-04-22T23:45:50.616 INFO:tasks.workunit.client.0.smithi031.stdout:1/719: dwrite d1/db/d16/d1f/d52/db0/fea [0,4194304] 0 2024-04-22T23:45:50.621 INFO:tasks.workunit.client.0.smithi031.stdout:9/765: dwrite d2/dd/d75/d64/d5d/d59/f78 [0,4194304] 0 2024-04-22T23:45:50.622 INFO:tasks.workunit.client.0.smithi031.stdout:9/766: fsync d2/d15/d5a/f9f 0 2024-04-22T23:45:50.645 INFO:tasks.workunit.client.0.smithi031.stdout:6/881: truncate d3/d2a/d3c/d40/d107/fd8 3569490 0 2024-04-22T23:45:50.646 INFO:tasks.workunit.client.0.smithi031.stdout:3/706: creat d4/d54/d88/da2/de0/ff7 x:0 0 0 2024-04-22T23:45:50.647 INFO:tasks.workunit.client.0.smithi031.stdout:3/707: stat d4/d54/d2c/d89/d2e/l76 0 2024-04-22T23:45:50.650 INFO:tasks.workunit.client.0.smithi031.stdout:7/729: creat d9/d15/dcf/ffc x:0 0 0 2024-04-22T23:45:50.651 INFO:tasks.workunit.client.0.smithi031.stdout:8/655: read d5/da/d12/f44 [562555,123456] 0 2024-04-22T23:45:50.651 INFO:tasks.workunit.client.0.smithi031.stdout:5/731: creat db/d16/d1d/d24/d58/d5d/fee x:0 0 0 2024-04-22T23:45:50.653 INFO:tasks.workunit.client.0.smithi031.stdout:1/720: symlink d1/db/d16/d2f/lf4 0 2024-04-22T23:45:50.655 INFO:tasks.workunit.client.0.smithi031.stdout:6/882: symlink d3/d35/d75/d7d/dc4/l132 0 2024-04-22T23:45:50.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/708: truncate d4/d6/d4c/f79 516986 0 2024-04-22T23:45:50.659 INFO:tasks.workunit.client.0.smithi031.stdout:7/730: unlink f2 0 2024-04-22T23:45:50.661 INFO:tasks.workunit.client.0.smithi031.stdout:8/656: readlink d5/da/d12/d2e/d4d/l89 0 2024-04-22T23:45:50.663 INFO:tasks.workunit.client.0.smithi031.stdout:9/767: creat d2/dd/d75/d80/df8/f101 x:0 0 0 2024-04-22T23:45:50.664 INFO:tasks.workunit.client.0.smithi031.stdout:6/883: creat d3/d1f/da3/f133 x:0 0 0 2024-04-22T23:45:50.667 INFO:tasks.workunit.client.0.smithi031.stdout:6/884: read d3/d2a/d3c/d40/d5b/fc5 [1814961,61567] 0 2024-04-22T23:45:50.671 INFO:tasks.workunit.client.0.smithi031.stdout:9/768: dwrite d2/dd/d75/d4a/f36 [0,4194304] 0 2024-04-22T23:45:50.676 INFO:tasks.workunit.client.0.smithi031.stdout:6/885: dread d3/d77/db5/d110/f113 [0,4194304] 0 2024-04-22T23:45:50.676 INFO:tasks.workunit.client.0.smithi031.stdout:9/769: dread d2/dd/d75/d64/f55 [0,4194304] 0 2024-04-22T23:45:50.679 INFO:tasks.workunit.client.0.smithi031.stdout:7/731: dread d9/d15/dcf/fd2 [0,4194304] 0 2024-04-22T23:45:50.680 INFO:tasks.workunit.client.0.smithi031.stdout:9/770: dread - d2/dc/d8b/d3d/d40/da4/fb3 zero size 2024-04-22T23:45:50.680 INFO:tasks.workunit.client.0.smithi031.stdout:5/732: getdents db/d16/d2a/d7c 0 2024-04-22T23:45:50.681 INFO:tasks.workunit.client.0.smithi031.stdout:1/721: getdents d1/db/d16 0 2024-04-22T23:45:50.685 INFO:tasks.workunit.client.0.smithi031.stdout:5/733: creat db/d16/d2a/d59/fef x:0 0 0 2024-04-22T23:45:50.687 INFO:tasks.workunit.client.0.smithi031.stdout:8/657: write d5/da/d83/d8d/f91 [3273085,59721] 0 2024-04-22T23:45:50.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/886: write d3/d2a/d3c/f73 [2313032,117031] 0 2024-04-22T23:45:50.692 INFO:tasks.workunit.client.0.smithi031.stdout:9/771: rmdir d2/dd/d65/dcd 0 2024-04-22T23:45:50.696 INFO:tasks.workunit.client.0.smithi031.stdout:5/734: dwrite db/d16/d2a/d36/d74/d82/d86/f91 [0,4194304] 0 2024-04-22T23:45:50.699 INFO:tasks.workunit.client.0.smithi031.stdout:1/722: dwrite d1/d21/d4b/d6f/da6/fc7 [0,4194304] 0 2024-04-22T23:45:50.699 INFO:tasks.workunit.client.0.smithi031.stdout:6/887: getdents d3/d2a 0 2024-04-22T23:45:50.702 INFO:tasks.workunit.client.0.smithi031.stdout:8/658: truncate d5/da/d12/d2e/d4d/d9a/fa9 3448680 0 2024-04-22T23:45:50.704 INFO:tasks.workunit.client.0.smithi031.stdout:1/723: fsync d1/db/d16/d1f/f26 0 2024-04-22T23:45:50.706 INFO:tasks.workunit.client.0.smithi031.stdout:5/735: rmdir db/d16/d1d/d24/d58/d5d/dd6 39 2024-04-22T23:45:50.707 INFO:tasks.workunit.client.0.smithi031.stdout:8/659: write d5/da/d12/d37/d4e/f81 [2595354,11631] 0 2024-04-22T23:45:50.707 INFO:tasks.workunit.client.0.smithi031.stdout:7/732: dread f3 [0,4194304] 0 2024-04-22T23:45:50.708 INFO:tasks.workunit.client.0.smithi031.stdout:9/772: link d2/dc/f17 d2/dd/d75/d64/d5d/d59/d8e/f102 0 2024-04-22T23:45:50.709 INFO:tasks.workunit.client.0.smithi031.stdout:8/660: read - d5/da/d12/d1d/f8b zero size 2024-04-22T23:45:50.710 INFO:tasks.workunit.client.0.smithi031.stdout:1/724: symlink d1/d21/d4b/d6f/lf5 0 2024-04-22T23:45:50.710 INFO:tasks.workunit.client.0.smithi031.stdout:6/888: dread d3/d1f/da3/fd5 [0,4194304] 0 2024-04-22T23:45:50.711 INFO:tasks.workunit.client.0.smithi031.stdout:8/661: write d5/da/fa2 [3316718,78943] 0 2024-04-22T23:45:50.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/773: rmdir d2/dc/d8b 39 2024-04-22T23:45:50.714 INFO:tasks.workunit.client.0.smithi031.stdout:8/662: write d5/da/fa2 [4404283,101882] 0 2024-04-22T23:45:50.715 INFO:tasks.workunit.client.0.smithi031.stdout:9/774: fsync d2/dd/d75/d64/d5d/d59/fc7 0 2024-04-22T23:45:50.716 INFO:tasks.workunit.client.0.smithi031.stdout:5/736: symlink db/d16/d2a/d59/d90/da4/lf0 0 2024-04-22T23:45:50.717 INFO:tasks.workunit.client.0.smithi031.stdout:5/737: readlink db/d16/d2a/d59/lea 0 2024-04-22T23:45:50.718 INFO:tasks.workunit.client.0.smithi031.stdout:5/738: chown db/d16/d2a/dca/fda 1719407 1 2024-04-22T23:45:50.720 INFO:tasks.workunit.client.0.smithi031.stdout:5/739: fdatasync db/d16/d1d/d24/d58/f51 0 2024-04-22T23:45:50.721 INFO:tasks.workunit.client.0.smithi031.stdout:7/733: mkdir d9/d14/d1f/d45/de3/dfd 0 2024-04-22T23:45:50.726 INFO:tasks.workunit.client.0.smithi031.stdout:5/740: mkdir db/d8d/df1 0 2024-04-22T23:45:50.727 INFO:tasks.workunit.client.0.smithi031.stdout:8/663: mkdir d5/d48/d61/d6b/d6c/dc8 0 2024-04-22T23:45:50.729 INFO:tasks.workunit.client.0.smithi031.stdout:9/775: rename d2/dd/d75/d4a/da0/dd0/ff3 to d2/dc/d8b/d3d/df9/f103 0 2024-04-22T23:45:50.730 INFO:tasks.workunit.client.0.smithi031.stdout:1/725: link d1/d21/d4b/d6f/dd8/faf d1/db3/ff6 0 2024-04-22T23:45:50.730 INFO:tasks.workunit.client.0.smithi031.stdout:5/741: readlink db/d16/d1d/d24/l4d 0 2024-04-22T23:45:50.731 INFO:tasks.workunit.client.0.smithi031.stdout:8/664: stat d5/l10 0 2024-04-22T23:45:50.732 INFO:tasks.workunit.client.0.smithi031.stdout:9/776: truncate d2/dc/d8b/fdf 922907 0 2024-04-22T23:45:50.735 INFO:tasks.workunit.client.0.smithi031.stdout:1/726: rename d1/db/d16/d1f/f8b to d1/d35/d8d/ff7 0 2024-04-22T23:45:50.736 INFO:tasks.workunit.client.0.smithi031.stdout:8/665: mkdir d5/da/d12/d37/d4e/dc9 0 2024-04-22T23:45:50.737 INFO:tasks.workunit.client.0.smithi031.stdout:9/777: mkdir d2/dc/d8b/d3d/d40/da4/d104 0 2024-04-22T23:45:50.739 INFO:tasks.workunit.client.0.smithi031.stdout:1/727: chown d1/de/d1e/d34/d41/lde 1 1 2024-04-22T23:45:50.740 INFO:tasks.workunit.client.0.smithi031.stdout:7/734: dread d9/d15/d3a/d41/f8f [0,4194304] 0 2024-04-22T23:45:50.742 INFO:tasks.workunit.client.0.smithi031.stdout:8/666: link d5/da/d12/d2e/d4d/d56/f71 d5/da/d12/d1d/fca 0 2024-04-22T23:45:50.742 INFO:tasks.workunit.client.0.smithi031.stdout:6/889: dread d3/d2a/d3c/f98 [0,4194304] 0 2024-04-22T23:45:50.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/728: write d1/db/d16/d2f/fae [1339258,2739] 0 2024-04-22T23:45:50.744 INFO:tasks.workunit.client.0.smithi031.stdout:7/735: write d9/d14/d22/fb9 [1256936,114622] 0 2024-04-22T23:45:50.746 INFO:tasks.workunit.client.0.smithi031.stdout:6/890: truncate d3/d77/fe4 278733 0 2024-04-22T23:45:50.748 INFO:tasks.workunit.client.0.smithi031.stdout:9/778: dwrite d2/dc/d8b/d3d/d40/da4/fab [0,4194304] 0 2024-04-22T23:45:50.748 INFO:tasks.workunit.client.0.smithi031.stdout:8/667: mkdir d5/da/d12/d37/d34/d6f/d5b/dcb 0 2024-04-22T23:45:50.749 INFO:tasks.workunit.client.0.smithi031.stdout:7/736: write d9/d15/d6d/d65/faf [1154072,35330] 0 2024-04-22T23:45:50.750 INFO:tasks.workunit.client.0.smithi031.stdout:8/668: readlink d5/lb3 0 2024-04-22T23:45:50.755 INFO:tasks.workunit.client.0.smithi031.stdout:5/742: truncate db/d16/d2a/d36/d74/d82/f9f 1141422 0 2024-04-22T23:45:50.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/669: mknod d5/d48/d61/ccc 0 2024-04-22T23:45:50.757 INFO:tasks.workunit.client.0.smithi031.stdout:1/729: dwrite d1/d35/d36/f9e [0,4194304] 0 2024-04-22T23:45:50.757 INFO:tasks.workunit.client.0.smithi031.stdout:7/737: symlink d9/d14/d22/d28/lfe 0 2024-04-22T23:45:50.761 INFO:tasks.workunit.client.0.smithi031.stdout:8/670: write d5/da/d12/d2e/d7e/fb4 [276067,7454] 0 2024-04-22T23:45:50.762 INFO:tasks.workunit.client.0.smithi031.stdout:7/738: fsync d9/d14/d22/d28/d7a/f95 0 2024-04-22T23:45:50.763 INFO:tasks.workunit.client.0.smithi031.stdout:9/779: dread d2/dd/d75/d4a/f36 [0,4194304] 0 2024-04-22T23:45:50.764 INFO:tasks.workunit.client.0.smithi031.stdout:1/730: rename d1/de/d1e/d34/fa2 to d1/d3f/d54/dba/ff8 0 2024-04-22T23:45:50.767 INFO:tasks.workunit.client.0.smithi031.stdout:1/731: write d1/db/d16/d2f/fae [2377547,107435] 0 2024-04-22T23:45:50.768 INFO:tasks.workunit.client.0.smithi031.stdout:1/732: dread - d1/db/d16/fe9 zero size 2024-04-22T23:45:50.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/733: fsync d1/db/d16/d2f/fc0 0 2024-04-22T23:45:50.770 INFO:tasks.workunit.client.0.smithi031.stdout:9/780: mkdir d2/dc/d6d/d105 0 2024-04-22T23:45:50.772 INFO:tasks.workunit.client.0.smithi031.stdout:8/671: dwrite d5/d48/d61/d6b/d6c/f73 [0,4194304] 0 2024-04-22T23:45:50.773 INFO:tasks.workunit.client.0.smithi031.stdout:9/781: readlink d2/dd/d75/d64/lc0 0 2024-04-22T23:45:50.774 INFO:tasks.workunit.client.0.smithi031.stdout:5/743: link db/c68 db/d8d/df1/cf2 0 2024-04-22T23:45:50.778 INFO:tasks.workunit.client.0.smithi031.stdout:8/672: fsync d5/da/d12/d37/d34/f4c 0 2024-04-22T23:45:50.784 INFO:tasks.workunit.client.0.smithi031.stdout:6/891: dwrite d3/d35/d75/d7d/dc4/f123 [0,4194304] 0 2024-04-22T23:45:50.784 INFO:tasks.workunit.client.0.smithi031.stdout:8/673: truncate d5/da/d12/f44 265572 0 2024-04-22T23:45:50.786 INFO:tasks.workunit.client.0.smithi031.stdout:9/782: dwrite d2/d15/fcc [0,4194304] 0 2024-04-22T23:45:50.786 INFO:tasks.workunit.client.0.smithi031.stdout:8/674: chown d5/da/df 2094987 1 2024-04-22T23:45:50.788 INFO:tasks.workunit.client.0.smithi031.stdout:1/734: write d1/de/d1e/d34/d41/f4c [1440876,68252] 0 2024-04-22T23:45:50.792 INFO:tasks.workunit.client.0.smithi031.stdout:7/739: dread d9/ff [0,4194304] 0 2024-04-22T23:45:50.795 INFO:tasks.workunit.client.0.smithi031.stdout:1/735: chown d1/d35/l48 3276 1 2024-04-22T23:45:50.796 INFO:tasks.workunit.client.0.smithi031.stdout:8/675: mkdir d5/da/d12/d37/d34/d6f/d85/dcd 0 2024-04-22T23:45:50.797 INFO:tasks.workunit.client.0.smithi031.stdout:5/744: dwrite db/d16/d1d/d24/fc1 [0,4194304] 0 2024-04-22T23:45:50.800 INFO:tasks.workunit.client.0.smithi031.stdout:9/783: mknod d2/dc/d8b/d3d/d40/da4/d104/c106 0 2024-04-22T23:45:50.801 INFO:tasks.workunit.client.0.smithi031.stdout:6/892: dwrite d3/d77/d8d/fa0 [4194304,4194304] 0 2024-04-22T23:45:50.805 INFO:tasks.workunit.client.0.smithi031.stdout:8/676: symlink d5/da/d12/d2e/d4d/lce 0 2024-04-22T23:45:50.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/740: read - d9/d14/d22/d28/f4a zero size 2024-04-22T23:45:50.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/745: dread db/d16/d1d/d24/fc1 [0,4194304] 0 2024-04-22T23:45:50.813 INFO:tasks.workunit.client.0.smithi031.stdout:6/893: rename d3/d35/c37 to d3/d1f/c134 0 2024-04-22T23:45:50.814 INFO:tasks.workunit.client.0.smithi031.stdout:9/784: creat d2/dc/d3e/f107 x:0 0 0 2024-04-22T23:45:50.814 INFO:tasks.workunit.client.0.smithi031.stdout:1/736: dread d1/db/f19 [0,4194304] 0 2024-04-22T23:45:50.815 INFO:tasks.workunit.client.0.smithi031.stdout:9/785: chown d2/dc/fb7 57 1 2024-04-22T23:45:50.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/746: creat db/d16/d1d/d24/d58/d5d/dd6/ff3 x:0 0 0 2024-04-22T23:45:50.819 INFO:tasks.workunit.client.0.smithi031.stdout:1/737: creat d1/d21/d4b/dbc/ff9 x:0 0 0 2024-04-22T23:45:50.821 INFO:tasks.workunit.client.0.smithi031.stdout:1/738: chown d1/db/d16/f25 7472 1 2024-04-22T23:45:50.822 INFO:tasks.workunit.client.0.smithi031.stdout:7/741: dread d9/f37 [0,4194304] 0 2024-04-22T23:45:50.823 INFO:tasks.workunit.client.0.smithi031.stdout:9/786: rename d2/dd/d75/d64/d5d/d59/cd1 to d2/dd/d75/d64/c108 0 2024-04-22T23:45:50.823 INFO:tasks.workunit.client.0.smithi031.stdout:6/894: dread d3/d35/d75/f89 [4194304,4194304] 0 2024-04-22T23:45:50.826 INFO:tasks.workunit.client.0.smithi031.stdout:1/739: fsync d1/de/d1e/d34/d41/f62 0 2024-04-22T23:45:50.827 INFO:tasks.workunit.client.0.smithi031.stdout:5/747: dwrite db/d8d/d9b/fcc [0,4194304] 0 2024-04-22T23:45:50.827 INFO:tasks.workunit.client.0.smithi031.stdout:7/742: chown d9/d14/d1f/d45/da6/lb8 7181770 1 2024-04-22T23:45:50.828 INFO:tasks.workunit.client.0.smithi031.stdout:9/787: truncate d2/dc/d3e/f99 1228288 0 2024-04-22T23:45:50.828 INFO:tasks.workunit.client.0.smithi031.stdout:1/740: stat d1/de/d1e/d34 0 2024-04-22T23:45:50.829 INFO:tasks.workunit.client.0.smithi031.stdout:6/895: mkdir d3/d35/d75/d7d/d135 0 2024-04-22T23:45:50.829 INFO:tasks.workunit.client.0.smithi031.stdout:7/743: chown d9/d14/d22/d5c/d5f 568 1 2024-04-22T23:45:50.831 INFO:tasks.workunit.client.0.smithi031.stdout:9/788: dread - d2/dd/d75/d80/df8/f101 zero size 2024-04-22T23:45:50.832 INFO:tasks.workunit.client.0.smithi031.stdout:5/748: truncate db/d16/d1d/d24/d32/fe3 631891 0 2024-04-22T23:45:50.833 INFO:tasks.workunit.client.0.smithi031.stdout:1/741: write d1/de/f11 [3260278,41764] 0 2024-04-22T23:45:50.833 INFO:tasks.workunit.client.0.smithi031.stdout:6/896: stat d3/d35/d86/c127 0 2024-04-22T23:45:50.834 INFO:tasks.workunit.client.0.smithi031.stdout:6/897: chown d3/c3f 1283021 1 2024-04-22T23:45:50.835 INFO:tasks.workunit.client.0.smithi031.stdout:7/744: dread d9/d14/d22/d5c/f9a [0,4194304] 0 2024-04-22T23:45:50.836 INFO:tasks.workunit.client.0.smithi031.stdout:9/789: fdatasync d2/dc/d8b/d3d/f51 0 2024-04-22T23:45:50.836 INFO:tasks.workunit.client.0.smithi031.stdout:6/898: chown d3/d2a/d3c/d40/d5b/l7c 14374 1 2024-04-22T23:45:50.841 INFO:tasks.workunit.client.0.smithi031.stdout:8/677: dread d5/da/df/d33/f65 [0,4194304] 0 2024-04-22T23:45:50.841 INFO:tasks.workunit.client.0.smithi031.stdout:9/790: read d2/dc/f12 [3685100,48905] 0 2024-04-22T23:45:50.842 INFO:tasks.workunit.client.0.smithi031.stdout:1/742: link d1/d3f/da1/df3/fb7 d1/db/ffa 0 2024-04-22T23:45:50.842 INFO:tasks.workunit.client.0.smithi031.stdout:6/899: creat d3/d35/d75/d7d/dc4/d124/f136 x:0 0 0 2024-04-22T23:45:50.843 INFO:tasks.workunit.client.0.smithi031.stdout:1/743: chown d1/d35/l48 926261365 1 2024-04-22T23:45:50.843 INFO:tasks.workunit.client.0.smithi031.stdout:7/745: rename d9/f1c to d9/d14/dd6/fff 0 2024-04-22T23:45:50.844 INFO:tasks.workunit.client.0.smithi031.stdout:6/900: fsync d3/d35/db1/f106 0 2024-04-22T23:45:50.844 INFO:tasks.workunit.client.0.smithi031.stdout:9/791: mkdir d2/dd/d75/d64/d5d/d109 0 2024-04-22T23:45:50.847 INFO:tasks.workunit.client.0.smithi031.stdout:9/792: write d2/dd/d75/d64/fa8 [1016008,100250] 0 2024-04-22T23:45:50.848 INFO:tasks.workunit.client.0.smithi031.stdout:1/744: unlink d1/db/d16/d2f/fc0 0 2024-04-22T23:45:50.849 INFO:tasks.workunit.client.0.smithi031.stdout:8/678: dwrite d5/d48/d61/d6b/fa1 [0,4194304] 0 2024-04-22T23:45:50.851 INFO:tasks.workunit.client.0.smithi031.stdout:6/901: mknod d3/d35/c137 0 2024-04-22T23:45:50.853 INFO:tasks.workunit.client.0.smithi031.stdout:7/746: symlink d9/d14/d1f/d45/da6/daa/dbb/dd8/l100 0 2024-04-22T23:45:50.853 INFO:tasks.workunit.client.0.smithi031.stdout:1/745: write d1/d21/f9b [546449,115344] 0 2024-04-22T23:45:50.855 INFO:tasks.workunit.client.0.smithi031.stdout:9/793: mkdir d2/dc/d8b/d3d/d40/da4/db1/dc5/d10a 0 2024-04-22T23:45:50.858 INFO:tasks.workunit.client.0.smithi031.stdout:9/794: write d2/dc/f25 [1193357,80265] 0 2024-04-22T23:45:50.858 INFO:tasks.workunit.client.0.smithi031.stdout:8/679: creat d5/da/d12/d37/d34/fcf x:0 0 0 2024-04-22T23:45:50.860 INFO:tasks.workunit.client.0.smithi031.stdout:9/795: write d2/dd/d75/d64/d5d/d59/f78 [4457079,64118] 0 2024-04-22T23:45:50.864 INFO:tasks.workunit.client.0.smithi031.stdout:7/747: rename d9/d15/d3f/ddc/f69 to d9/d14/d22/d5c/d5f/f101 0 2024-04-22T23:45:50.864 INFO:tasks.workunit.client.0.smithi031.stdout:1/746: link d1/d35/l48 d1/d3f/d54/d84/lfb 0 2024-04-22T23:45:50.868 INFO:tasks.workunit.client.0.smithi031.stdout:1/747: creat d1/d99/d63/ffc x:0 0 0 2024-04-22T23:45:50.869 INFO:tasks.workunit.client.0.smithi031.stdout:9/796: rmdir d2/dc/d8b/d3d/d40/da4/db1/db2 0 2024-04-22T23:45:50.875 INFO:tasks.workunit.client.0.smithi031.stdout:9/797: chown d2/dc/d8b/d3d/d40/l93 721 1 2024-04-22T23:45:50.875 INFO:tasks.workunit.client.0.smithi031.stdout:6/902: write d3/f15 [2868402,3666] 0 2024-04-22T23:45:50.877 INFO:tasks.workunit.client.0.smithi031.stdout:6/903: chown d3/d2a/d85/fab 26912 1 2024-04-22T23:45:50.879 INFO:tasks.workunit.client.0.smithi031.stdout:8/680: write d5/da/d12/d37/d4e/fc4 [3617667,80680] 0 2024-04-22T23:45:50.883 INFO:tasks.workunit.client.0.smithi031.stdout:1/748: write d1/d3f/da1/df3/fb7 [1492732,130771] 0 2024-04-22T23:45:50.885 INFO:tasks.workunit.client.0.smithi031.stdout:7/748: dwrite d9/d14/d22/d5c/f9e [0,4194304] 0 2024-04-22T23:45:50.888 INFO:tasks.workunit.client.0.smithi031.stdout:9/798: truncate d2/dc/d8b/d3d/d40/da4/ff4 1572493 0 2024-04-22T23:45:50.889 INFO:tasks.workunit.client.0.smithi031.stdout:9/799: readlink d2/dc/d8b/d3d/d40/da4/db1/le8 0 2024-04-22T23:45:50.890 INFO:tasks.workunit.client.0.smithi031.stdout:9/800: write d2/dc/d8b/d3d/d40/f39 [1257419,15529] 0 2024-04-22T23:45:50.891 INFO:tasks.workunit.client.0.smithi031.stdout:9/801: stat d2/dc/d8b/d3d/d40/da4/db1/dc5/def 0 2024-04-22T23:45:50.894 INFO:tasks.workunit.client.0.smithi031.stdout:7/749: rename d9/d15/d6d/d65/fb7 to d9/d15/d6d/d65/f102 0 2024-04-22T23:45:50.896 INFO:tasks.workunit.client.0.smithi031.stdout:8/681: rmdir d5/d48/d61/d6b/d6c/dc8 0 2024-04-22T23:45:50.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/682: chown d5/da/d12/d37/d34/l68 1390457903 1 2024-04-22T23:45:50.898 INFO:tasks.workunit.client.0.smithi031.stdout:8/683: write d5/da/d12/fb6 [494253,27785] 0 2024-04-22T23:45:50.906 INFO:tasks.workunit.client.0.smithi031.stdout:8/684: creat d5/da/d12/d37/d34/d6f/d5b/dcb/fd0 x:0 0 0 2024-04-22T23:45:50.912 INFO:tasks.workunit.client.0.smithi031.stdout:1/749: write d1/d21/d4b/d6f/dd8/faf [527293,69456] 0 2024-04-22T23:45:50.915 INFO:tasks.workunit.client.0.smithi031.stdout:7/750: dread d9/d15/d3a/d41/f54 [0,4194304] 0 2024-04-22T23:45:50.918 INFO:tasks.workunit.client.0.smithi031.stdout:8/685: getdents d5/da/d12/d37/d4e 0 2024-04-22T23:45:50.919 INFO:tasks.workunit.client.0.smithi031.stdout:7/751: rmdir d9/d14/d1f/d7f 39 2024-04-22T23:45:50.919 INFO:tasks.workunit.client.0.smithi031.stdout:8/686: chown d5/da/d12/d37/d4e/l72 30 1 2024-04-22T23:45:50.921 INFO:tasks.workunit.client.0.smithi031.stdout:1/750: dwrite d1/db/d16/f83 [0,4194304] 0 2024-04-22T23:45:50.927 INFO:tasks.workunit.client.0.smithi031.stdout:1/751: write d1/d3f/da1/df3/fa3 [4937096,14196] 0 2024-04-22T23:45:50.928 INFO:tasks.workunit.client.0.smithi031.stdout:7/752: write d9/f3b [639344,94241] 0 2024-04-22T23:45:50.953 INFO:tasks.workunit.client.0.smithi031.stdout:7/753: mkdir d9/d14/d1f/dc6/d103 0 2024-04-22T23:45:50.954 INFO:tasks.workunit.client.0.smithi031.stdout:0/715: sync 2024-04-22T23:45:50.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/752: dwrite d1/d35/d36/fbd [0,4194304] 0 2024-04-22T23:45:50.960 INFO:tasks.workunit.client.0.smithi031.stdout:1/753: write d1/db/d16/d1f/fe2 [4288054,2769] 0 2024-04-22T23:45:50.963 INFO:tasks.workunit.client.0.smithi031.stdout:0/716: dwrite d0/d13/d18/d1e/f6f [0,4194304] 0 2024-04-22T23:45:50.970 INFO:tasks.workunit.client.0.smithi031.stdout:1/754: dwrite d1/de/f11 [0,4194304] 0 2024-04-22T23:45:50.972 INFO:tasks.workunit.client.0.smithi031.stdout:0/717: rename d0/c8a to d0/d1/d2/d36/d11/d1a/d56/ce8 0 2024-04-22T23:45:50.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/755: mknod d1/db/d16/d2f/cfd 0 2024-04-22T23:45:50.976 INFO:tasks.workunit.client.0.smithi031.stdout:1/756: fdatasync d1/d21/d4b/f5d 0 2024-04-22T23:45:50.981 INFO:tasks.workunit.client.0.smithi031.stdout:1/757: symlink d1/db/d16/d2f/d81/lfe 0 2024-04-22T23:45:50.998 INFO:tasks.workunit.client.0.smithi031.stdout:8/687: truncate d5/da/d12/f14 913266 0 2024-04-22T23:45:51.003 INFO:tasks.workunit.client.0.smithi031.stdout:8/688: mkdir d5/d48/d61/d6b/d6c/dd1 0 2024-04-22T23:45:51.004 INFO:tasks.workunit.client.0.smithi031.stdout:1/758: dread d1/d3f/da1/df3/fb7 [0,4194304] 0 2024-04-22T23:45:51.007 INFO:tasks.workunit.client.0.smithi031.stdout:8/689: unlink d5/d48/d61/lc2 0 2024-04-22T23:45:51.010 INFO:tasks.workunit.client.0.smithi031.stdout:8/690: readlink d5/da/d83/d8d/lae 0 2024-04-22T23:45:51.011 INFO:tasks.workunit.client.0.smithi031.stdout:8/691: readlink d5/l64 0 2024-04-22T23:45:51.011 INFO:tasks.workunit.client.0.smithi031.stdout:7/754: dwrite d9/d15/d3a/d41/f5a [4194304,4194304] 0 2024-04-22T23:45:51.012 INFO:tasks.workunit.client.0.smithi031.stdout:8/692: fsync d5/da/d12/d37/d34/d6f/f4b 0 2024-04-22T23:45:51.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/759: creat d1/d21/d4e/dd9/fff x:0 0 0 2024-04-22T23:45:51.016 INFO:tasks.workunit.client.0.smithi031.stdout:7/755: stat d9/d14/d1f/d45/da6/daa/cd9 0 2024-04-22T23:45:51.017 INFO:tasks.workunit.client.0.smithi031.stdout:7/756: chown d9/d15/d3a/f51 129335229 1 2024-04-22T23:45:51.020 INFO:tasks.workunit.client.0.smithi031.stdout:8/693: creat d5/d48/d61/d6b/fd2 x:0 0 0 2024-04-22T23:45:51.021 INFO:tasks.workunit.client.0.smithi031.stdout:8/694: chown d5/da/d83/d8d/lae 6141 1 2024-04-22T23:45:51.026 INFO:tasks.workunit.client.0.smithi031.stdout:7/757: truncate d9/d15/d6d/d65/f6b 8751068 0 2024-04-22T23:45:51.029 INFO:tasks.workunit.client.0.smithi031.stdout:0/718: write d0/d13/d18/f6d [1373336,83174] 0 2024-04-22T23:45:51.029 INFO:tasks.workunit.client.0.smithi031.stdout:1/760: dread d1/db/d16/f25 [0,4194304] 0 2024-04-22T23:45:51.030 INFO:tasks.workunit.client.0.smithi031.stdout:7/758: creat d9/d14/d1f/d45/de3/f104 x:0 0 0 2024-04-22T23:45:51.030 INFO:tasks.workunit.client.0.smithi031.stdout:0/719: read d0/d1/d2/d36/d11/f14 [1416251,60717] 0 2024-04-22T23:45:51.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/761: symlink d1/d3f/l100 0 2024-04-22T23:45:51.033 INFO:tasks.workunit.client.0.smithi031.stdout:1/762: write d1/d3f/d54/dc6/fef [608865,58109] 0 2024-04-22T23:45:51.033 INFO:tasks.workunit.client.0.smithi031.stdout:7/759: symlink d9/d15/d3f/ddc/l105 0 2024-04-22T23:45:51.035 INFO:tasks.workunit.client.0.smithi031.stdout:7/760: write d9/d15/d3f/ddc/fdd [1284939,83899] 0 2024-04-22T23:45:51.035 INFO:tasks.workunit.client.0.smithi031.stdout:0/720: readlink d0/d1/d2/d36/d11/d1c/la0 0 2024-04-22T23:45:51.036 INFO:tasks.workunit.client.0.smithi031.stdout:7/761: rmdir d9 39 2024-04-22T23:45:51.038 INFO:tasks.workunit.client.0.smithi031.stdout:8/695: dread d5/da/d83/fc5 [0,4194304] 0 2024-04-22T23:45:51.039 INFO:tasks.workunit.client.0.smithi031.stdout:0/721: unlink d0/d13/d18/d6c/l96 0 2024-04-22T23:45:51.039 INFO:tasks.workunit.client.0.smithi031.stdout:0/722: stat d0/d13/l80 0 2024-04-22T23:45:51.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/696: write d5/da/d12/d37/d34/d6f/d5b/fbf [1684157,47109] 0 2024-04-22T23:45:51.042 INFO:tasks.workunit.client.0.smithi031.stdout:0/723: creat d0/d13/d18/d1e/dd3/fe9 x:0 0 0 2024-04-22T23:45:51.047 INFO:tasks.workunit.client.0.smithi031.stdout:8/697: dwrite d5/f8 [0,4194304] 0 2024-04-22T23:45:51.047 INFO:tasks.workunit.client.0.smithi031.stdout:7/762: rmdir d9/d14/d1f/d45/da6/dae 0 2024-04-22T23:45:51.048 INFO:tasks.workunit.client.0.smithi031.stdout:0/724: mkdir d0/d13/d18/d1e/d25/dea 0 2024-04-22T23:45:51.050 INFO:tasks.workunit.client.0.smithi031.stdout:8/698: unlink d5/da/f2c 0 2024-04-22T23:45:51.050 INFO:tasks.workunit.client.0.smithi031.stdout:7/763: mkdir d9/d15/d3f/ddc/d106 0 2024-04-22T23:45:51.051 INFO:tasks.workunit.client.0.smithi031.stdout:7/764: readlink d9/d14/d22/l97 0 2024-04-22T23:45:51.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/699: truncate d5/da/d12/d37/d34/fcf 14129 0 2024-04-22T23:45:51.051 INFO:tasks.workunit.client.0.smithi031.stdout:0/725: rmdir d0/d13/d18/d1e/d25/d8e/da7/d2b 39 2024-04-22T23:45:51.052 INFO:tasks.workunit.client.0.smithi031.stdout:0/726: dread - d0/d1/d2/f7a zero size 2024-04-22T23:45:51.057 INFO:tasks.workunit.client.0.smithi031.stdout:8/700: link d5/da/d12/d1d/f8b d5/da/d12/d2e/d4d/d56/fd3 0 2024-04-22T23:45:51.059 INFO:tasks.workunit.client.0.smithi031.stdout:0/727: rename d0/d1/d2/l6 to d0/d13/d18/d1e/d25/d8e/dac/dda/leb 0 2024-04-22T23:45:51.060 INFO:tasks.workunit.client.0.smithi031.stdout:8/701: chown d5/da/d83/d8d/c80 3729 1 2024-04-22T23:45:51.061 INFO:tasks.workunit.client.0.smithi031.stdout:0/728: fdatasync d0/d13/d18/d6c/f74 0 2024-04-22T23:45:51.061 INFO:tasks.workunit.client.0.smithi031.stdout:8/702: chown d5/da/d12/d2e/d7e/fb4 0 1 2024-04-22T23:45:51.062 INFO:tasks.workunit.client.0.smithi031.stdout:0/729: chown d0/d13/d18/d6c/la6 6160204 1 2024-04-22T23:45:51.066 INFO:tasks.workunit.client.0.smithi031.stdout:0/730: truncate d0/d13/fe3 4556850 0 2024-04-22T23:45:51.068 INFO:tasks.workunit.client.0.smithi031.stdout:0/731: symlink d0/d13/d18/d1e/dd3/lec 0 2024-04-22T23:45:51.070 INFO:tasks.workunit.client.0.smithi031.stdout:0/732: stat d0/d13/d18/d1e/d25/ldd 0 2024-04-22T23:45:51.074 INFO:tasks.workunit.client.0.smithi031.stdout:0/733: creat d0/d13/d18/d1e/d25/d8e/da7/fed x:0 0 0 2024-04-22T23:45:51.075 INFO:tasks.workunit.client.0.smithi031.stdout:0/734: stat d0/d1/d2/d36/d11/fe5 0 2024-04-22T23:45:51.077 INFO:tasks.workunit.client.0.smithi031.stdout:0/735: fdatasync d0/d13/d18/d1e/d25/d8e/dac/dda/fe1 0 2024-04-22T23:45:51.080 INFO:tasks.workunit.client.0.smithi031.stdout:0/736: getdents d0/d1 0 2024-04-22T23:45:51.082 INFO:tasks.workunit.client.0.smithi031.stdout:0/737: fdatasync d0/d1/d2/f23 0 2024-04-22T23:45:51.084 INFO:tasks.workunit.client.0.smithi031.stdout:0/738: rmdir d0/d1/d2/db4 39 2024-04-22T23:45:51.086 INFO:tasks.workunit.client.0.smithi031.stdout:0/739: write d0/d1/d2/d17/f58 [2239087,69750] 0 2024-04-22T23:45:51.088 INFO:tasks.workunit.client.0.smithi031.stdout:0/740: chown d0/d1/d2/d36/d11/d1a/d3c/f3e 454320 1 2024-04-22T23:45:51.089 INFO:tasks.workunit.client.0.smithi031.stdout:0/741: creat d0/fee x:0 0 0 2024-04-22T23:45:51.105 INFO:tasks.workunit.client.0.smithi031.stdout:0/742: write d0/d13/d18/f48 [668285,65346] 0 2024-04-22T23:45:51.105 INFO:tasks.workunit.client.0.smithi031.stdout:7/765: write d9/d15/d6d/d65/f6b [7798997,81461] 0 2024-04-22T23:45:51.108 INFO:tasks.workunit.client.0.smithi031.stdout:8/703: dwrite d5/da/d12/d2e/f7d [0,4194304] 0 2024-04-22T23:45:51.111 INFO:tasks.workunit.client.0.smithi031.stdout:0/743: dread d0/d13/d18/dbf/fdb [0,4194304] 0 2024-04-22T23:45:51.111 INFO:tasks.workunit.client.0.smithi031.stdout:7/766: fsync d9/d14/d22/d5c/d5f/fd7 0 2024-04-22T23:45:51.112 INFO:tasks.workunit.client.0.smithi031.stdout:8/704: mkdir d5/da/df/d33/dd4 0 2024-04-22T23:45:51.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/744: creat d0/d13/d18/d1e/fef x:0 0 0 2024-04-22T23:45:51.116 INFO:tasks.workunit.client.0.smithi031.stdout:7/767: mknod d9/d14/d1f/d45/de3/dfd/c107 0 2024-04-22T23:45:51.117 INFO:tasks.workunit.client.0.smithi031.stdout:8/705: symlink d5/da/d12/d37/d34/d6f/d85/dcd/ld5 0 2024-04-22T23:45:51.118 INFO:tasks.workunit.client.0.smithi031.stdout:0/745: rename d0/fd2 to d0/d13/d18/d1e/d25/d8e/dac/ff0 0 2024-04-22T23:45:51.121 INFO:tasks.workunit.client.0.smithi031.stdout:8/706: rmdir d5/d48/d61/d6b 39 2024-04-22T23:45:51.124 INFO:tasks.workunit.client.0.smithi031.stdout:8/707: stat d5/d48/d61/d6b/fd2 0 2024-04-22T23:45:51.124 INFO:tasks.workunit.client.0.smithi031.stdout:7/768: dwrite d9/d15/d3a/d41/f8f [0,4194304] 0 2024-04-22T23:45:51.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/708: creat d5/da/df/d33/fd6 x:0 0 0 2024-04-22T23:45:51.129 INFO:tasks.workunit.client.0.smithi031.stdout:7/769: symlink d9/d14/d22/d5c/l108 0 2024-04-22T23:45:51.130 INFO:tasks.workunit.client.0.smithi031.stdout:7/770: dread - d9/d15/fe1 zero size 2024-04-22T23:45:51.135 INFO:tasks.workunit.client.0.smithi031.stdout:0/746: dwrite d0/d1/d2/d36/d11/d1c/d54/f69 [0,4194304] 0 2024-04-22T23:45:51.135 INFO:tasks.workunit.client.0.smithi031.stdout:8/709: link d5/da/d83/d8d/ca7 d5/da/d12/cd7 0 2024-04-22T23:45:51.136 INFO:tasks.workunit.client.0.smithi031.stdout:8/710: stat d5/da/d12/d37/d34/f46 0 2024-04-22T23:45:51.137 INFO:tasks.workunit.client.0.smithi031.stdout:8/711: chown d5/f76 29712 1 2024-04-22T23:45:51.139 INFO:tasks.workunit.client.0.smithi031.stdout:0/747: mknod d0/d1/d2/d36/d11/d1c/d54/cf1 0 2024-04-22T23:45:51.141 INFO:tasks.workunit.client.0.smithi031.stdout:8/712: mkdir d5/dd8 0 2024-04-22T23:45:51.144 INFO:tasks.workunit.client.0.smithi031.stdout:7/771: dwrite d9/d15/dcf/fd2 [0,4194304] 0 2024-04-22T23:45:51.146 INFO:tasks.workunit.client.0.smithi031.stdout:0/748: rename d0/d13/d18/d1e/d25/fbb to d0/d13/d18/d1e/d25/d8e/ff2 0 2024-04-22T23:45:51.151 INFO:tasks.workunit.client.0.smithi031.stdout:0/749: symlink d0/d13/d18/d6c/dad/dba/de6/lf3 0 2024-04-22T23:45:51.152 INFO:tasks.workunit.client.0.smithi031.stdout:0/750: chown d0/d13/d18/d1e/d25/c53 644020 1 2024-04-22T23:45:51.153 INFO:tasks.workunit.client.0.smithi031.stdout:7/772: creat d9/d15/f109 x:0 0 0 2024-04-22T23:45:51.159 INFO:tasks.workunit.client.0.smithi031.stdout:0/751: creat d0/d1/d2/db4/ff4 x:0 0 0 2024-04-22T23:45:51.159 INFO:tasks.workunit.client.0.smithi031.stdout:8/713: write d5/fb [1430216,35561] 0 2024-04-22T23:45:51.160 INFO:tasks.workunit.client.0.smithi031.stdout:7/773: write f6 [978857,100] 0 2024-04-22T23:45:51.163 INFO:tasks.workunit.client.0.smithi031.stdout:7/774: mknod d9/d14/d1f/d45/de3/dfd/c10a 0 2024-04-22T23:45:51.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/775: fdatasync d9/d14/d1f/d7f/fc7 0 2024-04-22T23:45:51.168 INFO:tasks.workunit.client.0.smithi031.stdout:7/776: mkdir d9/d14/d1f/d45/da6/daa/d10b 0 2024-04-22T23:45:51.170 INFO:tasks.workunit.client.0.smithi031.stdout:8/714: dwrite d5/da/d12/d37/d34/f5e [0,4194304] 0 2024-04-22T23:45:51.172 INFO:tasks.workunit.client.0.smithi031.stdout:7/777: getdents d9/d15/d6d/d65/dea 0 2024-04-22T23:45:51.176 INFO:tasks.workunit.client.0.smithi031.stdout:2/772: sync 2024-04-22T23:45:51.176 INFO:tasks.workunit.client.0.smithi031.stdout:4/898: sync 2024-04-22T23:45:51.177 INFO:tasks.workunit.client.0.smithi031.stdout:2/773: stat d6/d45/d50 0 2024-04-22T23:45:51.178 INFO:tasks.workunit.client.0.smithi031.stdout:0/752: dread d0/d1/d2/d36/d11/d1a/d56/fae [0,4194304] 0 2024-04-22T23:45:51.180 INFO:tasks.workunit.client.0.smithi031.stdout:7/778: link d9/d14/d1f/d45/de3/dfd/c10a d9/d14/d22/d28/d7a/c10c 0 2024-04-22T23:45:51.182 INFO:tasks.workunit.client.0.smithi031.stdout:0/753: readlink d0/d13/d18/d1e/d25/d8e/l95 0 2024-04-22T23:45:51.183 INFO:tasks.workunit.client.0.smithi031.stdout:2/774: rename d6/dc/d57/d52/db5/lc8 to d6/dc/d49/lfc 0 2024-04-22T23:45:51.185 INFO:tasks.workunit.client.0.smithi031.stdout:8/715: dwrite d5/da/d12/d1d/f8b [0,4194304] 0 2024-04-22T23:45:51.185 INFO:tasks.workunit.client.0.smithi031.stdout:7/779: creat d9/d14/dd6/f10d x:0 0 0 2024-04-22T23:45:51.187 INFO:tasks.workunit.client.0.smithi031.stdout:8/716: readlink d5/da/df/d33/l60 0 2024-04-22T23:45:51.189 INFO:tasks.workunit.client.0.smithi031.stdout:0/754: truncate d0/d13/d18/d1e/d25/d8e/dac/dda/fc1 959269 0 2024-04-22T23:45:51.193 INFO:tasks.workunit.client.0.smithi031.stdout:0/755: read - d0/f61 zero size 2024-04-22T23:45:51.193 INFO:tasks.workunit.client.0.smithi031.stdout:8/717: mknod d5/da/d12/d37/d34/d6f/d5b/dcb/cd9 0 2024-04-22T23:45:51.194 INFO:tasks.workunit.client.0.smithi031.stdout:2/775: write d6/f60 [408825,19953] 0 2024-04-22T23:45:51.195 INFO:tasks.workunit.client.0.smithi031.stdout:4/899: dwrite db/d14/d2d/d49/d59/d6a/d41/f12c [0,4194304] 0 2024-04-22T23:45:51.195 INFO:tasks.workunit.client.0.smithi031.stdout:7/780: rename d9/d14/d1f/d45/da6/daa/db0/f6e to d9/d14/d1f/d45/da6/daa/d10b/f10e 0 2024-04-22T23:45:51.202 INFO:tasks.workunit.client.0.smithi031.stdout:2/776: rename d6/dc/d42/l63 to d6/lfd 0 2024-04-22T23:45:51.207 INFO:tasks.workunit.client.0.smithi031.stdout:7/781: dwrite d9/d14/d22/f2d [0,4194304] 0 2024-04-22T23:45:51.207 INFO:tasks.workunit.client.0.smithi031.stdout:4/900: dread db/d14/d2d/d49/d59/d6a/f10b [0,4194304] 0 2024-04-22T23:45:51.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/777: mkdir d6/d45/d50/dfe 0 2024-04-22T23:45:51.208 INFO:tasks.workunit.client.0.smithi031.stdout:0/756: dwrite d0/d13/d18/d1e/d25/d8e/da7/d2b/f90 [0,4194304] 0 2024-04-22T23:45:51.209 INFO:tasks.workunit.client.0.smithi031.stdout:7/782: readlink d9/d14/d22/d5c/la1 0 2024-04-22T23:45:51.210 INFO:tasks.workunit.client.0.smithi031.stdout:8/718: write d5/da/d12/d2e/d4d/d56/f5a [5934000,120054] 0 2024-04-22T23:45:51.214 INFO:tasks.workunit.client.0.smithi031.stdout:4/901: mkdir db/d14/d1d/d7b/d7f/d2a/d47/d80/d131 0 2024-04-22T23:45:51.216 INFO:tasks.workunit.client.0.smithi031.stdout:0/757: getdents d0/d13/d18/dbf/dc3 0 2024-04-22T23:45:51.217 INFO:tasks.workunit.client.0.smithi031.stdout:0/758: chown d0/d1/d3f/f8b 68389624 1 2024-04-22T23:45:51.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/783: write d9/d15/d3a/d41/fa3 [991288,101669] 0 2024-04-22T23:45:51.224 INFO:tasks.workunit.client.0.smithi031.stdout:4/902: dwrite db/d14/d2d/d49/d59/d6a/d63/d107/d129/dc6/dd2/fd9 [4194304,4194304] 0 2024-04-22T23:45:51.226 INFO:tasks.workunit.client.0.smithi031.stdout:8/719: creat d5/d48/d61/d6b/d6c/fda x:0 0 0 2024-04-22T23:45:51.228 INFO:tasks.workunit.client.0.smithi031.stdout:2/778: rename d6/dc/d42/d24/d32/l33 to d6/dc/d17/d69/d78/lff 0 2024-04-22T23:45:51.230 INFO:tasks.workunit.client.0.smithi031.stdout:7/784: getdents d9/d14/d1f/d7f 0 2024-04-22T23:45:51.232 INFO:tasks.workunit.client.0.smithi031.stdout:7/785: truncate d9/d14/d22/d5c/f9e 5122569 0 2024-04-22T23:45:51.234 INFO:tasks.workunit.client.0.smithi031.stdout:8/720: unlink d5/da/d12/d37/d4e/f81 0 2024-04-22T23:45:51.234 INFO:tasks.workunit.client.0.smithi031.stdout:7/786: write d9/d14/d1f/d45/de3/f104 [865750,81607] 0 2024-04-22T23:45:51.236 INFO:tasks.workunit.client.0.smithi031.stdout:2/779: mknod d6/dc/d48/d4b/c100 0 2024-04-22T23:45:51.236 INFO:tasks.workunit.client.0.smithi031.stdout:8/721: chown d5/da/d12/d2e/d4d/d56/f88 0 1 2024-04-22T23:45:51.240 INFO:tasks.workunit.client.0.smithi031.stdout:4/903: rename db/d14/d2d/d111 to db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/d132 0 2024-04-22T23:45:51.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/722: symlink d5/da/df/d33/ldb 0 2024-04-22T23:45:51.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/787: rename d9/d15/dcf/ff7 to d9/d15/d6d/da9/f10f 0 2024-04-22T23:45:51.248 INFO:tasks.workunit.client.0.smithi031.stdout:0/759: dread d0/d1/d2/d36/d11/d1c/f88 [0,4194304] 0 2024-04-22T23:45:51.249 INFO:tasks.workunit.client.0.smithi031.stdout:5/749: sync 2024-04-22T23:45:51.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/763: sync 2024-04-22T23:45:51.249 INFO:tasks.workunit.client.0.smithi031.stdout:3/709: sync 2024-04-22T23:45:51.249 INFO:tasks.workunit.client.0.smithi031.stdout:6/904: sync 2024-04-22T23:45:51.249 INFO:tasks.workunit.client.0.smithi031.stdout:9/802: sync 2024-04-22T23:45:51.251 INFO:tasks.workunit.client.0.smithi031.stdout:4/904: unlink db/f5f 0 2024-04-22T23:45:51.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/905: write d3/d2a/f12d [549168,76323] 0 2024-04-22T23:45:51.252 INFO:tasks.workunit.client.0.smithi031.stdout:2/780: link d6/dc/d17/c89 d6/dc/d49/d83/dc2/c101 0 2024-04-22T23:45:51.254 INFO:tasks.workunit.client.0.smithi031.stdout:2/781: readlink d6/dc/d48/l4a 0 2024-04-22T23:45:51.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/906: chown d3/d35/d75/d7d/dc4/f123 7956644 1 2024-04-22T23:45:51.255 INFO:tasks.workunit.client.0.smithi031.stdout:3/710: write d4/d83/dd8/de6/ff5 [431300,39858] 0 2024-04-22T23:45:51.255 INFO:tasks.workunit.client.0.smithi031.stdout:7/788: symlink d9/d14/d1f/l110 0 2024-04-22T23:45:51.255 INFO:tasks.workunit.client.0.smithi031.stdout:0/760: mknod d0/d13/d18/dbf/dc3/cf5 0 2024-04-22T23:45:51.256 INFO:tasks.workunit.client.0.smithi031.stdout:9/803: symlink d2/dc/d8b/d3d/d40/da4/db1/dc5/l10b 0 2024-04-22T23:45:51.257 INFO:tasks.workunit.client.0.smithi031.stdout:5/750: mknod db/d16/d2a/d7c/d89/cf4 0 2024-04-22T23:45:51.257 INFO:tasks.workunit.client.0.smithi031.stdout:6/907: chown d3/d1f/f4f 0 1 2024-04-22T23:45:51.258 INFO:tasks.workunit.client.0.smithi031.stdout:4/905: symlink db/d14/d1d/d7b/d7f/d2a/da1/l133 0 2024-04-22T23:45:51.262 INFO:tasks.workunit.client.0.smithi031.stdout:8/723: dread d5/da/d12/d37/d34/d6f/d85/d9b/f4f [0,4194304] 0 2024-04-22T23:45:51.265 INFO:tasks.workunit.client.0.smithi031.stdout:3/711: mkdir d4/d6/d11/d25/d44/d7f/d5f/dee/df8 0 2024-04-22T23:45:51.266 INFO:tasks.workunit.client.0.smithi031.stdout:5/751: mkdir db/d16/d1d/d24/ddf/df5 0 2024-04-22T23:45:51.271 INFO:tasks.workunit.client.0.smithi031.stdout:6/908: dwrite d3/d1f/d4c/d8e/fdd [4194304,4194304] 0 2024-04-22T23:45:51.271 INFO:tasks.workunit.client.0.smithi031.stdout:7/789: dwrite d9/d14/d22/d5c/f9a [0,4194304] 0 2024-04-22T23:45:51.273 INFO:tasks.workunit.client.0.smithi031.stdout:1/764: dread d1/de/d1e/d34/d41/f62 [0,4194304] 0 2024-04-22T23:45:51.273 INFO:tasks.workunit.client.0.smithi031.stdout:3/712: unlink d4/d6/d11/f37 0 2024-04-22T23:45:51.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/906: creat db/f134 x:0 0 0 2024-04-22T23:45:51.275 INFO:tasks.workunit.client.0.smithi031.stdout:5/752: fdatasync db/d16/f3e 0 2024-04-22T23:45:51.277 INFO:tasks.workunit.client.0.smithi031.stdout:8/724: creat d5/da/d12/d1d/d38/db9/fdc x:0 0 0 2024-04-22T23:45:51.279 INFO:tasks.workunit.client.0.smithi031.stdout:3/713: dread d4/d54/d2c/d89/d2e/d45/f60 [0,4194304] 0 2024-04-22T23:45:51.280 INFO:tasks.workunit.client.0.smithi031.stdout:2/782: write d6/dc/f4e [140472,88313] 0 2024-04-22T23:45:51.282 INFO:tasks.workunit.client.0.smithi031.stdout:7/790: truncate d9/f37 841969 0 2024-04-22T23:45:51.284 INFO:tasks.workunit.client.0.smithi031.stdout:2/783: write d6/dc/d9a/d6c/d86/f9b [210010,92754] 0 2024-04-22T23:45:51.286 INFO:tasks.workunit.client.0.smithi031.stdout:4/907: mkdir db/d14/d1d/d7b/d7f/d2a/da1/d135 0 2024-04-22T23:45:51.288 INFO:tasks.workunit.client.0.smithi031.stdout:0/761: write d0/d13/d18/d1e/f6a [1637003,38725] 0 2024-04-22T23:45:51.289 INFO:tasks.workunit.client.0.smithi031.stdout:0/762: readlink d0/d13/d18/d1e/d25/d8e/dac/dda/l9d 0 2024-04-22T23:45:51.290 INFO:tasks.workunit.client.0.smithi031.stdout:5/753: symlink db/d16/d1d/d24/d7d/lf6 0 2024-04-22T23:45:51.292 INFO:tasks.workunit.client.0.smithi031.stdout:0/763: readlink d0/d13/d18/d1e/d25/d8e/dac/dda/l9d 0 2024-04-22T23:45:51.292 INFO:tasks.workunit.client.0.smithi031.stdout:5/754: write db/d16/d2a/d36/fdb [809605,71073] 0 2024-04-22T23:45:51.293 INFO:tasks.workunit.client.0.smithi031.stdout:0/764: fsync d0/d1/d3f/fa1 0 2024-04-22T23:45:51.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/791: symlink d9/d14/d1f/d45/l111 0 2024-04-22T23:45:51.299 INFO:tasks.workunit.client.0.smithi031.stdout:0/765: mkdir d0/db5/df6 0 2024-04-22T23:45:51.302 INFO:tasks.workunit.client.0.smithi031.stdout:4/908: mkdir db/d14/d1d/dd8/d136 0 2024-04-22T23:45:51.305 INFO:tasks.workunit.client.0.smithi031.stdout:5/755: write db/d16/d2a/d59/da8/fac [1277321,102039] 0 2024-04-22T23:45:51.307 INFO:tasks.workunit.client.0.smithi031.stdout:6/909: dread d3/f15 [0,4194304] 0 2024-04-22T23:45:51.310 INFO:tasks.workunit.client.0.smithi031.stdout:7/792: dwrite d9/d14/f61 [0,4194304] 0 2024-04-22T23:45:51.317 INFO:tasks.workunit.client.0.smithi031.stdout:1/765: dwrite d1/db/f19 [0,4194304] 0 2024-04-22T23:45:51.317 INFO:tasks.workunit.client.0.smithi031.stdout:5/756: symlink db/d16/d2a/d7c/lf7 0 2024-04-22T23:45:51.317 INFO:tasks.workunit.client.0.smithi031.stdout:6/910: write d3/d1f/d60/f12a [716865,130690] 0 2024-04-22T23:45:51.320 INFO:tasks.workunit.client.0.smithi031.stdout:7/793: mknod d9/d14/d22/dbe/c112 0 2024-04-22T23:45:51.321 INFO:tasks.workunit.client.0.smithi031.stdout:6/911: readlink d3/l3b 0 2024-04-22T23:45:51.322 INFO:tasks.workunit.client.0.smithi031.stdout:4/909: chown db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62/f77 0 1 2024-04-22T23:45:51.323 INFO:tasks.workunit.client.0.smithi031.stdout:8/725: dwrite d5/f19 [0,4194304] 0 2024-04-22T23:45:51.324 INFO:tasks.workunit.client.0.smithi031.stdout:3/714: dwrite d4/d54/d6d/f9b [0,4194304] 0 2024-04-22T23:45:51.327 INFO:tasks.workunit.client.0.smithi031.stdout:2/784: dwrite d6/d45/f73 [0,4194304] 0 2024-04-22T23:45:51.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/912: unlink d3/d35/d75/d7d/c111 0 2024-04-22T23:45:51.330 INFO:tasks.workunit.client.0.smithi031.stdout:7/794: symlink d9/d14/d1f/d7f/l113 0 2024-04-22T23:45:51.332 INFO:tasks.workunit.client.0.smithi031.stdout:0/766: dread d0/d1/d2/d36/d11/f14 [0,4194304] 0 2024-04-22T23:45:51.333 INFO:tasks.workunit.client.0.smithi031.stdout:2/785: stat d6/dc/d48/d8a/d92 0 2024-04-22T23:45:51.335 INFO:tasks.workunit.client.0.smithi031.stdout:3/715: mknod d4/d6/d4c/d8c/cf9 0 2024-04-22T23:45:51.335 INFO:tasks.workunit.client.0.smithi031.stdout:6/913: chown d3/d35/f94 8960917 1 2024-04-22T23:45:51.336 INFO:tasks.workunit.client.0.smithi031.stdout:8/726: readlink d5/da/d12/d37/d34/l68 0 2024-04-22T23:45:51.337 INFO:tasks.workunit.client.0.smithi031.stdout:5/757: rename db/d8d/d9b/ld8 to db/d16/d1d/d24/d27/lf8 0 2024-04-22T23:45:51.338 INFO:tasks.workunit.client.0.smithi031.stdout:7/795: write d9/d14/d1f/d7f/fc7 [4846668,88941] 0 2024-04-22T23:45:51.338 INFO:tasks.workunit.client.0.smithi031.stdout:2/786: fsync d6/dc/d17/d69/f7f 0 2024-04-22T23:45:51.342 INFO:tasks.workunit.client.0.smithi031.stdout:0/767: truncate d0/d1/d2/d36/d11/fe5 336062 0 2024-04-22T23:45:51.342 INFO:tasks.workunit.client.0.smithi031.stdout:7/796: chown d9/c13 47935 1 2024-04-22T23:45:51.344 INFO:tasks.workunit.client.0.smithi031.stdout:3/716: creat d4/d6/d11/d25/d44/d7f/d5f/ffa x:0 0 0 2024-04-22T23:45:51.346 INFO:tasks.workunit.client.0.smithi031.stdout:2/787: mknod d6/dc/d42/d24/d32/c102 0 2024-04-22T23:45:51.347 INFO:tasks.workunit.client.0.smithi031.stdout:1/766: dwrite d1/d3f/d54/d84/f3e [4194304,4194304] 0 2024-04-22T23:45:51.349 INFO:tasks.workunit.client.0.smithi031.stdout:2/788: readlink d6/dc/d49/d83/dc2/le7 0 2024-04-22T23:45:51.350 INFO:tasks.workunit.client.0.smithi031.stdout:4/910: rename db/d14/d1d/d7b/d7f/d2a to db/d137 0 2024-04-22T23:45:51.351 INFO:tasks.workunit.client.0.smithi031.stdout:0/768: truncate d0/d13/d18/d1e/d25/d8e/dac/dda/fc1 1201186 0 2024-04-22T23:45:51.354 INFO:tasks.workunit.client.0.smithi031.stdout:1/767: dread d1/d35/d36/f9e [0,4194304] 0 2024-04-22T23:45:51.362 INFO:tasks.workunit.client.0.smithi031.stdout:2/789: symlink d6/dc/d9a/d6c/db2/df0/d94/l103 0 2024-04-22T23:45:51.363 INFO:tasks.workunit.client.0.smithi031.stdout:2/790: dread - d6/dc/d9a/d6c/db2/df0/d94/fe0 zero size 2024-04-22T23:45:51.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/769: unlink d0/d1/d3f/f8b 0 2024-04-22T23:45:51.366 INFO:tasks.workunit.client.0.smithi031.stdout:5/758: link f7 db/d16/d1d/d24/d32/ff9 0 2024-04-22T23:45:51.367 INFO:tasks.workunit.client.0.smithi031.stdout:7/797: creat d9/d15/f114 x:0 0 0 2024-04-22T23:45:51.368 INFO:tasks.workunit.client.0.smithi031.stdout:6/914: dread d3/d1f/d4c/d8e/fdd [0,4194304] 0 2024-04-22T23:45:51.370 INFO:tasks.workunit.client.0.smithi031.stdout:4/911: creat db/d14/d1d/dd8/d136/f138 x:0 0 0 2024-04-22T23:45:51.371 INFO:tasks.workunit.client.0.smithi031.stdout:2/791: rmdir d6/dc/d57/d52 39 2024-04-22T23:45:51.373 INFO:tasks.workunit.client.0.smithi031.stdout:2/792: write d6/dc/d42/d24/f39 [2858555,45159] 0 2024-04-22T23:45:51.373 INFO:tasks.workunit.client.0.smithi031.stdout:7/798: symlink d9/d14/dd6/l115 0 2024-04-22T23:45:51.376 INFO:tasks.workunit.client.0.smithi031.stdout:8/727: rename d5/da/d12/d37/d34/d6f/d85/d9b/f54 to d5/da/d12/d37/d34/d6f/fdd 0 2024-04-22T23:45:51.377 INFO:tasks.workunit.client.0.smithi031.stdout:4/912: chown db/d14/d1d/l2c 1768923469 1 2024-04-22T23:45:51.378 INFO:tasks.workunit.client.0.smithi031.stdout:4/913: stat db/d14/d2d/d49/d59/d6a/d63/d107/d40/fb2 0 2024-04-22T23:45:51.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/770: mknod d0/d1/d2/cf7 0 2024-04-22T23:45:51.381 INFO:tasks.workunit.client.0.smithi031.stdout:3/717: getdents d4/d6/d11 0 2024-04-22T23:45:51.384 INFO:tasks.workunit.client.0.smithi031.stdout:2/793: fdatasync d6/dc/d9a/d6c/db2/df0/f37 0 2024-04-22T23:45:51.385 INFO:tasks.workunit.client.0.smithi031.stdout:8/728: creat d5/da/d12/d37/d34/d6f/d5b/fde x:0 0 0 2024-04-22T23:45:51.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/914: creat db/d14/d1d/f139 x:0 0 0 2024-04-22T23:45:51.386 INFO:tasks.workunit.client.0.smithi031.stdout:0/771: rename d0/d13/f81 to d0/d1/d2/db4/de4/ff8 0 2024-04-22T23:45:51.388 INFO:tasks.workunit.client.0.smithi031.stdout:1/768: dwrite d1/d3f/d54/dba/f1a [0,4194304] 0 2024-04-22T23:45:51.390 INFO:tasks.workunit.client.0.smithi031.stdout:0/772: fdatasync d0/d1/f45 0 2024-04-22T23:45:51.390 INFO:tasks.workunit.client.0.smithi031.stdout:4/915: mkdir db/d14/d1d/d7b/dc1/d13a 0 2024-04-22T23:45:51.394 INFO:tasks.workunit.client.0.smithi031.stdout:2/794: creat d6/dc/d48/d4b/d6b/de8/de6/f104 x:0 0 0 2024-04-22T23:45:51.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/799: dwrite d9/d14/f75 [0,4194304] 0 2024-04-22T23:45:51.397 INFO:tasks.workunit.client.0.smithi031.stdout:8/729: rename d5/da/d12/f14 to d5/da/d12/d1d/d38/db9/fdf 0 2024-04-22T23:45:51.398 INFO:tasks.workunit.client.0.smithi031.stdout:0/773: truncate d0/d1/d2/db4/ff4 57732 0 2024-04-22T23:45:51.399 INFO:tasks.workunit.client.0.smithi031.stdout:2/795: dread - d6/dc/d48/d8a/d92/ff4 zero size 2024-04-22T23:45:51.401 INFO:tasks.workunit.client.0.smithi031.stdout:4/916: fsync db/d14/d2d/d49/d59/d6a/d63/dc9/f104 0 2024-04-22T23:45:51.402 INFO:tasks.workunit.client.0.smithi031.stdout:3/718: truncate d4/d6/d11/d25/d44/d7f/f7d 4835168 0 2024-04-22T23:45:51.403 INFO:tasks.workunit.client.0.smithi031.stdout:7/800: dread - d9/d15/fe4 zero size 2024-04-22T23:45:51.403 INFO:tasks.workunit.client.0.smithi031.stdout:1/769: dwrite d1/d21/f9b [0,4194304] 0 2024-04-22T23:45:51.407 INFO:tasks.workunit.client.0.smithi031.stdout:2/796: write d6/f60 [584992,82438] 0 2024-04-22T23:45:51.407 INFO:tasks.workunit.client.0.smithi031.stdout:7/801: fsync d9/d14/f61 0 2024-04-22T23:45:51.407 INFO:tasks.workunit.client.0.smithi031.stdout:0/774: readlink d0/d1/d2/d36/d11/d1c/d54/lb9 0 2024-04-22T23:45:51.410 INFO:tasks.workunit.client.0.smithi031.stdout:1/770: rmdir d1/db3 39 2024-04-22T23:45:51.412 INFO:tasks.workunit.client.0.smithi031.stdout:4/917: creat db/d137/d47/d80/f13b x:0 0 0 2024-04-22T23:45:51.413 INFO:tasks.workunit.client.0.smithi031.stdout:3/719: creat d4/d54/d6d/d98/ffb x:0 0 0 2024-04-22T23:45:51.415 INFO:tasks.workunit.client.0.smithi031.stdout:2/797: mknod d6/dc/d48/d8a/d92/c105 0 2024-04-22T23:45:51.416 INFO:tasks.workunit.client.0.smithi031.stdout:0/775: chown d0/d13/d18/d1e/d25/l76 616697 1 2024-04-22T23:45:51.417 INFO:tasks.workunit.client.0.smithi031.stdout:8/730: mkdir d5/da/d12/d2e/db0/dc3/de0 0 2024-04-22T23:45:51.420 INFO:tasks.workunit.client.0.smithi031.stdout:2/798: read d6/dc/d42/f2c [186681,34916] 0 2024-04-22T23:45:51.420 INFO:tasks.workunit.client.0.smithi031.stdout:4/918: mkdir db/d14/d2d/d49/d59/d6a/d63/d107/d129/dc6/d13c 0 2024-04-22T23:45:51.422 INFO:tasks.workunit.client.0.smithi031.stdout:7/802: mknod d9/d15/d3f/ddc/d106/c116 0 2024-04-22T23:45:51.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/919: chown db/d14/d2d/d49/d59/d6a/d63/d86/df2/d10f 86509 1 2024-04-22T23:45:51.423 INFO:tasks.workunit.client.0.smithi031.stdout:8/731: mkdir d5/d48/d61/d6b/de1 0 2024-04-22T23:45:51.424 INFO:tasks.workunit.client.0.smithi031.stdout:1/771: mkdir d1/db3/d101 0 2024-04-22T23:45:51.429 INFO:tasks.workunit.client.0.smithi031.stdout:4/920: chown db/d14/d2d/l38 8678 1 2024-04-22T23:45:51.433 INFO:tasks.workunit.client.0.smithi031.stdout:1/772: rename d1/db/d16/d2f to d1/d35/d8d/d102 0 2024-04-22T23:45:51.434 INFO:tasks.workunit.client.0.smithi031.stdout:7/803: creat d9/d14/d22/d5c/de0/f117 x:0 0 0 2024-04-22T23:45:51.435 INFO:tasks.workunit.client.0.smithi031.stdout:8/732: symlink d5/da/d12/d37/d34/d6f/le2 0 2024-04-22T23:45:51.436 INFO:tasks.workunit.client.0.smithi031.stdout:8/733: truncate d5/da/d12/d1d/fca 3101776 0 2024-04-22T23:45:51.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/921: symlink db/d14/d2d/d49/d59/d6a/d63/d107/d129/dde/l13d 0 2024-04-22T23:45:51.438 INFO:tasks.workunit.client.0.smithi031.stdout:2/799: rename d6/dc/d57/d52/db5/lcb to d6/dc/d48/d4b/d6b/de8/l106 0 2024-04-22T23:45:51.440 INFO:tasks.workunit.client.0.smithi031.stdout:7/804: symlink d9/d14/d1f/d45/da6/daa/db0/l118 0 2024-04-22T23:45:51.440 INFO:tasks.workunit.client.0.smithi031.stdout:8/734: truncate d5/da/df/f7f 389085 0 2024-04-22T23:45:51.441 INFO:tasks.workunit.client.0.smithi031.stdout:4/922: rmdir db/d14/d2d/d49/d59/d6a/d63/d107/d40 39 2024-04-22T23:45:51.442 INFO:tasks.workunit.client.0.smithi031.stdout:0/776: truncate d0/d1/d2/db4/ff4 47421 0 2024-04-22T23:45:51.443 INFO:tasks.workunit.client.0.smithi031.stdout:3/720: dread d4/d6/f23 [0,4194304] 0 2024-04-22T23:45:51.443 INFO:tasks.workunit.client.0.smithi031.stdout:1/773: rename d1/d3f/d54/lad to d1/d3f/l103 0 2024-04-22T23:45:51.444 INFO:tasks.workunit.client.0.smithi031.stdout:4/923: fsync db/d14/d1d/d7b/d7f/ff0 0 2024-04-22T23:45:51.449 INFO:tasks.workunit.client.0.smithi031.stdout:0/777: creat d0/d13/d18/dbf/ff9 x:0 0 0 2024-04-22T23:45:51.451 INFO:tasks.workunit.client.0.smithi031.stdout:1/774: truncate d1/d21/d4e/dd9/fff 469518 0 2024-04-22T23:45:51.452 INFO:tasks.workunit.client.0.smithi031.stdout:4/924: dread - db/d14/d2d/d49/d59/d6a/d63/d107/d129/f122 zero size 2024-04-22T23:45:51.455 INFO:tasks.workunit.client.0.smithi031.stdout:1/775: creat d1/d3f/f104 x:0 0 0 2024-04-22T23:45:51.459 INFO:tasks.workunit.client.0.smithi031.stdout:1/776: stat d1/d3f/da1/df3 0 2024-04-22T23:45:51.459 INFO:tasks.workunit.client.0.smithi031.stdout:8/735: dwrite d5/da/d12/d1d/d38/fc6 [0,4194304] 0 2024-04-22T23:45:51.459 INFO:tasks.workunit.client.0.smithi031.stdout:4/925: symlink db/d137/da1/d135/l13e 0 2024-04-22T23:45:51.464 INFO:tasks.workunit.client.0.smithi031.stdout:8/736: dread - d5/d48/fba zero size 2024-04-22T23:45:51.468 INFO:tasks.workunit.client.0.smithi031.stdout:3/721: link d4/d6/d22/l70 d4/d49/d72/lfc 0 2024-04-22T23:45:51.468 INFO:tasks.workunit.client.0.smithi031.stdout:7/805: dread d9/d14/d1f/d45/da6/daa/d10b/f10e [0,4194304] 0 2024-04-22T23:45:51.470 INFO:tasks.workunit.client.0.smithi031.stdout:0/778: dread d0/d13/d18/d1e/d25/d8e/da7/d2b/f90 [0,4194304] 0 2024-04-22T23:45:51.470 INFO:tasks.workunit.client.0.smithi031.stdout:1/777: creat d1/d35/d8d/d102/dc1/f105 x:0 0 0 2024-04-22T23:45:51.472 INFO:tasks.workunit.client.0.smithi031.stdout:2/800: dread d6/dc/d42/f67 [0,4194304] 0 2024-04-22T23:45:51.473 INFO:tasks.workunit.client.0.smithi031.stdout:1/778: chown d1/de/d1e/d34/d41/f44 33140771 1 2024-04-22T23:45:51.477 INFO:tasks.workunit.client.0.smithi031.stdout:8/737: unlink d5/da/d12/d37/d34/d6f/d85/dcd/ld5 0 2024-04-22T23:45:51.478 INFO:tasks.workunit.client.0.smithi031.stdout:0/779: dwrite d0/f3d [0,4194304] 0 2024-04-22T23:45:51.481 INFO:tasks.workunit.client.0.smithi031.stdout:2/801: dread d6/f8b [0,4194304] 0 2024-04-22T23:45:51.485 INFO:tasks.workunit.client.0.smithi031.stdout:0/780: dread d0/d1/d2/db4/de4/ff8 [0,4194304] 0 2024-04-22T23:45:51.485 INFO:tasks.workunit.client.0.smithi031.stdout:8/738: unlink d5/da/d12/d2e/d7e/c9d 0 2024-04-22T23:45:51.486 INFO:tasks.workunit.client.0.smithi031.stdout:7/806: dread d9/d14/d22/fb9 [0,4194304] 0 2024-04-22T23:45:51.487 INFO:tasks.workunit.client.0.smithi031.stdout:1/779: symlink d1/de/d1e/l106 0 2024-04-22T23:45:51.488 INFO:tasks.workunit.client.0.smithi031.stdout:7/807: stat d9/d15/c32 0 2024-04-22T23:45:51.489 INFO:tasks.workunit.client.0.smithi031.stdout:4/926: rename db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62/lb4 to db/d14/d2d/d49/d59/d6a/d63/d107/d129/dde/l13f 0 2024-04-22T23:45:51.491 INFO:tasks.workunit.client.0.smithi031.stdout:3/722: getdents d4/d6/d11/d25/d44/d7f/da5 0 2024-04-22T23:45:51.496 INFO:tasks.workunit.client.0.smithi031.stdout:7/808: symlink d9/d15/d6d/l119 0 2024-04-22T23:45:51.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/780: dread d1/d35/d8d/d102/f75 [0,4194304] 0 2024-04-22T23:45:51.499 INFO:tasks.workunit.client.0.smithi031.stdout:8/739: dwrite d5/da/d12/d1d/d38/f3f [0,4194304] 0 2024-04-22T23:45:51.500 INFO:tasks.workunit.client.0.smithi031.stdout:3/723: unlink d4/d54/d2c/d89/d2e/l76 0 2024-04-22T23:45:51.503 INFO:tasks.workunit.client.0.smithi031.stdout:0/781: link d0/fbe d0/d1/d2/d36/d11/d1a/ffa 0 2024-04-22T23:45:51.504 INFO:tasks.workunit.client.0.smithi031.stdout:4/927: write db/d14/d1d/d7b/dc1/fed [223250,125875] 0 2024-04-22T23:45:51.506 INFO:tasks.workunit.client.0.smithi031.stdout:2/802: dwrite d6/dc/d49/d83/dc2/feb [0,4194304] 0 2024-04-22T23:45:51.507 INFO:tasks.workunit.client.0.smithi031.stdout:1/781: chown d1/de/d1e/c61 52671 1 2024-04-22T23:45:51.509 INFO:tasks.workunit.client.0.smithi031.stdout:8/740: rmdir d5/da/d12/d37/d4e 39 2024-04-22T23:45:51.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/809: creat d9/d15/f11a x:0 0 0 2024-04-22T23:45:51.515 INFO:tasks.workunit.client.0.smithi031.stdout:8/741: rmdir d5/d48/d61/d6b 39 2024-04-22T23:45:51.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/810: chown d9/d15/d3a/d41/c46 4741317 1 2024-04-22T23:45:51.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/782: rename d0/d1/d2/d36/d11/f43 to d0/db5/df6/ffb 0 2024-04-22T23:45:51.519 INFO:tasks.workunit.client.0.smithi031.stdout:8/742: truncate d5/da/d12/d1d/d38/db9/fdc 445566 0 2024-04-22T23:45:51.520 INFO:tasks.workunit.client.0.smithi031.stdout:7/811: rmdir d9/d15/dbf 39 2024-04-22T23:45:51.525 INFO:tasks.workunit.client.0.smithi031.stdout:0/783: rename d0/d13/d18/d6c to d0/d1/d2/d36/d11/d1a/dfc 0 2024-04-22T23:45:51.526 INFO:tasks.workunit.client.0.smithi031.stdout:7/812: dread d9/d15/d3f/ddc/fdd [0,4194304] 0 2024-04-22T23:45:51.527 INFO:tasks.workunit.client.0.smithi031.stdout:8/743: truncate d5/da/f59 516125 0 2024-04-22T23:45:51.529 INFO:tasks.workunit.client.0.smithi031.stdout:8/744: write d5/da/d12/d37/faf [706364,114221] 0 2024-04-22T23:45:51.529 INFO:tasks.workunit.client.0.smithi031.stdout:0/784: write d0/d13/d18/d1e/d25/d8e/ff2 [1304842,44300] 0 2024-04-22T23:45:51.533 INFO:tasks.workunit.client.0.smithi031.stdout:0/785: dread d0/d1/d3f/fb0 [0,4194304] 0 2024-04-22T23:45:51.533 INFO:tasks.workunit.client.0.smithi031.stdout:1/782: truncate d1/de/d1e/d34/d41/f58 353343 0 2024-04-22T23:45:51.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/804: sync 2024-04-22T23:45:51.585 INFO:tasks.workunit.client.0.smithi031.stdout:8/745: symlink d5/da/df/le3 0 2024-04-22T23:45:51.588 INFO:tasks.workunit.client.0.smithi031.stdout:3/724: dwrite d4/d54/d6d/faa [0,4194304] 0 2024-04-22T23:45:51.588 INFO:tasks.workunit.client.0.smithi031.stdout:2/803: dread d6/dc/d49/d83/d85/fa1 [0,4194304] 0 2024-04-22T23:45:51.591 INFO:tasks.workunit.client.0.smithi031.stdout:3/725: dread - d4/d54/d2c/d89/d2e/d45/fe2 zero size 2024-04-22T23:45:51.593 INFO:tasks.workunit.client.0.smithi031.stdout:4/928: dread db/d14/d2d/d49/d59/d6a/d63/d107/d40/d62/f77 [0,4194304] 0 2024-04-22T23:45:51.594 INFO:tasks.workunit.client.0.smithi031.stdout:3/726: dread - d4/d6/d11/d25/d44/d7f/fc4 zero size 2024-04-22T23:45:51.596 INFO:tasks.workunit.client.0.smithi031.stdout:0/786: rename d0/d1/d3f/l94 to d0/d1/d2/d36/d11/d1a/d3c/lfd 0 2024-04-22T23:45:51.596 INFO:tasks.workunit.client.0.smithi031.stdout:1/783: dwrite d1/d3f/d54/dba/f1b [4194304,4194304] 0 2024-04-22T23:45:51.597 INFO:tasks.workunit.client.0.smithi031.stdout:2/804: symlink d6/dc/d49/d83/d85/da9/l107 0 2024-04-22T23:45:51.597 INFO:tasks.workunit.client.0.smithi031.stdout:1/784: chown d1/de/d1e 22027050 1 2024-04-22T23:45:51.601 INFO:tasks.workunit.client.0.smithi031.stdout:1/785: write d1/d99/d63/ffc [1029994,42491] 0 2024-04-22T23:45:51.603 INFO:tasks.workunit.client.0.smithi031.stdout:9/805: getdents d2/dc/d8b/d3d/d40/da4 0 2024-04-22T23:45:51.603 INFO:tasks.workunit.client.0.smithi031.stdout:3/727: creat d4/ffd x:0 0 0 2024-04-22T23:45:51.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/728: fsync d4/d54/d6d/d98/ffb 0 2024-04-22T23:45:51.606 INFO:tasks.workunit.client.0.smithi031.stdout:8/746: rename d5/da/d12/d37/d34/d6f/d5b/dcb/cd9 to d5/da/d12/d2e/ce4 0 2024-04-22T23:45:51.608 INFO:tasks.workunit.client.0.smithi031.stdout:0/787: truncate d0/d1/d2/db4/de4/ff8 1186136 0 2024-04-22T23:45:51.609 INFO:tasks.workunit.client.0.smithi031.stdout:0/788: chown d0/d1/d2/d36/d11/d1c 2056553 1 2024-04-22T23:45:51.610 INFO:tasks.workunit.client.0.smithi031.stdout:1/786: dwrite d1/d3f/d54/dba/d28/fa8 [0,4194304] 0 2024-04-22T23:45:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:4/929: link db/d14/d2d/d49/d59/d6a/d63/d86/ld4 db/d14/d2d/d49/d59/d6a/d63/d8b/l140 0 2024-04-22T23:45:51.611 INFO:tasks.workunit.client.0.smithi031.stdout:8/747: write d5/da/d12/d37/d4e/fb5 [919401,8800] 0 2024-04-22T23:45:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:9/806: symlink d2/dc/d6d/df1/l10c 0 2024-04-22T23:45:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:1/787: chown d1/de/d1e/d34/d66/lc3 27 1 2024-04-22T23:45:51.612 INFO:tasks.workunit.client.0.smithi031.stdout:0/789: chown d0/d13/d18/d1e/d25/d8e/dac/ld1 30929819 1 2024-04-22T23:45:51.617 INFO:tasks.workunit.client.0.smithi031.stdout:1/788: mkdir d1/de/d107 0 2024-04-22T23:45:51.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/790: rmdir d0/d1/d2/d36/d11/d1a/dfc/dad 39 2024-04-22T23:45:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:0/791: mknod d0/cfe 0 2024-04-22T23:45:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:9/807: link d2/dc/d8b/d3d/d40/da4/le0 d2/dc/d6d/l10d 0 2024-04-22T23:45:51.621 INFO:tasks.workunit.client.0.smithi031.stdout:2/805: dwrite d6/f19 [0,4194304] 0 2024-04-22T23:45:51.622 INFO:tasks.workunit.client.0.smithi031.stdout:2/806: readlink d6/d45/d50/l82 0 2024-04-22T23:45:51.624 INFO:tasks.workunit.client.0.smithi031.stdout:1/789: read d1/d35/d8d/d102/f50 [1916681,89972] 0 2024-04-22T23:45:51.625 INFO:tasks.workunit.client.0.smithi031.stdout:2/807: creat d6/dc/d48/f108 x:0 0 0 2024-04-22T23:45:51.628 INFO:tasks.workunit.client.0.smithi031.stdout:1/790: unlink d1/d3f/d54/l60 0 2024-04-22T23:45:51.630 INFO:tasks.workunit.client.0.smithi031.stdout:0/792: dwrite d0/d1/d2/d36/d11/d1a/d3c/f59 [0,4194304] 0 2024-04-22T23:45:51.633 INFO:tasks.workunit.client.0.smithi031.stdout:0/793: fdatasync d0/f61 0 2024-04-22T23:45:51.638 INFO:tasks.workunit.client.0.smithi031.stdout:1/791: getdents d1/d35/d8d/d102/dc1 0 2024-04-22T23:45:51.639 INFO:tasks.workunit.client.0.smithi031.stdout:1/792: fsync d1/f1d 0 2024-04-22T23:45:51.642 INFO:tasks.workunit.client.0.smithi031.stdout:1/793: getdents d1/d35/d8d 0 2024-04-22T23:45:51.645 INFO:tasks.workunit.client.0.smithi031.stdout:0/794: dwrite d0/d13/d18/d1e/d25/d8e/da7/f79 [0,4194304] 0 2024-04-22T23:45:51.647 INFO:tasks.workunit.client.0.smithi031.stdout:8/748: dread d5/d48/d61/d6b/fa1 [0,4194304] 0 2024-04-22T23:45:51.648 INFO:tasks.workunit.client.0.smithi031.stdout:1/794: rename d1/d21 to d1/d35/d36/d108 0 2024-04-22T23:45:51.650 INFO:tasks.workunit.client.0.smithi031.stdout:1/795: readlink d1/d35/d8d/d102/lf4 0 2024-04-22T23:45:51.650 INFO:tasks.workunit.client.0.smithi031.stdout:8/749: dread - d5/da/df/d33/fd6 zero size 2024-04-22T23:45:51.652 INFO:tasks.workunit.client.0.smithi031.stdout:1/796: creat d1/d3f/d54/d84/f109 x:0 0 0 2024-04-22T23:45:51.653 INFO:tasks.workunit.client.0.smithi031.stdout:8/750: write d5/da/d12/d37/d34/f5e [383940,17691] 0 2024-04-22T23:45:51.656 INFO:tasks.workunit.client.0.smithi031.stdout:8/751: mknod d5/da/d12/d1d/ce5 0 2024-04-22T23:45:51.657 INFO:tasks.workunit.client.0.smithi031.stdout:2/808: dwrite d6/ff [0,4194304] 0 2024-04-22T23:45:51.659 INFO:tasks.workunit.client.0.smithi031.stdout:8/752: creat d5/dd8/fe6 x:0 0 0 2024-04-22T23:45:51.665 INFO:tasks.workunit.client.0.smithi031.stdout:1/797: dwrite d1/d99/d63/f9c [0,4194304] 0 2024-04-22T23:45:51.665 INFO:tasks.workunit.client.0.smithi031.stdout:8/753: mkdir d5/da/d12/d37/d4e/de7 0 2024-04-22T23:45:51.670 INFO:tasks.workunit.client.0.smithi031.stdout:8/754: mkdir d5/da/d12/d2e/d4d/d56/de8 0 2024-04-22T23:45:51.671 INFO:tasks.workunit.client.0.smithi031.stdout:1/798: fdatasync d1/db/d16/d1f/d52/db0/fea 0 2024-04-22T23:45:51.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/755: stat d5/da/d12/d2e/d7e/fb4 0 2024-04-22T23:45:51.672 INFO:tasks.workunit.client.0.smithi031.stdout:2/809: truncate d6/dc/d17/f3c 1588692 0 2024-04-22T23:45:51.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/810: fsync d6/dc/d17/d69/fa8 0 2024-04-22T23:45:51.674 INFO:tasks.workunit.client.0.smithi031.stdout:1/799: symlink d1/d35/d36/l10a 0 2024-04-22T23:45:51.675 INFO:tasks.workunit.client.0.smithi031.stdout:8/756: symlink d5/da/df/d33/dd4/le9 0 2024-04-22T23:45:51.676 INFO:tasks.workunit.client.0.smithi031.stdout:2/811: symlink d6/dc/d48/d4b/d6b/de8/de6/l109 0 2024-04-22T23:45:51.676 INFO:tasks.workunit.client.0.smithi031.stdout:8/757: stat d5/d48/d61/d6b/fd2 0 2024-04-22T23:45:51.680 INFO:tasks.workunit.client.0.smithi031.stdout:0/795: dread d0/d13/d18/d1e/f30 [0,4194304] 0 2024-04-22T23:45:51.681 INFO:tasks.workunit.client.0.smithi031.stdout:8/758: symlink d5/d48/d61/lea 0 2024-04-22T23:45:51.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/800: dwrite d1/d3f/da1/df3/fa3 [0,4194304] 0 2024-04-22T23:45:51.683 INFO:tasks.workunit.client.0.smithi031.stdout:2/812: rename d6/dc/d17/d69/d78 to d6/dc/d49/d83/d85/da9/d10a 0 2024-04-22T23:45:51.685 INFO:tasks.workunit.client.0.smithi031.stdout:0/796: mknod d0/d13/d18/d1e/d25/d8e/da7/d2b/cff 0 2024-04-22T23:45:51.685 INFO:tasks.workunit.client.0.smithi031.stdout:8/759: symlink d5/da/d12/d37/d34/d6f/d85/d9b/leb 0 2024-04-22T23:45:51.686 INFO:tasks.workunit.client.0.smithi031.stdout:2/813: rmdir d6/dc/d48/dc1 39 2024-04-22T23:45:51.687 INFO:tasks.workunit.client.0.smithi031.stdout:8/760: write d5/da/d12/d1d/d38/fc6 [3440668,108417] 0 2024-04-22T23:45:51.688 INFO:tasks.workunit.client.0.smithi031.stdout:8/761: stat d5/da/d12/d37/d34/d6f/d5b/fbf 0 2024-04-22T23:45:51.688 INFO:tasks.workunit.client.0.smithi031.stdout:8/762: readlink d5/da/df/le3 0 2024-04-22T23:45:51.690 INFO:tasks.workunit.client.0.smithi031.stdout:1/801: rename d1/de/d1e/fe8 to d1/d35/d36/d108/d4e/dd9/f10b 0 2024-04-22T23:45:51.691 INFO:tasks.workunit.client.0.smithi031.stdout:0/797: creat d0/d1/d2/d36/d11/d1a/f100 x:0 0 0 2024-04-22T23:45:51.692 INFO:tasks.workunit.client.0.smithi031.stdout:0/798: read - d0/d1/d3f/fa1 zero size 2024-04-22T23:45:51.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/799: stat d0/f86 0 2024-04-22T23:45:51.693 INFO:tasks.workunit.client.0.smithi031.stdout:8/763: creat d5/da/d83/d8d/fec x:0 0 0 2024-04-22T23:45:51.694 INFO:tasks.workunit.client.0.smithi031.stdout:2/814: mkdir d6/dc/d49/d83/d85/da9/d10a/d10b 0 2024-04-22T23:45:51.696 INFO:tasks.workunit.client.0.smithi031.stdout:1/802: creat d1/d35/d36/d108/d4b/d6f/da6/dec/f10c x:0 0 0 2024-04-22T23:45:51.696 INFO:tasks.workunit.client.0.smithi031.stdout:0/800: write d0/d1/d2/d36/d11/d1a/d3c/fd4 [2636468,14793] 0 2024-04-22T23:45:51.699 INFO:tasks.workunit.client.0.smithi031.stdout:2/815: mknod d6/dc/d48/d4b/d6b/de8/de6/c10c 0 2024-04-22T23:45:51.700 INFO:tasks.workunit.client.0.smithi031.stdout:2/816: write d6/f22 [4853725,112013] 0 2024-04-22T23:45:51.701 INFO:tasks.workunit.client.0.smithi031.stdout:0/801: symlink d0/d1/d2/d17/l101 0 2024-04-22T23:45:51.702 INFO:tasks.workunit.client.0.smithi031.stdout:8/764: symlink d5/da/d12/d2e/d4d/d9a/db1/dbb/led 0 2024-04-22T23:45:51.704 INFO:tasks.workunit.client.0.smithi031.stdout:2/817: read d6/f9 [4664097,15020] 0 2024-04-22T23:45:51.705 INFO:tasks.workunit.client.0.smithi031.stdout:0/802: mkdir d0/d1/d2/d17/d102 0 2024-04-22T23:45:51.709 INFO:tasks.workunit.client.0.smithi031.stdout:0/803: fdatasync d0/d13/d18/d1e/d25/f82 0 2024-04-22T23:45:51.711 INFO:tasks.workunit.client.0.smithi031.stdout:0/804: readlink d0/d13/d18/d1e/d25/d8e/da7/d2b/l7b 0 2024-04-22T23:45:51.712 INFO:tasks.workunit.client.0.smithi031.stdout:6/915: sync 2024-04-22T23:45:51.712 INFO:tasks.workunit.client.0.smithi031.stdout:5/759: sync 2024-04-22T23:45:51.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/818: truncate d6/dc/d42/d24/f8f 1647636 0 2024-04-22T23:45:51.718 INFO:tasks.workunit.client.0.smithi031.stdout:5/760: creat db/d16/d2a/d59/d90/da4/ffa x:0 0 0 2024-04-22T23:45:51.719 INFO:tasks.workunit.client.0.smithi031.stdout:1/803: dwrite d1/db/d16/d1f/fc5 [0,4194304] 0 2024-04-22T23:45:51.724 INFO:tasks.workunit.client.0.smithi031.stdout:0/805: dwrite d0/f7f [0,4194304] 0 2024-04-22T23:45:51.724 INFO:tasks.workunit.client.0.smithi031.stdout:2/819: unlink d6/dc/d49/d83/dc2/c101 0 2024-04-22T23:45:51.727 INFO:tasks.workunit.client.0.smithi031.stdout:5/761: creat db/d16/d2a/d59/ffb x:0 0 0 2024-04-22T23:45:51.727 INFO:tasks.workunit.client.0.smithi031.stdout:6/916: write d3/d2a/d3c/d40/d107/fd8 [2176950,106898] 0 2024-04-22T23:45:51.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/917: dread - d3/d2a/d85/f12e zero size 2024-04-22T23:45:51.731 INFO:tasks.workunit.client.0.smithi031.stdout:1/804: symlink d1/de/d1e/l10d 0 2024-04-22T23:45:51.733 INFO:tasks.workunit.client.0.smithi031.stdout:2/820: mkdir d6/dc/d48/d4b/d10d 0 2024-04-22T23:45:51.734 INFO:tasks.workunit.client.0.smithi031.stdout:8/765: dread d5/da/d12/d2e/d4d/d56/f88 [0,4194304] 0 2024-04-22T23:45:51.735 INFO:tasks.workunit.client.0.smithi031.stdout:2/821: dread - d6/d45/d50/fed zero size 2024-04-22T23:45:51.735 INFO:tasks.workunit.client.0.smithi031.stdout:0/806: symlink d0/d1/d2/d36/d11/d1a/dfc/dad/l103 0 2024-04-22T23:45:51.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/822: chown d6/dc/d42/d24/d32/fc4 39 1 2024-04-22T23:45:51.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/805: symlink d1/de/d1e/d34/d41/l10e 0 2024-04-22T23:45:51.737 INFO:tasks.workunit.client.0.smithi031.stdout:6/918: dwrite d3/d1f/d4c/f103 [0,4194304] 0 2024-04-22T23:45:51.739 INFO:tasks.workunit.client.0.smithi031.stdout:5/762: rename db/d16/d1d/d24/f63 to db/d16/d1d/d24/d58/ffc 0 2024-04-22T23:45:51.833 INFO:tasks.workunit.client.0.smithi031.stdout:6/919: symlink d3/d2a/d85/l138 0 2024-04-22T23:45:51.833 INFO:tasks.workunit.client.0.smithi031.stdout:1/806: truncate d1/d35/d8d/d102/f3b 4782316 0 2024-04-22T23:45:51.836 INFO:tasks.workunit.client.0.smithi031.stdout:5/763: getdents db/d16/d2a/d7c/d89 0 2024-04-22T23:45:51.836 INFO:tasks.workunit.client.0.smithi031.stdout:0/807: truncate d0/d1/d2/d36/f2f 1939325 0 2024-04-22T23:45:51.836 INFO:tasks.workunit.client.0.smithi031.stdout:1/807: rmdir d1/db/d16/d1f/d52 39 2024-04-22T23:45:51.839 INFO:tasks.workunit.client.0.smithi031.stdout:5/764: chown db/l3a 0 1 2024-04-22T23:45:51.840 INFO:tasks.workunit.client.0.smithi031.stdout:2/823: dwrite d6/dc/d48/d4b/f7d [0,4194304] 0 2024-04-22T23:45:51.840 INFO:tasks.workunit.client.0.smithi031.stdout:1/808: symlink d1/db/dca/l10f 0 2024-04-22T23:45:51.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/824: mkdir d6/dc/d9a/d10e 0 2024-04-22T23:45:51.845 INFO:tasks.workunit.client.0.smithi031.stdout:2/825: chown d6/dc/d48/d4b/d6b/de8/de6/l109 1133060 1 2024-04-22T23:45:51.846 INFO:tasks.workunit.client.0.smithi031.stdout:0/808: mkdir d0/d13/d18/d1e/d25/d8e/da7/d104 0 2024-04-22T23:45:51.850 INFO:tasks.workunit.client.0.smithi031.stdout:1/809: rename d1/d35/d36/d108/d4b/dbc to d1/d3f/d54/dc6/d110 0 2024-04-22T23:45:51.853 INFO:tasks.workunit.client.0.smithi031.stdout:1/810: mknod d1/d35/d36/d108/d4e/c111 0 2024-04-22T23:45:51.854 INFO:tasks.workunit.client.0.smithi031.stdout:6/920: dread d3/d2a/d3c/d40/f4a [0,4194304] 0 2024-04-22T23:45:51.856 INFO:tasks.workunit.client.0.smithi031.stdout:0/809: mkdir d0/d13/d18/d1e/d25/dea/d105 0 2024-04-22T23:45:51.860 INFO:tasks.workunit.client.0.smithi031.stdout:0/810: mknod d0/d13/d18/c106 0 2024-04-22T23:45:51.861 INFO:tasks.workunit.client.0.smithi031.stdout:0/811: write d0/d13/d18/d1e/d25/d8e/da7/fed [725341,2248] 0 2024-04-22T23:45:51.861 INFO:tasks.workunit.client.0.smithi031.stdout:5/765: write db/d16/d1d/d24/d32/ff9 [472059,43648] 0 2024-04-22T23:45:51.863 INFO:tasks.workunit.client.0.smithi031.stdout:2/826: getdents d6/dc/d42/d24 0 2024-04-22T23:45:51.864 INFO:tasks.workunit.client.0.smithi031.stdout:1/811: getdents d1/de/d1e 0 2024-04-22T23:45:51.866 INFO:tasks.workunit.client.0.smithi031.stdout:0/812: symlink d0/d13/d18/d1e/d25/dea/d105/l107 0 2024-04-22T23:45:51.866 INFO:tasks.workunit.client.0.smithi031.stdout:1/812: chown d1/d3f/d54/dba/c10 89 1 2024-04-22T23:45:51.867 INFO:tasks.workunit.client.0.smithi031.stdout:0/813: write d0/f7f [285478,21323] 0 2024-04-22T23:45:51.868 INFO:tasks.workunit.client.0.smithi031.stdout:5/766: rename db/d16/d2a/d59/ffb to db/d16/ffd 0 2024-04-22T23:45:51.869 INFO:tasks.workunit.client.0.smithi031.stdout:0/814: symlink d0/d1/d2/d17/l108 0 2024-04-22T23:45:51.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/921: dread d3/d1f/f5c [0,4194304] 0 2024-04-22T23:45:51.871 INFO:tasks.workunit.client.0.smithi031.stdout:2/827: rename d6/dc/ff1 to d6/dc/d48/d4b/d6b/de8/f10f 0 2024-04-22T23:45:51.871 INFO:tasks.workunit.client.0.smithi031.stdout:6/922: write d3/da4/ffb [794570,110955] 0 2024-04-22T23:45:51.872 INFO:tasks.workunit.client.0.smithi031.stdout:1/813: symlink d1/l112 0 2024-04-22T23:45:51.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/923: stat d3/d35/cdf 0 2024-04-22T23:45:51.873 INFO:tasks.workunit.client.0.smithi031.stdout:0/815: readlink d0/d13/d18/d1e/l97 0 2024-04-22T23:45:51.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/828: mknod d6/dc/d48/d4b/d6b/de8/c110 0 2024-04-22T23:45:51.874 INFO:tasks.workunit.client.0.smithi031.stdout:0/816: write d0/d1/d3f/fa1 [306222,15864] 0 2024-04-22T23:45:51.875 INFO:tasks.workunit.client.0.smithi031.stdout:1/814: creat d1/d35/d8d/d102/f113 x:0 0 0 2024-04-22T23:45:51.876 INFO:tasks.workunit.client.0.smithi031.stdout:6/924: mkdir d3/d77/d139 0 2024-04-22T23:45:51.879 INFO:tasks.workunit.client.0.smithi031.stdout:0/817: creat d0/de0/f109 x:0 0 0 2024-04-22T23:45:51.880 INFO:tasks.workunit.client.0.smithi031.stdout:0/818: fsync d0/d13/d18/d1e/fef 0 2024-04-22T23:45:51.881 INFO:tasks.workunit.client.0.smithi031.stdout:6/925: rmdir d3/d2a/d3c/d40/d7a 39 2024-04-22T23:45:51.881 INFO:tasks.workunit.client.0.smithi031.stdout:0/819: chown d0/d13/d18/dbf/dc3/cf5 219355083 1 2024-04-22T23:45:51.884 INFO:tasks.workunit.client.0.smithi031.stdout:2/829: link l5 d6/dc/d48/d4b/d6b/de8/da4/l111 0 2024-04-22T23:45:51.885 INFO:tasks.workunit.client.0.smithi031.stdout:2/830: chown d6/dc/d42/d24/c36 57 1 2024-04-22T23:45:51.886 INFO:tasks.workunit.client.0.smithi031.stdout:6/926: symlink d3/d77/db5/d110/l13a 0 2024-04-22T23:45:51.890 INFO:tasks.workunit.client.0.smithi031.stdout:2/831: truncate d6/dc/d48/d4b/fb0 116863 0 2024-04-22T23:45:51.892 INFO:tasks.workunit.client.0.smithi031.stdout:5/767: dread db/d16/d1d/d24/d27/fb0 [0,4194304] 0 2024-04-22T23:45:51.892 INFO:tasks.workunit.client.0.smithi031.stdout:1/815: dread d1/db/d16/d1f/fe2 [0,4194304] 0 2024-04-22T23:45:51.893 INFO:tasks.workunit.client.0.smithi031.stdout:2/832: fdatasync d6/dc/d48/d4b/f7d 0 2024-04-22T23:45:51.895 INFO:tasks.workunit.client.0.smithi031.stdout:6/927: dwrite d3/d2a/f12d [0,4194304] 0 2024-04-22T23:45:51.901 INFO:tasks.workunit.client.0.smithi031.stdout:2/833: dwrite d6/dc/d48/f108 [0,4194304] 0 2024-04-22T23:45:51.902 INFO:tasks.workunit.client.0.smithi031.stdout:2/834: fsync d6/dc/f1a 0 2024-04-22T23:45:51.902 INFO:tasks.workunit.client.0.smithi031.stdout:6/928: unlink d3/d2a/f12d 0 2024-04-22T23:45:51.903 INFO:tasks.workunit.client.0.smithi031.stdout:2/835: chown d6/dc/d9a/d6c/db2/df0/f54 2 1 2024-04-22T23:45:51.905 INFO:tasks.workunit.client.0.smithi031.stdout:6/929: write d3/d35/d75/d7d/dc4/d124/f136 [725887,4482] 0 2024-04-22T23:45:51.907 INFO:tasks.workunit.client.0.smithi031.stdout:5/768: creat db/d16/d2a/d36/d74/d82/d86/ffe x:0 0 0 2024-04-22T23:45:51.907 INFO:tasks.workunit.client.0.smithi031.stdout:3/729: sync 2024-04-22T23:45:51.908 INFO:tasks.workunit.client.0.smithi031.stdout:7/813: sync 2024-04-22T23:45:51.913 INFO:tasks.workunit.client.0.smithi031.stdout:1/816: fdatasync d1/de/d1e/d34/d41/f58 0 2024-04-22T23:45:51.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/836: unlink d6/dc/d48/d8a/d92/c105 0 2024-04-22T23:45:51.921 INFO:tasks.workunit.client.0.smithi031.stdout:3/730: read - d4/d54/d2c/d89/f91 zero size 2024-04-22T23:45:51.924 INFO:tasks.workunit.client.0.smithi031.stdout:0/820: dwrite d0/d1/d2/d36/d11/d1a/ffa [0,4194304] 0 2024-04-22T23:45:51.934 INFO:tasks.workunit.client.0.smithi031.stdout:2/837: creat d6/dc/d9a/d6c/db2/df0/d94/db4/f112 x:0 0 0 2024-04-22T23:45:51.935 INFO:tasks.workunit.client.0.smithi031.stdout:5/769: truncate db/d16/d1d/d24/d27/f57 7209239 0 2024-04-22T23:45:51.939 INFO:tasks.workunit.client.0.smithi031.stdout:3/731: symlink d4/d83/dd8/lfe 0 2024-04-22T23:45:51.942 INFO:tasks.workunit.client.0.smithi031.stdout:0/821: rename d0/d13/d18/l89 to d0/db5/l10a 0 2024-04-22T23:45:51.976 INFO:tasks.workunit.client.0.smithi031.stdout:2/838: mknod d6/dc/d9a/c113 0 2024-04-22T23:45:51.977 INFO:tasks.workunit.client.0.smithi031.stdout:5/770: rmdir db/d16/d1d/d24/d32 39 2024-04-22T23:45:51.982 INFO:tasks.workunit.client.0.smithi031.stdout:1/817: creat d1/f114 x:0 0 0 2024-04-22T23:45:51.987 INFO:tasks.workunit.client.0.smithi031.stdout:2/839: rmdir d6/dc/d9a/d6c/d86 39 2024-04-22T23:45:51.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/818: dwrite d1/de/d1e/f64 [0,4194304] 0 2024-04-22T23:45:51.991 INFO:tasks.workunit.client.0.smithi031.stdout:1/819: read d1/d3f/d54/dba/f1b [1933281,25189] 0 2024-04-22T23:45:52.000 INFO:tasks.workunit.client.0.smithi031.stdout:2/840: mkdir d6/dc/d49/d83/d114 0 2024-04-22T23:45:52.004 INFO:tasks.workunit.client.0.smithi031.stdout:6/930: write d3/d2a/d3c/db6/ff2 [2015509,109564] 0 2024-04-22T23:45:52.010 INFO:tasks.workunit.client.0.smithi031.stdout:2/841: symlink d6/dc/d17/d69/d90/l115 0 2024-04-22T23:45:52.011 INFO:tasks.workunit.client.0.smithi031.stdout:2/842: truncate d6/dc/d49/d83/d85/fa1 4305137 0 2024-04-22T23:45:52.012 INFO:tasks.workunit.client.0.smithi031.stdout:2/843: rename d6/dc to d6/dc/d9a/d6c/de4/d116 22 2024-04-22T23:45:52.013 INFO:tasks.workunit.client.0.smithi031.stdout:6/931: symlink d3/d1f/d4c/df9/l13b 0 2024-04-22T23:45:52.014 INFO:tasks.workunit.client.0.smithi031.stdout:2/844: write d6/dc/d42/d24/f39 [391810,124197] 0 2024-04-22T23:45:52.014 INFO:tasks.workunit.client.0.smithi031.stdout:5/771: dread db/d16/d2a/f2c [0,4194304] 0 2024-04-22T23:45:52.018 INFO:tasks.workunit.client.0.smithi031.stdout:6/932: mkdir d3/d1f/d4c/d8e/d13c 0 2024-04-22T23:45:52.021 INFO:tasks.workunit.client.0.smithi031.stdout:2/845: unlink d6/f60 0 2024-04-22T23:45:52.023 INFO:tasks.workunit.client.0.smithi031.stdout:5/772: creat db/d16/d2a/d36/d74/d82/d86/db8/fff x:0 0 0 2024-04-22T23:45:52.026 INFO:tasks.workunit.client.0.smithi031.stdout:6/933: truncate d3/f39 2724373 0 2024-04-22T23:45:52.030 INFO:tasks.workunit.client.0.smithi031.stdout:5/773: dwrite db/d16/d2a/d59/d90/da4/ffa [0,4194304] 0 2024-04-22T23:45:52.033 INFO:tasks.workunit.client.0.smithi031.stdout:6/934: mknod d3/c13d 0 2024-04-22T23:45:52.034 INFO:tasks.workunit.client.0.smithi031.stdout:9/808: sync 2024-04-22T23:45:52.034 INFO:tasks.workunit.client.0.smithi031.stdout:4/930: sync 2024-04-22T23:45:52.034 INFO:tasks.workunit.client.0.smithi031.stdout:8/766: sync 2024-04-22T23:45:52.036 INFO:tasks.workunit.client.0.smithi031.stdout:8/767: chown d5/da/d12/d37/d4e 15049 1 2024-04-22T23:45:52.037 INFO:tasks.workunit.client.0.smithi031.stdout:5/774: dread - db/d16/fc5 zero size 2024-04-22T23:45:52.042 INFO:tasks.workunit.client.0.smithi031.stdout:6/935: mkdir d3/d77/d8d/d13e 0 2024-04-22T23:45:52.051 INFO:tasks.workunit.client.0.smithi031.stdout:9/809: dread d2/d15/f54 [0,4194304] 0 2024-04-22T23:45:52.054 INFO:tasks.workunit.client.0.smithi031.stdout:3/732: write d4/d6/da/ff [121940,89867] 0 2024-04-22T23:45:52.054 INFO:tasks.workunit.client.0.smithi031.stdout:9/810: chown d2/dc/f25 5 1 2024-04-22T23:45:52.055 INFO:tasks.workunit.client.0.smithi031.stdout:0/822: dwrite d0/d1/d2/d36/d11/d1a/d3c/f3e [4194304,4194304] 0 2024-04-22T23:45:52.056 INFO:tasks.workunit.client.0.smithi031.stdout:3/733: readlink d4/d6/d22/l28 0 2024-04-22T23:45:52.056 INFO:tasks.workunit.client.0.smithi031.stdout:9/811: readlink d2/d15/d5a/l84 0 2024-04-22T23:45:52.057 INFO:tasks.workunit.client.0.smithi031.stdout:3/734: stat d4/d54/d6d/l8a 0 2024-04-22T23:45:52.058 INFO:tasks.workunit.client.0.smithi031.stdout:9/812: read - d2/d15/d5a/fdc zero size 2024-04-22T23:45:52.058 INFO:tasks.workunit.client.0.smithi031.stdout:2/846: getdents d6/dc/d9a/d6c/db2/df0 0 2024-04-22T23:45:52.059 INFO:tasks.workunit.client.0.smithi031.stdout:5/775: dread db/fd [0,4194304] 0 2024-04-22T23:45:52.060 INFO:tasks.workunit.client.0.smithi031.stdout:3/735: fdatasync d4/d83/dd8/de6/ff5 0 2024-04-22T23:45:52.060 INFO:tasks.workunit.client.0.smithi031.stdout:8/768: dread d5/f8 [0,4194304] 0 2024-04-22T23:45:52.061 INFO:tasks.workunit.client.0.smithi031.stdout:9/813: fsync d2/dd/d75/d64/d5d/d59/f78 0 2024-04-22T23:45:52.062 INFO:tasks.workunit.client.0.smithi031.stdout:4/931: creat db/d14/d1d/d7b/f141 x:0 0 0 2024-04-22T23:45:52.064 INFO:tasks.workunit.client.0.smithi031.stdout:9/814: write d2/dd/d75/d64/fa7 [416199,122270] 0 2024-04-22T23:45:52.073 INFO:tasks.workunit.client.0.smithi031.stdout:6/936: dread d3/d2a/d3c/f73 [0,4194304] 0 2024-04-22T23:45:52.074 INFO:tasks.workunit.client.0.smithi031.stdout:4/932: dwrite db/d14/d2d/db7/f124 [0,4194304] 0 2024-04-22T23:45:52.074 INFO:tasks.workunit.client.0.smithi031.stdout:1/820: dwrite d1/d35/d8d/d102/f75 [0,4194304] 0 2024-04-22T23:45:52.075 INFO:tasks.workunit.client.0.smithi031.stdout:1/821: chown d1/cb9 0 1 2024-04-22T23:45:52.077 INFO:tasks.workunit.client.0.smithi031.stdout:8/769: mknod d5/da/d12/d1d/d38/db9/cee 0 2024-04-22T23:45:52.079 INFO:tasks.workunit.client.0.smithi031.stdout:5/776: truncate db/d16/d2a/d36/f38 2899055 0 2024-04-22T23:45:52.082 INFO:tasks.workunit.client.0.smithi031.stdout:9/815: read d2/dc/faa [209593,98285] 0 2024-04-22T23:45:52.084 INFO:tasks.workunit.client.0.smithi031.stdout:7/814: sync 2024-04-22T23:45:52.094 INFO:tasks.workunit.client.0.smithi031.stdout:3/736: symlink d4/d6/d11/dda/lff 0 2024-04-22T23:45:52.095 INFO:tasks.workunit.client.0.smithi031.stdout:2/847: getdents d6/dc/d48/d4b/d10d 0 2024-04-22T23:45:52.097 INFO:tasks.workunit.client.0.smithi031.stdout:6/937: mkdir d3/d1f/d4c/d8e/d13f 0 2024-04-22T23:45:52.097 INFO:tasks.workunit.client.0.smithi031.stdout:2/848: write d6/dc/d48/d8a/d92/ff8 [986537,62089] 0 2024-04-22T23:45:52.098 INFO:tasks.workunit.client.0.smithi031.stdout:2/849: read - d6/dc/d9a/d6c/db2/df0/d94/dab/ffb zero size 2024-04-22T23:45:52.100 INFO:tasks.workunit.client.0.smithi031.stdout:4/933: rmdir db/d14/d1d/d7b/d7f 39 2024-04-22T23:45:52.101 INFO:tasks.workunit.client.0.smithi031.stdout:4/934: chown db/d14/d2d/d49/d59/d83 3026498 1 2024-04-22T23:45:52.102 INFO:tasks.workunit.client.0.smithi031.stdout:2/850: read d6/dc/d48/d4b/d6b/f6f [1525828,72817] 0 2024-04-22T23:45:52.107 INFO:tasks.workunit.client.0.smithi031.stdout:7/815: mknod d9/d14/d1f/d45/da6/daa/dbb/c11b 0 2024-04-22T23:45:52.112 INFO:tasks.workunit.client.0.smithi031.stdout:6/938: write d3/f15 [85556,70951] 0 2024-04-22T23:45:52.113 INFO:tasks.workunit.client.0.smithi031.stdout:6/939: fsync d3/d1f/d4c/f103 0 2024-04-22T23:45:52.114 INFO:tasks.workunit.client.0.smithi031.stdout:6/940: write d3/d1f/d4c/de2/ffa [251309,120992] 0 2024-04-22T23:45:52.116 INFO:tasks.workunit.client.0.smithi031.stdout:4/935: mknod db/d14/d2d/d49/d59/d6a/d63/d86/c142 0 2024-04-22T23:45:52.118 INFO:tasks.workunit.client.0.smithi031.stdout:1/822: fdatasync d1/db/d16/fd1 0 2024-04-22T23:45:52.122 INFO:tasks.workunit.client.0.smithi031.stdout:1/823: truncate d1/d35/d36/d108/d4e/f71 4651649 0 2024-04-22T23:45:52.122 INFO:tasks.workunit.client.0.smithi031.stdout:1/824: chown d1/db/d16/d1f/f26 0 1 2024-04-22T23:45:52.124 INFO:tasks.workunit.client.0.smithi031.stdout:0/823: dwrite d0/d13/d18/d1e/f30 [0,4194304] 0 2024-04-22T23:45:52.124 INFO:tasks.workunit.client.0.smithi031.stdout:9/816: symlink d2/dc/d3e/da1/l10e 0 2024-04-22T23:45:52.127 INFO:tasks.workunit.client.0.smithi031.stdout:7/816: dread d9/d15/dcf/fd2 [0,4194304] 0 2024-04-22T23:45:52.127 INFO:tasks.workunit.client.0.smithi031.stdout:9/817: read d2/dc/f4c [554014,109312] 0 2024-04-22T23:45:52.127 INFO:tasks.workunit.client.0.smithi031.stdout:3/737: unlink d4/d6/d4c/f79 0 2024-04-22T23:45:52.129 INFO:tasks.workunit.client.0.smithi031.stdout:7/817: readlink d9/d15/d6d/d65/l67 0 2024-04-22T23:45:52.133 INFO:tasks.workunit.client.0.smithi031.stdout:1/825: dwrite d1/db3/ff6 [0,4194304] 0 2024-04-22T23:45:52.148 INFO:tasks.workunit.client.0.smithi031.stdout:5/777: write db/d16/d2a/d36/d74/fbc [411504,41124] 0 2024-04-22T23:45:52.148 INFO:tasks.workunit.client.0.smithi031.stdout:0/824: creat d0/d13/d18/d1e/d25/d8e/da7/d2b/f10b x:0 0 0 2024-04-22T23:45:52.153 INFO:tasks.workunit.client.0.smithi031.stdout:7/818: symlink d9/d15/dcf/l11c 0 2024-04-22T23:45:52.154 INFO:tasks.workunit.client.0.smithi031.stdout:7/819: fsync d9/d15/d3a/d41/f5a 0 2024-04-22T23:45:52.155 INFO:tasks.workunit.client.0.smithi031.stdout:7/820: write d9/d14/d22/d5c/d5f/fdf [908371,37488] 0 2024-04-22T23:45:52.156 INFO:tasks.workunit.client.0.smithi031.stdout:1/826: truncate d1/db/d16/fe9 934449 0 2024-04-22T23:45:52.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/941: mknod d3/d2a/d3c/d40/c140 0 2024-04-22T23:45:52.161 INFO:tasks.workunit.client.0.smithi031.stdout:8/770: getdents d5/da/d12/d2e/db0 0 2024-04-22T23:45:52.165 INFO:tasks.workunit.client.0.smithi031.stdout:5/778: read db/d16/d1d/f93 [1058232,77446] 0 2024-04-22T23:45:52.166 INFO:tasks.workunit.client.0.smithi031.stdout:3/738: dread d4/f6a [0,4194304] 0 2024-04-22T23:45:52.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/818: mknod d2/c10f 0 2024-04-22T23:45:52.170 INFO:tasks.workunit.client.0.smithi031.stdout:7/821: mkdir d9/d14/d1f/d45/de3/d11d 0 2024-04-22T23:45:52.171 INFO:tasks.workunit.client.0.smithi031.stdout:7/822: chown d9/d14/d22/dbe 0 1 2024-04-22T23:45:52.172 INFO:tasks.workunit.client.0.smithi031.stdout:6/942: dread d3/d35/fc0 [0,4194304] 0 2024-04-22T23:45:52.174 INFO:tasks.workunit.client.0.smithi031.stdout:9/819: dread d2/dd/d75/d64/fa8 [0,4194304] 0 2024-04-22T23:45:52.174 INFO:tasks.workunit.client.0.smithi031.stdout:5/779: dwrite db/d16/d2a/d36/d74/d82/d86/f91 [0,4194304] 0 2024-04-22T23:45:52.175 INFO:tasks.workunit.client.0.smithi031.stdout:0/825: mkdir d0/d1/d2/d17/d102/d10c 0 2024-04-22T23:45:52.176 INFO:tasks.workunit.client.0.smithi031.stdout:6/943: write d3/d35/d75/d7d/dc4/f123 [207323,56465] 0 2024-04-22T23:45:52.178 INFO:tasks.workunit.client.0.smithi031.stdout:2/851: write d6/dc/d17/f3c [751308,2225] 0 2024-04-22T23:45:52.179 INFO:tasks.workunit.client.0.smithi031.stdout:3/739: write d4/d6/d11/d25/d44/d7f/d5f/fb1 [3842070,76077] 0 2024-04-22T23:45:52.181 INFO:tasks.workunit.client.0.smithi031.stdout:1/827: creat d1/db/d16/da0/f115 x:0 0 0 2024-04-22T23:45:52.183 INFO:tasks.workunit.client.0.smithi031.stdout:4/936: getdents db/d14/d2d 0 2024-04-22T23:45:52.184 INFO:tasks.workunit.client.0.smithi031.stdout:7/823: unlink d9/d14/dd6/f10d 0 2024-04-22T23:45:52.186 INFO:tasks.workunit.client.0.smithi031.stdout:8/771: dwrite d5/d48/d61/d6b/d6c/daa/fb8 [0,4194304] 0 2024-04-22T23:45:52.189 INFO:tasks.workunit.client.0.smithi031.stdout:0/826: symlink d0/d13/d18/d1e/d25/dea/d105/l10d 0 2024-04-22T23:45:52.191 INFO:tasks.workunit.client.0.smithi031.stdout:1/828: symlink d1/d3f/d90/l116 0 2024-04-22T23:45:52.192 INFO:tasks.workunit.client.0.smithi031.stdout:3/740: symlink d4/d6/d11/ddf/dc3/l100 0 2024-04-22T23:45:52.193 INFO:tasks.workunit.client.0.smithi031.stdout:4/937: symlink db/d137/d47/d80/l143 0 2024-04-22T23:45:52.194 INFO:tasks.workunit.client.0.smithi031.stdout:3/741: read d4/d6/d11/d25/d44/f8e [3305756,41168] 0 2024-04-22T23:45:52.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:52 smithi031 ceph-mon[26721]: pgmap v32: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 61 MiB/s rd, 224 MiB/s wr, 557 op/s 2024-04-22T23:45:52.194 INFO:tasks.workunit.client.0.smithi031.stdout:4/938: stat db/d14/d2d/d49/d59/d6a 0 2024-04-22T23:45:52.195 INFO:tasks.workunit.client.0.smithi031.stdout:3/742: dread - d4/d54/d2c/d89/d2e/d45/fc5 zero size 2024-04-22T23:45:52.195 INFO:tasks.workunit.client.0.smithi031.stdout:7/824: mknod d9/d14/d1f/d7f/c11e 0 2024-04-22T23:45:52.198 INFO:tasks.workunit.client.0.smithi031.stdout:2/852: symlink d6/dc/d42/l117 0 2024-04-22T23:45:52.200 INFO:tasks.workunit.client.0.smithi031.stdout:9/820: creat d2/dc/d8b/d3d/d40/da4/db1/dc5/d10a/f110 x:0 0 0 2024-04-22T23:45:52.201 INFO:tasks.workunit.client.0.smithi031.stdout:2/853: stat d6/dc/d17/d69/f97 0 2024-04-22T23:45:52.202 INFO:tasks.workunit.client.0.smithi031.stdout:0/827: unlink d0/d1/d2/d36/d11/d1a/d3c/f59 0 2024-04-22T23:45:52.203 INFO:tasks.workunit.client.0.smithi031.stdout:1/829: symlink d1/d3f/l117 0 2024-04-22T23:45:52.205 INFO:tasks.workunit.client.0.smithi031.stdout:5/780: creat db/d16/d1d/d24/d32/ded/f100 x:0 0 0 2024-04-22T23:45:52.205 INFO:tasks.workunit.client.0.smithi031.stdout:4/939: stat db/d14/d1d/d7b/ce7 0 2024-04-22T23:45:52.205 INFO:tasks.workunit.client.0.smithi031.stdout:7/825: dwrite d9/d15/d3a/d41/fa3 [0,4194304] 0 2024-04-22T23:45:52.206 INFO:tasks.workunit.client.0.smithi031.stdout:2/854: readlink d6/dc/d49/d83/d85/da9/d10a/l96 0 2024-04-22T23:45:52.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/772: mknod d5/da/d12/d37/d4e/de7/cef 0 2024-04-22T23:45:52.208 INFO:tasks.workunit.client.0.smithi031.stdout:0/828: dread - d0/d13/d18/d1e/d25/d8e/dac/dda/fcc zero size 2024-04-22T23:45:52.209 INFO:tasks.workunit.client.0.smithi031.stdout:6/944: creat d3/d35/d86/f141 x:0 0 0 2024-04-22T23:45:52.212 INFO:tasks.workunit.client.0.smithi031.stdout:1/830: mkdir d1/d99/d7d/d118 0 2024-04-22T23:45:52.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/855: chown d6/dc/d48/d4b/d6b/cf6 47253 1 2024-04-22T23:45:52.215 INFO:tasks.workunit.client.0.smithi031.stdout:7/826: mknod d9/d14/d22/d5c/de0/c11f 0 2024-04-22T23:45:52.216 INFO:tasks.workunit.client.1.smithi044.stderr:++ readlink -f fsstress 2024-04-22T23:45:52.216 INFO:tasks.workunit.client.0.smithi031.stdout:9/821: dread d2/dc/d8b/d3d/d40/da4/ff4 [0,4194304] 0 2024-04-22T23:45:52.216 INFO:tasks.workunit.client.0.smithi031.stdout:4/940: creat db/d14/d2d/d49/d59/d6a/f144 x:0 0 0 2024-04-22T23:45:52.218 INFO:tasks.workunit.client.1.smithi044.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-22T23:45:52.218 INFO:tasks.workunit.client.1.smithi044.stderr:+ popd 2024-04-22T23:45:52.219 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:45:52.219 INFO:tasks.workunit.client.1.smithi044.stderr:+ popd 2024-04-22T23:45:52.220 INFO:tasks.workunit.client.1.smithi044.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:45:52.220 INFO:tasks.workunit.client.1.smithi044.stderr:++ mktemp -d -p . 2024-04-22T23:45:52.225 INFO:tasks.workunit.client.1.smithi044.stderr:+ T=./tmp.vPCY9unfEj 2024-04-22T23:45:52.225 INFO:tasks.workunit.client.1.smithi044.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.vPCY9unfEj -l 1 -n 1000 -p 10 -v 2024-04-22T23:45:52.226 INFO:tasks.workunit.client.0.smithi031.stdout:2/856: dread d6/dc/f62 [0,4194304] 0 2024-04-22T23:45:52.231 INFO:tasks.workunit.client.0.smithi031.stdout:4/941: creat db/d137/da1/da7/f145 x:0 0 0 2024-04-22T23:45:52.233 INFO:tasks.workunit.client.0.smithi031.stdout:7/827: rename d9/d14/d22/d5c/de0/c11f to d9/d14/c120 0 2024-04-22T23:45:52.234 INFO:tasks.workunit.client.0.smithi031.stdout:2/857: dwrite d6/dc/d9a/d6c/db2/df0/d94/dab/ffb [0,4194304] 0 2024-04-22T23:45:52.235 INFO:tasks.workunit.client.0.smithi031.stdout:4/942: chown db/d14/d2d/d49/d59/d6a/d63/cdb 112 1 2024-04-22T23:45:52.236 INFO:tasks.workunit.client.1.smithi044.stdout:seed = 1713720140 2024-04-22T23:45:52.236 INFO:tasks.workunit.client.0.smithi031.stdout:3/743: dwrite d4/f6a [0,4194304] 0 2024-04-22T23:45:52.238 INFO:tasks.workunit.client.0.smithi031.stdout:4/943: write db/d14/d2d/d49/d59/d6a/d63/d86/f12e [249468,28154] 0 2024-04-22T23:45:52.238 INFO:tasks.workunit.client.0.smithi031.stdout:3/744: write d4/f6a [3844890,33805] 0 2024-04-22T23:45:52.239 INFO:tasks.workunit.client.0.smithi031.stdout:8/773: write d5/d48/d61/d6b/fc1 [331682,56164] 0 2024-04-22T23:45:52.240 INFO:tasks.workunit.client.0.smithi031.stdout:1/831: write d1/de/d1e/d34/d41/f44 [1780172,128182] 0 2024-04-22T23:45:52.243 INFO:tasks.workunit.client.1.smithi044.stdout:1/0: chown . 3 1 2024-04-22T23:45:52.243 INFO:tasks.workunit.client.1.smithi044.stdout:1/1: chown . 3200 1 2024-04-22T23:45:52.244 INFO:tasks.workunit.client.0.smithi031.stdout:6/945: link d3/d35/d75/d7d/dc4/f123 d3/d1f/d4c/de2/f142 0 2024-04-22T23:45:52.249 INFO:tasks.workunit.client.0.smithi031.stdout:5/781: link db/d16/d2a/d36/d74/lc7 db/d16/d1d/d24/ddf/l101 0 2024-04-22T23:45:52.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/0: chown . 3037 1 2024-04-22T23:45:52.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/1: link - no file 2024-04-22T23:45:52.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/2: write - no filename 2024-04-22T23:45:52.250 INFO:tasks.workunit.client.1.smithi044.stdout:2/3: write - no filename 2024-04-22T23:45:52.252 INFO:tasks.workunit.client.0.smithi031.stdout:2/858: rename d6/dc/d9a/d6c/lbf to d6/dce/l118 0 2024-04-22T23:45:52.253 INFO:tasks.workunit.client.0.smithi031.stdout:3/745: dread d4/d6/d11/d25/f36 [0,4194304] 0 2024-04-22T23:45:52.258 INFO:tasks.workunit.client.1.smithi044.stdout:3/0: rename - no filename 2024-04-22T23:45:52.258 INFO:tasks.workunit.client.1.smithi044.stdout:3/1: truncate - no filename 2024-04-22T23:45:52.258 INFO:tasks.workunit.client.1.smithi044.stdout:3/2: write - no filename 2024-04-22T23:45:52.258 INFO:tasks.workunit.client.1.smithi044.stdout:3/3: dread - no filename 2024-04-22T23:45:52.259 INFO:tasks.workunit.client.0.smithi031.stdout:3/746: dread d4/d6/d22/ff1 [0,4194304] 0 2024-04-22T23:45:52.259 INFO:tasks.workunit.client.0.smithi031.stdout:0/829: rmdir d0/d1/d2/d17/d102/d10c 0 2024-04-22T23:45:52.262 INFO:tasks.workunit.client.1.smithi044.stdout:4/0: unlink - no file 2024-04-22T23:45:52.262 INFO:tasks.workunit.client.1.smithi044.stdout:4/1: chown . 3 1 2024-04-22T23:45:52.262 INFO:tasks.workunit.client.1.smithi044.stdout:4/2: dwrite - no filename 2024-04-22T23:45:52.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/4: creat f0 x:0 0 0 2024-04-22T23:45:52.265 INFO:tasks.workunit.client.1.smithi044.stdout:2/5: dread - f0 zero size 2024-04-22T23:45:52.268 INFO:tasks.workunit.client.1.smithi044.stdout:3/4: mknod c0 0 2024-04-22T23:45:52.270 INFO:tasks.workunit.client.1.smithi044.stdout:2/6: rename f0 to f1 0 2024-04-22T23:45:52.270 INFO:tasks.workunit.client.1.smithi044.stdout:2/7: readlink - no filename 2024-04-22T23:45:52.270 INFO:tasks.workunit.client.1.smithi044.stdout:2/8: stat f1 0 2024-04-22T23:45:52.270 INFO:tasks.workunit.client.1.smithi044.stdout:2/9: dread - f1 zero size 2024-04-22T23:45:52.272 INFO:tasks.workunit.client.1.smithi044.stdout:0/0: fdatasync - no filename 2024-04-22T23:45:52.272 INFO:tasks.workunit.client.1.smithi044.stdout:0/1: write - no filename 2024-04-22T23:45:52.272 INFO:tasks.workunit.client.1.smithi044.stdout:0/2: dread - no filename 2024-04-22T23:45:52.272 INFO:tasks.workunit.client.1.smithi044.stdout:4/3: creat f0 x:0 0 0 2024-04-22T23:45:52.273 INFO:tasks.workunit.client.1.smithi044.stdout:3/5: mknod c1 0 2024-04-22T23:45:52.273 INFO:tasks.workunit.client.1.smithi044.stdout:3/6: rmdir - no directory 2024-04-22T23:45:52.274 INFO:tasks.workunit.client.1.smithi044.stdout:6/0: creat f0 x:0 0 0 2024-04-22T23:45:52.274 INFO:tasks.workunit.client.0.smithi031.stdout:9/822: write d2/dc/f17 [732054,85663] 0 2024-04-22T23:45:52.275 INFO:tasks.workunit.client.0.smithi031.stdout:7/828: creat d9/d15/d6d/d65/dea/f121 x:0 0 0 2024-04-22T23:45:52.275 INFO:tasks.workunit.client.0.smithi031.stdout:5/782: rmdir db/d16/d1d/d24/d58/db1 39 2024-04-22T23:45:52.276 INFO:tasks.workunit.client.1.smithi044.stdout:6/1: dread - f0 zero size 2024-04-22T23:45:52.277 INFO:tasks.workunit.client.1.smithi044.stdout:5/0: stat - no entries 2024-04-22T23:45:52.277 INFO:tasks.workunit.client.0.smithi031.stdout:7/829: write d9/d15/d6d/d65/faf [754228,3858] 0 2024-04-22T23:45:52.278 INFO:tasks.workunit.client.1.smithi044.stdout:6/2: chown f0 426207 1 2024-04-22T23:45:52.279 INFO:tasks.workunit.client.1.smithi044.stdout:4/4: rename f0 to f1 0 2024-04-22T23:45:52.279 INFO:tasks.workunit.client.0.smithi031.stdout:1/832: mknod d1/d35/c119 0 2024-04-22T23:45:52.280 INFO:tasks.workunit.client.1.smithi044.stdout:4/5: read - f1 zero size 2024-04-22T23:45:52.281 INFO:tasks.workunit.client.1.smithi044.stdout:3/7: rename c1 to c2 0 2024-04-22T23:45:52.282 INFO:tasks.workunit.client.0.smithi031.stdout:2/859: symlink d6/dc/d48/d4b/d10d/l119 0 2024-04-22T23:45:52.283 INFO:tasks.workunit.client.1.smithi044.stdout:2/10: dwrite f1 [0,4194304] 0 2024-04-22T23:45:52.283 INFO:tasks.workunit.client.1.smithi044.stdout:2/11: rmdir - no directory 2024-04-22T23:45:52.283 INFO:tasks.workunit.client.0.smithi031.stdout:2/860: write d6/dc/d9a/d6c/f76 [319473,37357] 0 2024-04-22T23:45:52.284 INFO:tasks.workunit.client.0.smithi031.stdout:2/861: dread - d6/dc/d9a/d6c/db2/df0/fd8 zero size 2024-04-22T23:45:52.284 INFO:tasks.workunit.client.1.smithi044.stdout:7/0: dwrite - no filename 2024-04-22T23:45:52.284 INFO:tasks.workunit.client.0.smithi031.stdout:2/862: fsync d6/dc/d42/d24/fe2 0 2024-04-22T23:45:52.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/747: fsync d4/d49/f6f 0 2024-04-22T23:45:52.286 INFO:tasks.workunit.client.1.smithi044.stdout:0/3: mkdir d0 0 2024-04-22T23:45:52.287 INFO:tasks.workunit.client.0.smithi031.stdout:2/863: chown d6/dc/d42/d24/l66 16326 1 2024-04-22T23:45:52.287 INFO:tasks.workunit.client.0.smithi031.stdout:0/830: mknod d0/d1/d2/d36/d11/d1a/dfc/dad/dba/c10e 0 2024-04-22T23:45:52.288 INFO:tasks.workunit.client.0.smithi031.stdout:2/864: fsync d6/dc/d48/d4b/d6b/de8/ff9 0 2024-04-22T23:45:52.289 INFO:tasks.workunit.client.0.smithi031.stdout:0/831: write d0/f3d [3562827,107207] 0 2024-04-22T23:45:52.289 INFO:tasks.workunit.client.0.smithi031.stdout:0/832: stat d0/de0/f109 0 2024-04-22T23:45:52.295 INFO:tasks.workunit.client.0.smithi031.stdout:5/783: write db/d16/d2a/d7c/fa7 [3743764,42240] 0 2024-04-22T23:45:52.298 INFO:tasks.workunit.client.1.smithi044.stdout:4/6: mknod c2 0 2024-04-22T23:45:52.301 INFO:tasks.workunit.client.1.smithi044.stdout:3/8: rename c0 to c3 0 2024-04-22T23:45:52.302 INFO:tasks.workunit.client.1.smithi044.stdout:3/9: read - no filename 2024-04-22T23:45:52.303 INFO:tasks.workunit.client.0.smithi031.stdout:7/830: write d9/ff [3768420,28499] 0 2024-04-22T23:45:52.305 INFO:tasks.workunit.client.0.smithi031.stdout:1/833: creat d1/d3f/d54/f11a x:0 0 0 2024-04-22T23:45:52.308 INFO:tasks.workunit.client.1.smithi044.stdout:2/12: fsync f1 0 2024-04-22T23:45:52.308 INFO:tasks.workunit.client.1.smithi044.stdout:8/0: chown . 376238 1 2024-04-22T23:45:52.308 INFO:tasks.workunit.client.1.smithi044.stdout:8/1: chown . 1052 1 2024-04-22T23:45:52.309 INFO:tasks.workunit.client.0.smithi031.stdout:3/748: dread - d4/d54/f78 zero size 2024-04-22T23:45:52.311 INFO:tasks.workunit.client.0.smithi031.stdout:1/834: dwrite d1/d35/d36/d108/d4e/dd9/fff [0,4194304] 0 2024-04-22T23:45:52.313 INFO:tasks.workunit.client.1.smithi044.stdout:5/1: creat f0 x:0 0 0 2024-04-22T23:45:52.313 INFO:tasks.workunit.client.1.smithi044.stdout:5/2: readlink - no filename 2024-04-22T23:45:52.314 INFO:tasks.workunit.client.0.smithi031.stdout:1/835: read d1/d35/d36/d108/d4e/f71 [726271,16061] 0 2024-04-22T23:45:52.315 INFO:tasks.workunit.client.0.smithi031.stdout:2/865: mkdir d6/dc/d9a/d6c/db2/df0/d94/dab/d11a 0 2024-04-22T23:45:52.316 INFO:tasks.workunit.client.1.smithi044.stdout:6/3: link f0 f1 0 2024-04-22T23:45:52.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/833: read d0/d13/d18/d1e/d25/f85 [3502910,8411] 0 2024-04-22T23:45:52.319 INFO:tasks.workunit.client.0.smithi031.stdout:9/823: write d2/dd/d75/d64/fa8 [8501,124034] 0 2024-04-22T23:45:52.320 INFO:tasks.workunit.client.1.smithi044.stdout:4/7: write f1 [506546,6700] 0 2024-04-22T23:45:52.321 INFO:tasks.workunit.client.0.smithi031.stdout:5/784: mkdir db/d16/d1d/d24/d27/d9e/d102 0 2024-04-22T23:45:52.322 INFO:tasks.workunit.client.0.smithi031.stdout:5/785: write db/d16/d2a/d36/fdb [218552,43365] 0 2024-04-22T23:45:52.322 INFO:tasks.workunit.client.1.smithi044.stdout:3/10: chown c2 173262 1 2024-04-22T23:45:52.322 INFO:tasks.workunit.client.1.smithi044.stdout:3/11: write - no filename 2024-04-22T23:45:52.325 INFO:tasks.workunit.client.1.smithi044.stdout:9/0: write - no filename 2024-04-22T23:45:52.326 INFO:tasks.workunit.client.0.smithi031.stdout:8/774: getdents d5/da/d12/d1d 0 2024-04-22T23:45:52.326 INFO:tasks.workunit.client.0.smithi031.stdout:8/775: write d5/d48/fba [344441,31384] 0 2024-04-22T23:45:52.327 INFO:tasks.workunit.client.1.smithi044.stdout:7/1: symlink l0 0 2024-04-22T23:45:52.327 INFO:tasks.workunit.client.1.smithi044.stdout:7/2: dread - no filename 2024-04-22T23:45:52.328 INFO:tasks.workunit.client.1.smithi044.stdout:0/4: symlink d0/l1 0 2024-04-22T23:45:52.328 INFO:tasks.workunit.client.1.smithi044.stdout:0/5: write - no filename 2024-04-22T23:45:52.333 INFO:tasks.workunit.client.0.smithi031.stdout:6/946: rename d3/d2a/d85/d96/c109 to d3/d1f/c143 0 2024-04-22T23:45:52.336 INFO:tasks.workunit.client.1.smithi044.stdout:4/8: mknod c3 0 2024-04-22T23:45:52.338 INFO:tasks.workunit.client.1.smithi044.stdout:3/12: symlink l4 0 2024-04-22T23:45:52.342 INFO:tasks.workunit.client.0.smithi031.stdout:3/749: dread d4/d49/f94 [0,4194304] 0 2024-04-22T23:45:52.343 INFO:tasks.workunit.client.1.smithi044.stdout:2/13: link f1 f2 0 2024-04-22T23:45:52.346 INFO:tasks.workunit.client.1.smithi044.stdout:7/3: creat f1 x:0 0 0 2024-04-22T23:45:52.347 INFO:tasks.workunit.client.1.smithi044.stdout:6/4: truncate f1 106680 0 2024-04-22T23:45:52.347 INFO:tasks.workunit.client.1.smithi044.stdout:6/5: readlink - no filename 2024-04-22T23:45:52.348 INFO:tasks.workunit.client.1.smithi044.stdout:0/6: mkdir d0/d2 0 2024-04-22T23:45:52.348 INFO:tasks.workunit.client.1.smithi044.stdout:0/7: truncate - no filename 2024-04-22T23:45:52.348 INFO:tasks.workunit.client.1.smithi044.stdout:0/8: dwrite - no filename 2024-04-22T23:45:52.350 INFO:tasks.workunit.client.0.smithi031.stdout:0/834: mknod d0/d13/d18/d1e/d25/d8e/dac/dda/c10f 0 2024-04-22T23:45:52.351 INFO:tasks.workunit.client.1.smithi044.stdout:4/9: symlink l4 0 2024-04-22T23:45:52.351 INFO:tasks.workunit.client.1.smithi044.stdout:4/10: chown c2 0 1 2024-04-22T23:45:52.353 INFO:tasks.workunit.client.1.smithi044.stdout:3/13: creat f5 x:0 0 0 2024-04-22T23:45:52.353 INFO:tasks.workunit.client.1.smithi044.stdout:3/14: read - f5 zero size 2024-04-22T23:45:52.354 INFO:tasks.workunit.client.0.smithi031.stdout:7/831: link d9/d15/d6d/dcc/fd0 d9/d15/dcf/f122 0 2024-04-22T23:45:52.355 INFO:tasks.workunit.client.1.smithi044.stdout:9/1: mknod c0 0 2024-04-22T23:45:52.356 INFO:tasks.workunit.client.0.smithi031.stdout:8/776: fsync d5/da/d12/d2e/d4d/f52 0 2024-04-22T23:45:52.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/14: write f2 [4127222,56954] 0 2024-04-22T23:45:52.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/6: mknod c2 0 2024-04-22T23:45:52.358 INFO:tasks.workunit.client.1.smithi044.stdout:2/15: write f2 [4224051,128342] 0 2024-04-22T23:45:52.361 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:52 smithi044 ceph-mon[34463]: pgmap v32: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 61 MiB/s rd, 224 MiB/s wr, 557 op/s 2024-04-22T23:45:52.362 INFO:tasks.workunit.client.0.smithi031.stdout:7/832: dread d9/d15/d3a/f51 [0,4194304] 0 2024-04-22T23:45:52.362 INFO:tasks.workunit.client.0.smithi031.stdout:1/836: unlink d1/de/d1e/d34/d41/c32 0 2024-04-22T23:45:52.363 INFO:tasks.workunit.client.0.smithi031.stdout:1/837: fsync d1/d35/d8d/d102/fae 0 2024-04-22T23:45:52.363 INFO:tasks.workunit.client.1.smithi044.stdout:4/11: mkdir d5 0 2024-04-22T23:45:52.366 INFO:tasks.workunit.client.1.smithi044.stdout:3/15: mknod c6 0 2024-04-22T23:45:52.366 INFO:tasks.workunit.client.1.smithi044.stdout:4/12: dread f1 [0,4194304] 0 2024-04-22T23:45:52.366 INFO:tasks.workunit.client.1.smithi044.stdout:4/13: stat c2 0 2024-04-22T23:45:52.368 INFO:tasks.workunit.client.1.smithi044.stdout:9/2: creat f1 x:0 0 0 2024-04-22T23:45:52.368 INFO:tasks.workunit.client.1.smithi044.stdout:9/3: rmdir - no directory 2024-04-22T23:45:52.370 INFO:tasks.workunit.client.0.smithi031.stdout:3/750: mknod d4/d6/d22/c101 0 2024-04-22T23:45:52.370 INFO:tasks.workunit.client.0.smithi031.stdout:3/751: dread - d4/d54/fe8 zero size 2024-04-22T23:45:52.371 INFO:tasks.workunit.client.1.smithi044.stdout:6/7: creat f3 x:0 0 0 2024-04-22T23:45:52.373 INFO:tasks.workunit.client.1.smithi044.stdout:0/9: symlink d0/d2/l3 0 2024-04-22T23:45:52.373 INFO:tasks.workunit.client.1.smithi044.stdout:0/10: fsync - no filename 2024-04-22T23:45:52.374 INFO:tasks.workunit.client.1.smithi044.stdout:3/16: dwrite f5 [0,4194304] 0 2024-04-22T23:45:52.375 INFO:tasks.workunit.client.1.smithi044.stdout:3/17: chown f5 356643 1 2024-04-22T23:45:52.375 INFO:tasks.workunit.client.0.smithi031.stdout:0/835: symlink d0/d1/d2/d36/d11/d1c/l110 0 2024-04-22T23:45:52.376 INFO:tasks.workunit.client.0.smithi031.stdout:0/836: read d0/d13/d18/d1e/d25/d8e/dac/dda/fc1 [422147,28589] 0 2024-04-22T23:45:52.376 INFO:tasks.workunit.client.0.smithi031.stdout:0/837: stat d0/d1/d2/d36/d11/d1a/l4a 0 2024-04-22T23:45:52.376 INFO:tasks.workunit.client.0.smithi031.stdout:5/786: mknod db/d16/d1d/d24/c103 0 2024-04-22T23:45:52.377 INFO:tasks.workunit.client.1.smithi044.stdout:5/3: getdents . 0 2024-04-22T23:45:52.377 INFO:tasks.workunit.client.1.smithi044.stdout:5/4: fsync f0 0 2024-04-22T23:45:52.378 INFO:tasks.workunit.client.1.smithi044.stdout:5/5: fdatasync f0 0 2024-04-22T23:45:52.378 INFO:tasks.workunit.client.0.smithi031.stdout:0/838: write d0/d13/d18/d1e/d25/d8e/da7/d2b/f64 [1405790,100698] 0 2024-04-22T23:45:52.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/866: truncate d6/dc/d57/fdb 180313 0 2024-04-22T23:45:52.383 INFO:tasks.workunit.client.0.smithi031.stdout:2/867: chown d6/dc/d48/d4b/fe3 0 1 2024-04-22T23:45:52.383 INFO:tasks.workunit.client.1.smithi044.stdout:9/4: mkdir d2 0 2024-04-22T23:45:52.384 INFO:tasks.workunit.client.1.smithi044.stdout:9/5: write f1 [469649,50113] 0 2024-04-22T23:45:52.385 INFO:tasks.workunit.client.1.smithi044.stdout:9/6: write f1 [203777,72288] 0 2024-04-22T23:45:52.387 INFO:tasks.workunit.client.1.smithi044.stdout:6/8: stat f0 0 2024-04-22T23:45:52.387 INFO:tasks.workunit.client.1.smithi044.stdout:6/9: rmdir - no directory 2024-04-22T23:45:52.390 INFO:tasks.workunit.client.0.smithi031.stdout:3/752: creat d4/d6/d11/d25/d44/d7f/da5/f102 x:0 0 0 2024-04-22T23:45:52.391 INFO:tasks.workunit.client.1.smithi044.stdout:0/11: creat d0/f4 x:0 0 0 2024-04-22T23:45:52.391 INFO:tasks.workunit.client.1.smithi044.stdout:0/12: readlink d0/l1 0 2024-04-22T23:45:52.392 INFO:tasks.workunit.client.0.smithi031.stdout:9/824: creat d2/dc/f111 x:0 0 0 2024-04-22T23:45:52.392 INFO:tasks.workunit.client.1.smithi044.stdout:9/7: dread f1 [0,4194304] 0 2024-04-22T23:45:52.392 INFO:tasks.workunit.client.0.smithi031.stdout:9/825: fdatasync d2/dc/f25 0 2024-04-22T23:45:52.393 INFO:tasks.workunit.client.1.smithi044.stdout:9/8: write f1 [1298935,42647] 0 2024-04-22T23:45:52.395 INFO:tasks.workunit.client.0.smithi031.stdout:4/944: sync 2024-04-22T23:45:52.398 INFO:tasks.workunit.client.1.smithi044.stdout:5/6: creat f1 x:0 0 0 2024-04-22T23:45:52.398 INFO:tasks.workunit.client.0.smithi031.stdout:5/787: dread db/f29 [0,4194304] 0 2024-04-22T23:45:52.399 INFO:tasks.workunit.client.1.smithi044.stdout:5/7: write f0 [522260,55160] 0 2024-04-22T23:45:52.401 INFO:tasks.workunit.client.1.smithi044.stdout:0/13: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:45:52.404 INFO:tasks.workunit.client.0.smithi031.stdout:0/839: symlink d0/l111 0 2024-04-22T23:45:52.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/840: chown d0/d13/d18/d1e/l97 82426 1 2024-04-22T23:45:52.412 INFO:tasks.workunit.client.0.smithi031.stdout:6/947: creat d3/d2a/d85/d96/f144 x:0 0 0 2024-04-22T23:45:52.419 INFO:tasks.workunit.client.1.smithi044.stdout:6/10: write f0 [761563,67230] 0 2024-04-22T23:45:52.419 INFO:tasks.workunit.client.1.smithi044.stdout:6/11: dread - f3 zero size 2024-04-22T23:45:52.420 INFO:tasks.workunit.client.1.smithi044.stdout:0/14: creat d0/d2/f5 x:0 0 0 2024-04-22T23:45:52.425 INFO:tasks.workunit.client.0.smithi031.stdout:0/841: creat d0/db5/df6/f112 x:0 0 0 2024-04-22T23:45:52.425 INFO:tasks.workunit.client.1.smithi044.stdout:3/18: truncate f5 30436 0 2024-04-22T23:45:52.426 INFO:tasks.workunit.client.1.smithi044.stdout:2/16: getdents . 0 2024-04-22T23:45:52.426 INFO:tasks.workunit.client.1.smithi044.stdout:2/17: chown f1 0 1 2024-04-22T23:45:52.427 INFO:tasks.workunit.client.0.smithi031.stdout:2/868: write d6/dc/d9a/d6c/db2/df0/f54 [970331,77942] 0 2024-04-22T23:45:52.428 INFO:tasks.workunit.client.0.smithi031.stdout:8/777: rename d5/da/df/d33/l60 to d5/da/d12/d37/lf0 0 2024-04-22T23:45:52.429 INFO:tasks.workunit.client.0.smithi031.stdout:1/838: write d1/de/d1e/d34/d66/f92 [994328,112959] 0 2024-04-22T23:45:52.431 INFO:tasks.workunit.client.1.smithi044.stdout:9/9: creat d2/f3 x:0 0 0 2024-04-22T23:45:52.433 INFO:tasks.workunit.client.1.smithi044.stdout:5/8: link f1 f2 0 2024-04-22T23:45:52.436 INFO:tasks.workunit.client.1.smithi044.stdout:3/19: creat f7 x:0 0 0 2024-04-22T23:45:52.436 INFO:tasks.workunit.client.0.smithi031.stdout:9/826: mknod d2/d15/daf/de5/c112 0 2024-04-22T23:45:52.437 INFO:tasks.workunit.client.1.smithi044.stdout:2/18: creat f3 x:0 0 0 2024-04-22T23:45:52.438 INFO:tasks.workunit.client.0.smithi031.stdout:4/945: symlink db/d14/d1d/d7b/d7f/dbf/l146 0 2024-04-22T23:45:52.442 INFO:tasks.workunit.client.1.smithi044.stdout:2/19: dread f2 [0,4194304] 0 2024-04-22T23:45:52.443 INFO:tasks.workunit.client.0.smithi031.stdout:9/827: dwrite d2/dc/d8b/d3d/d40/f44 [0,4194304] 0 2024-04-22T23:45:52.444 INFO:tasks.workunit.client.1.smithi044.stdout:5/9: symlink l3 0 2024-04-22T23:45:52.445 INFO:tasks.workunit.client.0.smithi031.stdout:9/828: write d2/d15/f7c [1727052,48242] 0 2024-04-22T23:45:52.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/842: unlink d0/d1/d2/d36/d11/d1a/dfc/f74 0 2024-04-22T23:45:52.446 INFO:tasks.workunit.client.0.smithi031.stdout:0/843: dread - d0/db5/df6/f112 zero size 2024-04-22T23:45:52.449 INFO:tasks.workunit.client.1.smithi044.stdout:3/20: creat f8 x:0 0 0 2024-04-22T23:45:52.449 INFO:tasks.workunit.client.1.smithi044.stdout:3/21: fdatasync f7 0 2024-04-22T23:45:52.452 INFO:tasks.workunit.client.0.smithi031.stdout:7/833: rename d9/d14/d22/d28/f82 to d9/d15/d6d/da9/f123 0 2024-04-22T23:45:52.454 INFO:tasks.workunit.client.0.smithi031.stdout:1/839: chown d1/db/d16/cf2 2204 1 2024-04-22T23:45:52.455 INFO:tasks.workunit.client.0.smithi031.stdout:8/778: mkdir d5/da/d12/d37/d34/d6f/d85/df1 0 2024-04-22T23:45:52.457 INFO:tasks.workunit.client.1.smithi044.stdout:5/10: fdatasync f1 0 2024-04-22T23:45:52.460 INFO:tasks.workunit.client.0.smithi031.stdout:5/788: rmdir db/d8d/dd9 0 2024-04-22T23:45:52.462 INFO:tasks.workunit.client.1.smithi044.stdout:3/22: mkdir d9 0 2024-04-22T23:45:52.462 INFO:tasks.workunit.client.1.smithi044.stdout:3/23: readlink l4 0 2024-04-22T23:45:52.463 INFO:tasks.workunit.client.1.smithi044.stdout:3/24: write f7 [951909,26737] 0 2024-04-22T23:45:52.465 INFO:tasks.workunit.client.0.smithi031.stdout:2/869: symlink d6/dc/d17/l11b 0 2024-04-22T23:45:52.466 INFO:tasks.workunit.client.0.smithi031.stdout:2/870: chown d6/dc/d48/d4b/d6b/de8/c110 6173454 1 2024-04-22T23:45:52.466 INFO:tasks.workunit.client.1.smithi044.stdout:6/12: getdents . 0 2024-04-22T23:45:52.470 INFO:tasks.workunit.client.1.smithi044.stdout:5/11: mknod c4 0 2024-04-22T23:45:52.472 INFO:tasks.workunit.client.1.smithi044.stdout:0/15: getdents d0 0 2024-04-22T23:45:52.473 INFO:tasks.workunit.client.1.smithi044.stdout:9/10: dwrite f1 [0,4194304] 0 2024-04-22T23:45:52.479 INFO:tasks.workunit.client.0.smithi031.stdout:4/946: link db/d14/d1d/f76 db/d14/d2d/f147 0 2024-04-22T23:45:52.480 INFO:tasks.workunit.client.1.smithi044.stdout:6/13: mknod c4 0 2024-04-22T23:45:52.481 INFO:tasks.workunit.client.1.smithi044.stdout:6/14: rmdir - no directory 2024-04-22T23:45:52.481 INFO:tasks.workunit.client.1.smithi044.stdout:6/15: write f0 [953943,88690] 0 2024-04-22T23:45:52.483 INFO:tasks.workunit.client.0.smithi031.stdout:5/789: mkdir db/d16/d1d/d24/d32/d41/d104 0 2024-04-22T23:45:52.485 INFO:tasks.workunit.client.1.smithi044.stdout:3/25: mknod d9/ca 0 2024-04-22T23:45:52.487 INFO:tasks.workunit.client.0.smithi031.stdout:2/871: creat d6/dc/d48/d4b/d6b/de8/da4/f11c x:0 0 0 2024-04-22T23:45:52.487 INFO:tasks.workunit.client.0.smithi031.stdout:4/947: dwrite db/d14/d1d/d7b/d7f/ff0 [0,4194304] 0 2024-04-22T23:45:52.487 INFO:tasks.workunit.client.1.smithi044.stdout:9/11: creat d2/f4 x:0 0 0 2024-04-22T23:45:52.491 INFO:tasks.workunit.client.1.smithi044.stdout:6/16: mkdir d5 0 2024-04-22T23:45:52.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/26: mkdir d9/db 0 2024-04-22T23:45:52.494 INFO:tasks.workunit.client.0.smithi031.stdout:9/829: creat d2/dd/d75/f113 x:0 0 0 2024-04-22T23:45:52.494 INFO:tasks.workunit.client.1.smithi044.stdout:9/12: mkdir d2/d5 0 2024-04-22T23:45:52.496 INFO:tasks.workunit.client.1.smithi044.stdout:9/13: truncate d2/f3 467618 0 2024-04-22T23:45:52.497 INFO:tasks.workunit.client.1.smithi044.stdout:3/27: dread f7 [0,4194304] 0 2024-04-22T23:45:52.498 INFO:tasks.workunit.client.0.smithi031.stdout:2/872: symlink d6/dc/d49/l11d 0 2024-04-22T23:45:52.499 INFO:tasks.workunit.client.1.smithi044.stdout:6/17: creat d5/f6 x:0 0 0 2024-04-22T23:45:52.499 INFO:tasks.workunit.client.1.smithi044.stdout:6/18: chown c2 27611293 1 2024-04-22T23:45:52.500 INFO:tasks.workunit.client.1.smithi044.stdout:9/14: mkdir d2/d6 0 2024-04-22T23:45:52.500 INFO:tasks.workunit.client.1.smithi044.stdout:6/19: dread - d5/f6 zero size 2024-04-22T23:45:52.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/840: mknod d1/d3f/d54/dc6/d110/c11b 0 2024-04-22T23:45:52.502 INFO:tasks.workunit.client.1.smithi044.stdout:3/28: read f5 [28299,6866] 0 2024-04-22T23:45:52.503 INFO:tasks.workunit.client.1.smithi044.stdout:3/29: dread - f8 zero size 2024-04-22T23:45:52.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/830: chown d2/dc/d8b/d3d/d40/da4/db1/lf5 242130 1 2024-04-22T23:45:52.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/831: chown d2/d15/d5a/fdc 55944 1 2024-04-22T23:45:52.508 INFO:tasks.workunit.client.1.smithi044.stdout:6/20: symlink d5/l7 0 2024-04-22T23:45:52.514 INFO:tasks.workunit.client.1.smithi044.stdout:3/30: rename c2 to d9/cc 0 2024-04-22T23:45:52.514 INFO:tasks.workunit.client.0.smithi031.stdout:5/790: dread db/d16/d1d/d24/f76 [0,4194304] 0 2024-04-22T23:45:52.515 INFO:tasks.workunit.client.0.smithi031.stdout:8/779: getdents d5/da/df/d33 0 2024-04-22T23:45:52.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/844: write d0/d1/d2/db4/ff4 [1086579,43342] 0 2024-04-22T23:45:52.518 INFO:tasks.workunit.client.0.smithi031.stdout:5/791: dread - db/d16/d1d/d24/d58/d5d/dd6/ff3 zero size 2024-04-22T23:45:52.519 INFO:tasks.workunit.client.0.smithi031.stdout:2/873: dread d6/dc/d9a/d6c/db2/df0/f54 [0,4194304] 0 2024-04-22T23:45:52.519 INFO:tasks.workunit.client.0.smithi031.stdout:0/845: write d0/d13/d18/d1e/f6a [5063770,34096] 0 2024-04-22T23:45:52.520 INFO:tasks.workunit.client.0.smithi031.stdout:1/841: creat d1/de/d1e/d94/f11c x:0 0 0 2024-04-22T23:45:52.520 INFO:tasks.workunit.client.0.smithi031.stdout:5/792: fdatasync db/d16/d2a/d59/f75 0 2024-04-22T23:45:52.522 INFO:tasks.workunit.client.0.smithi031.stdout:9/832: fdatasync d2/dd/f2b 0 2024-04-22T23:45:52.522 INFO:tasks.workunit.client.1.smithi044.stdout:3/31: mknod d9/db/cd 0 2024-04-22T23:45:52.523 INFO:tasks.workunit.client.0.smithi031.stdout:9/833: chown d2/dc/f17 80 1 2024-04-22T23:45:52.524 INFO:tasks.workunit.client.0.smithi031.stdout:8/780: mkdir d5/da/d12/d2e/d4d/d9a/db1/df2 0 2024-04-22T23:45:52.528 INFO:tasks.workunit.client.0.smithi031.stdout:2/874: dwrite d6/dc/d9a/d6c/db2/df0/d94/fe0 [0,4194304] 0 2024-04-22T23:45:52.529 INFO:tasks.workunit.client.1.smithi044.stdout:6/21: rmdir d5 39 2024-04-22T23:45:52.532 INFO:tasks.workunit.client.1.smithi044.stdout:9/15: truncate f1 3906411 0 2024-04-22T23:45:52.537 INFO:tasks.workunit.client.0.smithi031.stdout:5/793: dread db/d16/d1d/d24/d58/ffc [0,4194304] 0 2024-04-22T23:45:52.537 INFO:tasks.workunit.client.1.smithi044.stdout:6/22: mkdir d5/d8 0 2024-04-22T23:45:52.538 INFO:tasks.workunit.client.1.smithi044.stdout:6/23: chown c2 217756733 1 2024-04-22T23:45:52.541 INFO:tasks.workunit.client.0.smithi031.stdout:2/875: creat d6/dc/d9a/d6c/db2/df0/d94/dab/f11e x:0 0 0 2024-04-22T23:45:52.541 INFO:tasks.workunit.client.1.smithi044.stdout:9/16: dwrite d2/f4 [0,4194304] 0 2024-04-22T23:45:52.542 INFO:tasks.workunit.client.0.smithi031.stdout:2/876: chown d6/dc/d48/d8a/d92/ff4 1836 1 2024-04-22T23:45:52.543 INFO:tasks.workunit.client.0.smithi031.stdout:2/877: write d6/fc6 [1066183,97061] 0 2024-04-22T23:45:52.543 INFO:tasks.workunit.client.0.smithi031.stdout:2/878: chown d6/dc/d49/d83/d85 179 1 2024-04-22T23:45:52.544 INFO:tasks.workunit.client.1.smithi044.stdout:6/24: unlink f3 0 2024-04-22T23:45:52.545 INFO:tasks.workunit.client.1.smithi044.stdout:6/25: rename d5/d8 to d5/d8/d9 22 2024-04-22T23:45:52.546 INFO:tasks.workunit.client.1.smithi044.stdout:6/26: readlink d5/l7 0 2024-04-22T23:45:52.546 INFO:tasks.workunit.client.1.smithi044.stdout:6/27: readlink d5/l7 0 2024-04-22T23:45:52.547 INFO:tasks.workunit.client.0.smithi031.stdout:9/834: write d2/dc/fb7 [41858,76607] 0 2024-04-22T23:45:52.548 INFO:tasks.workunit.client.0.smithi031.stdout:9/835: chown d2/dd/f2b 958346791 1 2024-04-22T23:45:52.554 INFO:tasks.workunit.client.1.smithi044.stdout:9/17: creat d2/d5/f7 x:0 0 0 2024-04-22T23:45:52.554 INFO:tasks.workunit.client.0.smithi031.stdout:8/781: truncate d5/da/d12/d37/d34/f4c 1818677 0 2024-04-22T23:45:52.555 INFO:tasks.workunit.client.1.smithi044.stdout:6/28: creat d5/d8/fa x:0 0 0 2024-04-22T23:45:52.555 INFO:tasks.workunit.client.0.smithi031.stdout:9/836: dwrite d2/dc/d3e/f99 [0,4194304] 0 2024-04-22T23:45:52.556 INFO:tasks.workunit.client.1.smithi044.stdout:6/29: truncate d5/f6 730650 0 2024-04-22T23:45:52.557 INFO:tasks.workunit.client.0.smithi031.stdout:1/842: unlink d1/d3f/d90/c6c 0 2024-04-22T23:45:52.559 INFO:tasks.workunit.client.0.smithi031.stdout:1/843: write d1/d35/d36/d108/d4e/dd9/fff [2040816,71824] 0 2024-04-22T23:45:52.561 INFO:tasks.workunit.client.0.smithi031.stdout:1/844: chown d1/d3f/d54/dba/d28/fa5 13889 1 2024-04-22T23:45:52.563 INFO:tasks.workunit.client.1.smithi044.stdout:9/18: creat d2/d6/f8 x:0 0 0 2024-04-22T23:45:52.566 INFO:tasks.workunit.client.1.smithi044.stdout:6/30: dwrite f1 [0,4194304] 0 2024-04-22T23:45:52.567 INFO:tasks.workunit.client.0.smithi031.stdout:9/837: creat d2/dd/d75/d80/f114 x:0 0 0 2024-04-22T23:45:52.569 INFO:tasks.workunit.client.0.smithi031.stdout:2/879: fsync d6/dc/d57/fdb 0 2024-04-22T23:45:52.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/845: write d1/db/ffa [4671020,28230] 0 2024-04-22T23:45:52.572 INFO:tasks.workunit.client.0.smithi031.stdout:0/846: getdents d0/d1/d2/d36/d11/d1a/dfc/dad/dba 0 2024-04-22T23:45:52.573 INFO:tasks.workunit.client.1.smithi044.stdout:9/19: dwrite d2/f4 [0,4194304] 0 2024-04-22T23:45:52.573 INFO:tasks.workunit.client.0.smithi031.stdout:8/782: mknod d5/da/d12/d37/d34/d6f/cf3 0 2024-04-22T23:45:52.575 INFO:tasks.workunit.client.0.smithi031.stdout:0/847: chown d0/d1/d2/d36/d11/d1a/dfc/dad/lcb 95988683 1 2024-04-22T23:45:52.577 INFO:tasks.workunit.client.0.smithi031.stdout:0/848: chown d0/d13/d18/d1e/d25/d8e/da7/d2b/l7b 9810213 1 2024-04-22T23:45:52.578 INFO:tasks.workunit.client.0.smithi031.stdout:2/880: symlink d6/d45/d50/l11f 0 2024-04-22T23:45:52.579 INFO:tasks.workunit.client.0.smithi031.stdout:0/849: readlink d0/d1/d2/d36/d11/d1a/d56/l71 0 2024-04-22T23:45:52.582 INFO:tasks.workunit.client.1.smithi044.stdout:9/20: mkdir d2/d9 0 2024-04-22T23:45:52.583 INFO:tasks.workunit.client.1.smithi044.stdout:9/21: write d2/f4 [811366,210] 0 2024-04-22T23:45:52.585 INFO:tasks.workunit.client.0.smithi031.stdout:3/753: sync 2024-04-22T23:45:52.586 INFO:tasks.workunit.client.0.smithi031.stdout:8/783: unlink d5/da/d12/d37/d34/d6f/d85/d9b/f4f 0 2024-04-22T23:45:52.587 INFO:tasks.workunit.client.1.smithi044.stdout:9/22: dread d2/f4 [0,4194304] 0 2024-04-22T23:45:52.590 INFO:tasks.workunit.client.1.smithi044.stdout:9/23: creat d2/d6/fa x:0 0 0 2024-04-22T23:45:52.591 INFO:tasks.workunit.client.1.smithi044.stdout:9/24: write d2/f3 [26446,80990] 0 2024-04-22T23:45:52.591 INFO:tasks.workunit.client.0.smithi031.stdout:2/881: creat d6/dc/d9a/d6c/db2/df0/f120 x:0 0 0 2024-04-22T23:45:52.592 INFO:tasks.workunit.client.0.smithi031.stdout:0/850: truncate d0/d1/d2/d36/d11/d1a/d56/fc6 1954084 0 2024-04-22T23:45:52.593 INFO:tasks.workunit.client.0.smithi031.stdout:5/794: link db/d16/l45 db/d16/d2a/l105 0 2024-04-22T23:45:52.596 INFO:tasks.workunit.client.1.smithi044.stdout:9/25: unlink f1 0 2024-04-22T23:45:52.598 INFO:tasks.workunit.client.0.smithi031.stdout:9/838: rename d2/dc/d8b/d3d/d40/da4/db1/cd2 to d2/dd/d75/d64/c115 0 2024-04-22T23:45:52.600 INFO:tasks.workunit.client.0.smithi031.stdout:5/795: symlink db/d16/d2a/d59/d90/da4/l106 0 2024-04-22T23:45:52.600 INFO:tasks.workunit.client.1.smithi044.stdout:9/26: mknod d2/d9/cb 0 2024-04-22T23:45:52.601 INFO:tasks.workunit.client.0.smithi031.stdout:1/846: mkdir d1/db/d16/d1f/d52/d11d 0 2024-04-22T23:45:52.602 INFO:tasks.workunit.client.0.smithi031.stdout:3/754: link d4/d83/fab d4/d6/d4c/d8c/d93/f103 0 2024-04-22T23:45:52.602 INFO:tasks.workunit.client.1.smithi044.stdout:9/27: mkdir d2/dc 0 2024-04-22T23:45:52.603 INFO:tasks.workunit.client.1.smithi044.stdout:9/28: readlink - no filename 2024-04-22T23:45:52.605 INFO:tasks.workunit.client.1.smithi044.stdout:9/29: unlink d2/d6/f8 0 2024-04-22T23:45:52.606 INFO:tasks.workunit.client.1.smithi044.stdout:9/30: dread - d2/d5/f7 zero size 2024-04-22T23:45:52.608 INFO:tasks.workunit.client.0.smithi031.stdout:9/839: dwrite d2/dc/f10 [4194304,4194304] 0 2024-04-22T23:45:52.610 INFO:tasks.workunit.client.1.smithi044.stdout:9/31: dread d2/f4 [0,4194304] 0 2024-04-22T23:45:52.611 INFO:tasks.workunit.client.0.smithi031.stdout:3/755: mknod d4/d54/d2c/d89/d2e/c104 0 2024-04-22T23:45:52.611 INFO:tasks.workunit.client.0.smithi031.stdout:2/882: dread d6/dc/d57/d52/db5/fc0 [0,4194304] 0 2024-04-22T23:45:52.611 INFO:tasks.workunit.client.0.smithi031.stdout:8/784: truncate d5/d48/d61/d6b/d6c/f70 473373 0 2024-04-22T23:45:52.614 INFO:tasks.workunit.client.0.smithi031.stdout:9/840: write d2/d15/d5a/fa5 [173473,115723] 0 2024-04-22T23:45:52.614 INFO:tasks.workunit.client.0.smithi031.stdout:3/756: chown d4/d6/d11/d25/d44/d7f/d5f/ffa 20940 1 2024-04-22T23:45:52.615 INFO:tasks.workunit.client.1.smithi044.stdout:9/32: fsync d2/d6/fa 0 2024-04-22T23:45:52.616 INFO:tasks.workunit.client.0.smithi031.stdout:5/796: creat db/d16/d2a/d36/f107 x:0 0 0 2024-04-22T23:45:52.616 INFO:tasks.workunit.client.0.smithi031.stdout:8/785: fdatasync d5/d48/d61/d6b/d6c/f73 0 2024-04-22T23:45:52.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/883: truncate d6/f19 4850027 0 2024-04-22T23:45:52.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/884: write d6/dc/f4e [1328049,57850] 0 2024-04-22T23:45:52.620 INFO:tasks.workunit.client.1.smithi044.stdout:9/33: dread d2/f3 [0,4194304] 0 2024-04-22T23:45:52.620 INFO:tasks.workunit.client.1.smithi044.stdout:9/34: truncate d2/f3 903093 0 2024-04-22T23:45:52.621 INFO:tasks.workunit.client.0.smithi031.stdout:9/841: dread d2/dc/d3e/f41 [4194304,4194304] 0 2024-04-22T23:45:52.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/35: symlink d2/d5/ld 0 2024-04-22T23:45:52.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/36: readlink d2/d5/ld 0 2024-04-22T23:45:52.628 INFO:tasks.workunit.client.0.smithi031.stdout:5/797: symlink db/d16/d1d/d24/d32/ded/l108 0 2024-04-22T23:45:52.629 INFO:tasks.workunit.client.1.smithi044.stdout:9/37: symlink d2/le 0 2024-04-22T23:45:52.630 INFO:tasks.workunit.client.1.smithi044.stdout:9/38: fdatasync d2/d5/f7 0 2024-04-22T23:45:52.630 INFO:tasks.workunit.client.0.smithi031.stdout:2/885: rmdir d6/dc/d57/d52/db5 39 2024-04-22T23:45:52.634 INFO:tasks.workunit.client.0.smithi031.stdout:8/786: symlink d5/da/d12/d2e/d7e/d7c/lf4 0 2024-04-22T23:45:52.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/851: dwrite d0/d13/fe3 [4194304,4194304] 0 2024-04-22T23:45:52.641 INFO:tasks.workunit.client.0.smithi031.stdout:0/852: write d0/f3d [355797,3632] 0 2024-04-22T23:45:52.642 INFO:tasks.workunit.client.0.smithi031.stdout:9/842: dread d2/dc/d8b/d3d/d40/da4/fab [0,4194304] 0 2024-04-22T23:45:52.645 INFO:tasks.workunit.client.0.smithi031.stdout:7/834: sync 2024-04-22T23:45:52.645 INFO:tasks.workunit.client.0.smithi031.stdout:4/948: sync 2024-04-22T23:45:52.645 INFO:tasks.workunit.client.0.smithi031.stdout:6/948: sync 2024-04-22T23:45:52.645 INFO:tasks.workunit.client.0.smithi031.stdout:1/847: dwrite d1/d35/d8d/ff7 [0,4194304] 0 2024-04-22T23:45:52.646 INFO:tasks.workunit.client.1.smithi044.stdout:9/39: truncate d2/f4 1592126 0 2024-04-22T23:45:52.647 INFO:tasks.workunit.client.0.smithi031.stdout:0/853: chown d0/d13/d18/d1e/d25/d8e/da7/d2b 66 1 2024-04-22T23:45:52.648 INFO:tasks.workunit.client.1.smithi044.stdout:9/40: mknod d2/d9/cf 0 2024-04-22T23:45:52.650 INFO:tasks.workunit.client.0.smithi031.stdout:8/787: rename d5/da/d12/d37/d4e/de7/cef to d5/da/df/d33/dd4/cf5 0 2024-04-22T23:45:52.652 INFO:tasks.workunit.client.1.smithi044.stdout:9/41: link d2/d9/cb d2/d5/c10 0 2024-04-22T23:45:52.653 INFO:tasks.workunit.client.0.smithi031.stdout:3/757: dwrite d4/d6/d4c/d8c/d93/fd6 [0,4194304] 0 2024-04-22T23:45:52.655 INFO:tasks.workunit.client.1.smithi044.stdout:9/42: mknod d2/d6/c11 0 2024-04-22T23:45:52.656 INFO:tasks.workunit.client.1.smithi044.stdout:9/43: fdatasync d2/f3 0 2024-04-22T23:45:52.657 INFO:tasks.workunit.client.0.smithi031.stdout:5/798: dread db/d16/d2a/d7c/fa7 [0,4194304] 0 2024-04-22T23:45:52.657 INFO:tasks.workunit.client.1.smithi044.stdout:9/44: chown d2/d9/cf 28 1 2024-04-22T23:45:52.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/835: creat d9/d15/d6d/d65/dea/f124 x:0 0 0 2024-04-22T23:45:52.658 INFO:tasks.workunit.client.0.smithi031.stdout:3/758: read d4/d6/fca [25618,15197] 0 2024-04-22T23:45:52.660 INFO:tasks.workunit.client.0.smithi031.stdout:7/836: write d9/d14/d22/d5c/d5f/fdf [902968,86276] 0 2024-04-22T23:45:52.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/848: unlink d1/db/d16/d1f/d52/db0/fea 0 2024-04-22T23:45:52.665 INFO:tasks.workunit.client.0.smithi031.stdout:0/854: creat d0/d1/d2/d36/d11/d1a/f113 x:0 0 0 2024-04-22T23:45:52.668 INFO:tasks.workunit.client.0.smithi031.stdout:5/799: rmdir db/d16/d1d/d24/d58/d5d 39 2024-04-22T23:45:52.670 INFO:tasks.workunit.client.0.smithi031.stdout:2/886: dread d6/dc/d9a/fb9 [0,4194304] 0 2024-04-22T23:45:52.671 INFO:tasks.workunit.client.1.smithi044.stdout:9/45: dread d2/f4 [0,4194304] 0 2024-04-22T23:45:52.671 INFO:tasks.workunit.client.0.smithi031.stdout:1/849: mknod d1/d35/dbb/c11e 0 2024-04-22T23:45:52.672 INFO:tasks.workunit.client.0.smithi031.stdout:7/837: mkdir d9/d14/d22/d28/d125 0 2024-04-22T23:45:52.674 INFO:tasks.workunit.client.0.smithi031.stdout:4/949: mknod db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/c148 0 2024-04-22T23:45:52.677 INFO:tasks.workunit.client.1.smithi044.stdout:9/46: rmdir d2/d5 39 2024-04-22T23:45:52.678 INFO:tasks.workunit.client.0.smithi031.stdout:4/950: dread db/d14/d2d/d49/d59/d6a/d41/fc3 [0,4194304] 0 2024-04-22T23:45:52.679 INFO:tasks.workunit.client.0.smithi031.stdout:9/843: write d2/dd/d75/d64/d5d/d59/d8e/f91 [704792,93319] 0 2024-04-22T23:45:52.679 INFO:tasks.workunit.client.0.smithi031.stdout:3/759: symlink d4/d54/d56/daf/l105 0 2024-04-22T23:45:52.680 INFO:tasks.workunit.client.1.smithi044.stdout:9/47: mknod d2/d6/c12 0 2024-04-22T23:45:52.681 INFO:tasks.workunit.client.1.smithi044.stdout:9/48: truncate d2/d6/fa 247923 0 2024-04-22T23:45:52.682 INFO:tasks.workunit.client.0.smithi031.stdout:6/949: write d3/d2a/d3c/d40/f4a [4683178,86753] 0 2024-04-22T23:45:52.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/760: chown d4/d6/d11/d25/d44/d7f/d5f/f9d 32972 1 2024-04-22T23:45:52.682 INFO:tasks.workunit.client.0.smithi031.stdout:2/887: symlink d6/dc/d48/d4b/d6b/de8/l121 0 2024-04-22T23:45:52.684 INFO:tasks.workunit.client.0.smithi031.stdout:6/950: chown d3/d1f/d60/cc8 11722 1 2024-04-22T23:45:52.685 INFO:tasks.workunit.client.1.smithi044.stdout:9/49: creat d2/d6/f13 x:0 0 0 2024-04-22T23:45:52.686 INFO:tasks.workunit.client.1.smithi044.stdout:9/50: write d2/f3 [954809,71450] 0 2024-04-22T23:45:52.687 INFO:tasks.workunit.client.1.smithi044.stdout:9/51: truncate d2/d6/f13 340153 0 2024-04-22T23:45:52.687 INFO:tasks.workunit.client.0.smithi031.stdout:1/850: rename d1/d35/d36/d108/d4b/d6f/dd8/faf to d1/de/d1e/d94/f11f 0 2024-04-22T23:45:52.689 INFO:tasks.workunit.client.0.smithi031.stdout:9/844: dwrite d2/dd/d75/d64/d5d/d59/fc7 [0,4194304] 0 2024-04-22T23:45:52.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/838: dwrite d9/d15/d3a/d41/f54 [0,4194304] 0 2024-04-22T23:45:52.692 INFO:tasks.workunit.client.0.smithi031.stdout:3/761: truncate d4/d6/d22/ff4 2183597 0 2024-04-22T23:45:52.695 INFO:tasks.workunit.client.0.smithi031.stdout:3/762: fsync d4/d6/d11/dc0/fcf 0 2024-04-22T23:45:52.695 INFO:tasks.workunit.client.0.smithi031.stdout:2/888: unlink d6/dc/d9a/d6c/db2/df0/d94/l103 0 2024-04-22T23:45:52.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/951: symlink db/d14/d2d/d49/d59/d6a/d63/d86/df2/d10f/l149 0 2024-04-22T23:45:52.696 INFO:tasks.workunit.client.0.smithi031.stdout:0/855: write d0/d1/d2/f23 [97109,69359] 0 2024-04-22T23:45:52.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/851: mkdir d1/d3f/d54/d84/d120 0 2024-04-22T23:45:52.699 INFO:tasks.workunit.client.0.smithi031.stdout:2/889: dread - d6/dc/d42/d24/fe2 zero size 2024-04-22T23:45:52.700 INFO:tasks.workunit.client.0.smithi031.stdout:0/856: chown d0/d1/d2/d36/d11/d1a/f100 1228819 1 2024-04-22T23:45:52.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/852: chown d1/d35/l48 2 1 2024-04-22T23:45:52.706 INFO:tasks.workunit.client.0.smithi031.stdout:1/853: chown d1/d35/d36/d108/d4e 31600566 1 2024-04-22T23:45:52.707 INFO:tasks.workunit.client.0.smithi031.stdout:5/800: creat db/d16/d1d/d24/d58/d5d/f109 x:0 0 0 2024-04-22T23:45:52.707 INFO:tasks.workunit.client.0.smithi031.stdout:3/763: dwrite d4/d6/d11/f19 [4194304,4194304] 0 2024-04-22T23:45:52.710 INFO:tasks.workunit.client.0.smithi031.stdout:5/801: chown db/d16/d2a/l33 8462810 1 2024-04-22T23:45:52.711 INFO:tasks.workunit.client.0.smithi031.stdout:6/951: dread d3/d2a/d3c/db6/ff2 [4194304,4194304] 0 2024-04-22T23:45:52.713 INFO:tasks.workunit.client.0.smithi031.stdout:5/802: fdatasync db/d16/d1d/d24/d58/d5d/f77 0 2024-04-22T23:45:52.713 INFO:tasks.workunit.client.0.smithi031.stdout:0/857: symlink d0/d1/d2/d36/d11/d1a/dfc/dad/dba/l114 0 2024-04-22T23:45:52.718 INFO:tasks.workunit.client.0.smithi031.stdout:1/854: creat d1/de/d1e/d34/d66/f121 x:0 0 0 2024-04-22T23:45:52.721 INFO:tasks.workunit.client.0.smithi031.stdout:3/764: rmdir d4/d6/d11/d3e 39 2024-04-22T23:45:52.723 INFO:tasks.workunit.client.0.smithi031.stdout:3/765: write d4/d6/da/f67 [302644,83249] 0 2024-04-22T23:45:52.723 INFO:tasks.workunit.client.0.smithi031.stdout:7/839: link d9/d14/d1f/d7f/l57 d9/d14/d1f/dc6/df4/l126 0 2024-04-22T23:45:52.724 INFO:tasks.workunit.client.1.smithi044.stdout:9/52: write d2/d6/f13 [1077604,32323] 0 2024-04-22T23:45:52.726 INFO:tasks.workunit.client.0.smithi031.stdout:7/840: write d9/d15/f60 [2162454,14281] 0 2024-04-22T23:45:52.727 INFO:tasks.workunit.client.0.smithi031.stdout:1/855: dwrite d1/d3f/d54/dba/f98 [0,4194304] 0 2024-04-22T23:45:52.727 INFO:tasks.workunit.client.0.smithi031.stdout:0/858: creat d0/d1/d2/db4/f115 x:0 0 0 2024-04-22T23:45:52.728 INFO:tasks.workunit.client.0.smithi031.stdout:4/952: write db/d14/d1d/d7b/d7f/fe8 [814950,30697] 0 2024-04-22T23:45:52.728 INFO:tasks.workunit.client.1.smithi044.stdout:9/53: chown d2/d9/cb 0 1 2024-04-22T23:45:52.728 INFO:tasks.workunit.client.0.smithi031.stdout:7/841: chown d9/d15/d6d/d65/f72 11506350 1 2024-04-22T23:45:52.734 INFO:tasks.workunit.client.0.smithi031.stdout:0/859: write d0/d13/d18/d1e/d25/d8e/dac/dda/fe1 [425759,123974] 0 2024-04-22T23:45:52.736 INFO:tasks.workunit.client.1.smithi044.stdout:9/54: dwrite d2/d6/f13 [0,4194304] 0 2024-04-22T23:45:52.736 INFO:tasks.workunit.client.0.smithi031.stdout:3/766: fsync d4/d54/d56/daf/fb7 0 2024-04-22T23:45:52.736 INFO:tasks.workunit.client.0.smithi031.stdout:0/860: dread - d0/d1/d2/d36/d11/d1a/f113 zero size 2024-04-22T23:45:52.736 INFO:tasks.workunit.client.0.smithi031.stdout:2/890: dread d6/dc/d42/d24/f34 [4194304,4194304] 0 2024-04-22T23:45:52.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/856: creat d1/d35/d8d/d102/d81/f122 x:0 0 0 2024-04-22T23:45:52.739 INFO:tasks.workunit.client.1.smithi044.stdout:9/55: dread - d2/d5/f7 zero size 2024-04-22T23:45:52.740 INFO:tasks.workunit.client.0.smithi031.stdout:2/891: write d6/dc/d17/d69/fa8 [293018,106006] 0 2024-04-22T23:45:52.741 INFO:tasks.workunit.client.0.smithi031.stdout:0/861: unlink d0/f3d 0 2024-04-22T23:45:52.742 INFO:tasks.workunit.client.0.smithi031.stdout:3/767: symlink d4/d6/d11/d25/d44/l106 0 2024-04-22T23:45:52.745 INFO:tasks.workunit.client.1.smithi044.stdout:9/56: symlink d2/dc/l14 0 2024-04-22T23:45:52.745 INFO:tasks.workunit.client.1.smithi044.stdout:9/57: stat d2/d9 0 2024-04-22T23:45:52.747 INFO:tasks.workunit.client.1.smithi044.stdout:9/58: creat d2/dc/f15 x:0 0 0 2024-04-22T23:45:52.749 INFO:tasks.workunit.client.0.smithi031.stdout:6/952: truncate d3/d77/d8d/fe0 2032457 0 2024-04-22T23:45:52.750 INFO:tasks.workunit.client.0.smithi031.stdout:7/842: dwrite d9/d14/d1f/d7f/ffa [0,4194304] 0 2024-04-22T23:45:52.752 INFO:tasks.workunit.client.0.smithi031.stdout:1/857: truncate d1/d35/d8d/d102/f50 3389571 0 2024-04-22T23:45:52.756 INFO:tasks.workunit.client.0.smithi031.stdout:6/953: write d3/d35/db1/f11f [174877,79168] 0 2024-04-22T23:45:52.756 INFO:tasks.workunit.client.0.smithi031.stdout:2/892: dwrite d6/dc/d48/fe1 [0,4194304] 0 2024-04-22T23:45:52.756 INFO:tasks.workunit.client.0.smithi031.stdout:0/862: getdents d0/d13/d18/d1e/d25/d8e/da7/dc9 0 2024-04-22T23:45:52.758 INFO:tasks.workunit.client.0.smithi031.stdout:1/858: fsync d1/d35/d8d/d102/dc1/f105 0 2024-04-22T23:45:52.758 INFO:tasks.workunit.client.1.smithi044.stdout:9/59: fdatasync d2/d6/f13 0 2024-04-22T23:45:52.758 INFO:tasks.workunit.client.1.smithi044.stdout:9/60: readlink d2/dc/l14 0 2024-04-22T23:45:52.760 INFO:tasks.workunit.client.0.smithi031.stdout:3/768: dwrite d4/d83/dd8/de6/ff5 [0,4194304] 0 2024-04-22T23:45:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:2/893: fsync d6/dc/d42/d24/f8f 0 2024-04-22T23:45:52.767 INFO:tasks.workunit.client.0.smithi031.stdout:6/954: mkdir d3/d2a/d3c/d5e/d145 0 2024-04-22T23:45:52.769 INFO:tasks.workunit.client.1.smithi044.stdout:9/61: dwrite d2/f3 [0,4194304] 0 2024-04-22T23:45:52.770 INFO:tasks.workunit.client.1.smithi044.stdout:9/62: chown d2/d6/fa 2083749 1 2024-04-22T23:45:52.773 INFO:tasks.workunit.client.0.smithi031.stdout:0/863: rmdir d0/d1/d2/d36/d11/d1c 39 2024-04-22T23:45:52.774 INFO:tasks.workunit.client.0.smithi031.stdout:3/769: unlink d4/d49/d72/lfc 0 2024-04-22T23:45:52.777 INFO:tasks.workunit.client.0.smithi031.stdout:7/843: dread d9/f4b [0,4194304] 0 2024-04-22T23:45:52.779 INFO:tasks.workunit.client.0.smithi031.stdout:7/844: write d9/f3b [8456637,59977] 0 2024-04-22T23:45:52.780 INFO:tasks.workunit.client.0.smithi031.stdout:2/894: fsync d6/dc/d57/fdb 0 2024-04-22T23:45:52.781 INFO:tasks.workunit.client.0.smithi031.stdout:0/864: mknod d0/d13/d18/dbf/dc3/c116 0 2024-04-22T23:45:52.783 INFO:tasks.workunit.client.0.smithi031.stdout:3/770: mknod d4/d49/d72/c107 0 2024-04-22T23:45:52.785 INFO:tasks.workunit.client.0.smithi031.stdout:7/845: rmdir d9/d15/d6d/da9 39 2024-04-22T23:45:52.788 INFO:tasks.workunit.client.0.smithi031.stdout:1/859: rename d1/la to d1/d35/l123 0 2024-04-22T23:45:52.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/895: rmdir d6/dc/d9a/d6c/db2 39 2024-04-22T23:45:52.792 INFO:tasks.workunit.client.0.smithi031.stdout:3/771: mkdir d4/d54/d2c/d89/d2e/d8d/d108 0 2024-04-22T23:45:52.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/865: creat d0/d1/d2/d36/d11/d1a/d56/f117 x:0 0 0 2024-04-22T23:45:52.794 INFO:tasks.workunit.client.0.smithi031.stdout:0/866: read d0/fbe [1745502,111205] 0 2024-04-22T23:45:52.798 INFO:tasks.workunit.client.0.smithi031.stdout:1/860: dwrite d1/d35/d36/d108/d4b/d6f/da6/dec/f10c [0,4194304] 0 2024-04-22T23:45:52.800 INFO:tasks.workunit.client.0.smithi031.stdout:6/955: dread d3/f38 [0,4194304] 0 2024-04-22T23:45:52.801 INFO:tasks.workunit.client.0.smithi031.stdout:1/861: mkdir d1/de/d1e/d34/d66/d124 0 2024-04-22T23:45:52.802 INFO:tasks.workunit.client.0.smithi031.stdout:0/867: mknod d0/d1/d2/d36/d11/d1a/dfc/c118 0 2024-04-22T23:45:52.803 INFO:tasks.workunit.client.0.smithi031.stdout:2/896: write d6/dc/f62 [586529,25989] 0 2024-04-22T23:45:52.806 INFO:tasks.workunit.client.0.smithi031.stdout:6/956: symlink d3/d1f/d4c/d8e/d13f/l146 0 2024-04-22T23:45:52.806 INFO:tasks.workunit.client.0.smithi031.stdout:0/868: stat d0/d1/d2/d36/d11/d1a/dfc/dad/dba 0 2024-04-22T23:45:52.807 INFO:tasks.workunit.client.0.smithi031.stdout:2/897: fdatasync d6/dc/f9f 0 2024-04-22T23:45:52.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/846: dwrite d9/d15/d6d/d65/f86 [0,4194304] 0 2024-04-22T23:45:52.810 INFO:tasks.workunit.client.0.smithi031.stdout:1/862: getdents d1/db/dca 0 2024-04-22T23:45:52.816 INFO:tasks.workunit.client.0.smithi031.stdout:7/847: creat d9/d15/d3f/ddc/f127 x:0 0 0 2024-04-22T23:45:52.817 INFO:tasks.workunit.client.0.smithi031.stdout:6/957: write d3/d35/f94 [169540,12739] 0 2024-04-22T23:45:52.817 INFO:tasks.workunit.client.0.smithi031.stdout:7/848: dread - d9/d15/d6d/d65/dea/f121 zero size 2024-04-22T23:45:52.820 INFO:tasks.workunit.client.0.smithi031.stdout:2/898: unlink d6/dc/d9a/d6c/db2/df0/d94/dab/cdf 0 2024-04-22T23:45:52.824 INFO:tasks.workunit.client.0.smithi031.stdout:1/863: truncate d1/d35/d36/d108/fb5 1347074 0 2024-04-22T23:45:52.824 INFO:tasks.workunit.client.0.smithi031.stdout:2/899: truncate d6/dc/d9a/d6c/db2/fd6 245121 0 2024-04-22T23:45:52.825 INFO:tasks.workunit.client.0.smithi031.stdout:7/849: creat d9/d14/d22/d28/d125/f128 x:0 0 0 2024-04-22T23:45:52.827 INFO:tasks.workunit.client.0.smithi031.stdout:1/864: readlink d1/d3f/l100 0 2024-04-22T23:45:52.829 INFO:tasks.workunit.client.0.smithi031.stdout:7/850: stat d9/d15/d6d/d65/f102 0 2024-04-22T23:45:52.829 INFO:tasks.workunit.client.0.smithi031.stdout:0/869: dwrite d0/db5/df6/ffb [4194304,4194304] 0 2024-04-22T23:45:52.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/865: stat d1/d35/d8d/d102/dc1 0 2024-04-22T23:45:52.832 INFO:tasks.workunit.client.0.smithi031.stdout:7/851: mkdir d9/d14/dd6/d129 0 2024-04-22T23:45:52.833 INFO:tasks.workunit.client.0.smithi031.stdout:1/866: write d1/d35/d8d/d102/fae [215358,62894] 0 2024-04-22T23:45:52.836 INFO:tasks.workunit.client.0.smithi031.stdout:1/867: read d1/d35/d8d/d102/f75 [4137491,124064] 0 2024-04-22T23:45:52.843 INFO:tasks.workunit.client.0.smithi031.stdout:7/852: link d9/d14/d1f/d45/de3/f104 d9/d14/d1f/d45/f12a 0 2024-04-22T23:45:52.845 INFO:tasks.workunit.client.0.smithi031.stdout:1/868: dwrite d1/d35/d36/d108/feb [0,4194304] 0 2024-04-22T23:45:52.846 INFO:tasks.workunit.client.0.smithi031.stdout:6/958: dwrite d3/d35/d86/f10e [0,4194304] 0 2024-04-22T23:45:52.851 INFO:tasks.workunit.client.0.smithi031.stdout:2/900: dread d6/dc/d42/d24/f34 [0,4194304] 0 2024-04-22T23:45:52.854 INFO:tasks.workunit.client.0.smithi031.stdout:7/853: dread d9/d14/d1f/d45/da6/daa/db0/fb6 [0,4194304] 0 2024-04-22T23:45:52.854 INFO:tasks.workunit.client.0.smithi031.stdout:0/870: dwrite d0/d13/d18/d1e/d25/f85 [0,4194304] 0 2024-04-22T23:45:52.857 INFO:tasks.workunit.client.0.smithi031.stdout:1/869: rename d1/db3 to d1/de/d1e/d94/d125 0 2024-04-22T23:45:52.858 INFO:tasks.workunit.client.0.smithi031.stdout:7/854: dread - d9/d14/d22/d5c/d5f/f98 zero size 2024-04-22T23:45:52.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/871: symlink d0/d1/d2/db4/de4/l119 0 2024-04-22T23:45:52.859 INFO:tasks.workunit.client.0.smithi031.stdout:1/870: write d1/d35/d36/d108/d4e/dd9/fff [1017818,24105] 0 2024-04-22T23:45:52.860 INFO:tasks.workunit.client.0.smithi031.stdout:2/901: mknod d6/dc/c122 0 2024-04-22T23:45:52.861 INFO:tasks.workunit.client.0.smithi031.stdout:1/871: stat d1/db/d16/cf2 0 2024-04-22T23:45:52.863 INFO:tasks.workunit.client.0.smithi031.stdout:7/855: unlink d9/d15/ffb 0 2024-04-22T23:45:52.863 INFO:tasks.workunit.client.0.smithi031.stdout:0/872: creat d0/d13/d18/d1e/f11a x:0 0 0 2024-04-22T23:45:52.864 INFO:tasks.workunit.client.0.smithi031.stdout:0/873: fdatasync d0/d13/d18/d1e/f6e 0 2024-04-22T23:45:52.866 INFO:tasks.workunit.client.0.smithi031.stdout:2/902: unlink d6/dc/d42/d24/c31 0 2024-04-22T23:45:52.869 INFO:tasks.workunit.client.0.smithi031.stdout:1/872: symlink d1/d35/d8d/d102/d81/l126 0 2024-04-22T23:45:52.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/874: mkdir d0/d1/d2/d36/d11/d1a/dfc/dad/dba/d11b 0 2024-04-22T23:45:52.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/856: dwrite d9/d15/dcf/ffc [0,4194304] 0 2024-04-22T23:45:52.875 INFO:tasks.workunit.client.0.smithi031.stdout:2/903: rename d6/dc/d42/f2c to d6/dc/d17/d69/d90/f123 0 2024-04-22T23:45:52.878 INFO:tasks.workunit.client.0.smithi031.stdout:0/875: dread d0/d13/d18/dbf/fdb [0,4194304] 0 2024-04-22T23:45:52.882 INFO:tasks.workunit.client.0.smithi031.stdout:7/857: rename d9/l4e to d9/d15/d6d/dcc/l12b 0 2024-04-22T23:45:52.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/873: truncate d1/d35/d36/fbd 2172077 0 2024-04-22T23:45:52.888 INFO:tasks.workunit.client.0.smithi031.stdout:6/959: dread d3/d2a/d3c/f98 [0,4194304] 0 2024-04-22T23:45:52.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/874: write d1/db/d16/f25 [2746503,110566] 0 2024-04-22T23:45:52.891 INFO:tasks.workunit.client.0.smithi031.stdout:6/960: mkdir d3/d1f/d60/d147 0 2024-04-22T23:45:52.897 INFO:tasks.workunit.client.0.smithi031.stdout:6/961: symlink d3/d1f/d4c/de2/l148 0 2024-04-22T23:45:52.899 INFO:tasks.workunit.client.0.smithi031.stdout:7/858: truncate d9/d15/d6d/d65/f6b 520586 0 2024-04-22T23:45:52.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/875: getdents d1/db/dca 0 2024-04-22T23:45:52.900 INFO:tasks.workunit.client.0.smithi031.stdout:0/876: dwrite d0/d1/d2/d36/d11/d1c/f88 [0,4194304] 0 2024-04-22T23:45:52.901 INFO:tasks.workunit.client.0.smithi031.stdout:6/962: mknod d3/d35/d75/d7d/dc4/d124/c149 0 2024-04-22T23:45:52.904 INFO:tasks.workunit.client.0.smithi031.stdout:1/876: mkdir d1/db/d127 0 2024-04-22T23:45:52.905 INFO:tasks.workunit.client.0.smithi031.stdout:7/859: truncate d9/d14/d1f/d45/da6/daa/db0/fad 1099757 0 2024-04-22T23:45:52.906 INFO:tasks.workunit.client.0.smithi031.stdout:1/877: write d1/de/d1e/d34/d41/f44 [3631195,44215] 0 2024-04-22T23:45:52.908 INFO:tasks.workunit.client.0.smithi031.stdout:6/963: fdatasync d3/d35/fc2 0 2024-04-22T23:45:52.911 INFO:tasks.workunit.client.0.smithi031.stdout:7/860: dread d9/d14/d22/d5c/f80 [0,4194304] 0 2024-04-22T23:45:52.911 INFO:tasks.workunit.client.0.smithi031.stdout:1/878: unlink d1/de/l68 0 2024-04-22T23:45:52.914 INFO:tasks.workunit.client.0.smithi031.stdout:6/964: rename d3/d128 to d3/d35/d75/d7d/d135/d14a 0 2024-04-22T23:45:52.915 INFO:tasks.workunit.client.0.smithi031.stdout:1/879: mknod d1/d35/d36/d108/d4b/c128 0 2024-04-22T23:45:52.916 INFO:tasks.workunit.client.0.smithi031.stdout:7/861: creat d9/d15/d6d/d65/dea/f12c x:0 0 0 2024-04-22T23:45:52.918 INFO:tasks.workunit.client.0.smithi031.stdout:6/965: mknod d3/d1f/d4c/de2/c14b 0 2024-04-22T23:45:52.922 INFO:tasks.workunit.client.0.smithi031.stdout:6/966: mkdir d3/d2a/d3c/d40/d107/d14c 0 2024-04-22T23:45:52.927 INFO:tasks.workunit.client.0.smithi031.stdout:1/880: dwrite d1/d35/d36/f9e [4194304,4194304] 0 2024-04-22T23:45:52.928 INFO:tasks.workunit.client.0.smithi031.stdout:7/862: dwrite d9/d14/d1f/d45/da6/daa/d10b/f10e [0,4194304] 0 2024-04-22T23:45:52.932 INFO:tasks.workunit.client.0.smithi031.stdout:6/967: link d3/d1f/d60/f84 d3/f14d 0 2024-04-22T23:45:52.943 INFO:tasks.workunit.client.0.smithi031.stdout:1/881: creat d1/d35/d36/f129 x:0 0 0 2024-04-22T23:45:52.944 INFO:tasks.workunit.client.0.smithi031.stdout:7/863: mkdir d9/d15/d3a/d41/d12d 0 2024-04-22T23:45:52.999 INFO:tasks.workunit.client.0.smithi031.stdout:7/864: fdatasync d9/d15/d3f/ddc/fe5 0 2024-04-22T23:45:52.999 INFO:tasks.workunit.client.0.smithi031.stdout:7/865: stat f3 0 2024-04-22T23:45:53.002 INFO:tasks.workunit.client.0.smithi031.stdout:6/968: getdents d3/d2a/d3c/d40/d7a 0 2024-04-22T23:45:53.005 INFO:tasks.workunit.client.0.smithi031.stdout:6/969: fdatasync d3/f25 0 2024-04-22T23:45:53.008 INFO:tasks.workunit.client.0.smithi031.stdout:6/970: creat d3/d1f/d4c/df7/f14e x:0 0 0 2024-04-22T23:45:53.014 INFO:tasks.workunit.client.0.smithi031.stdout:7/866: dread f3 [0,4194304] 0 2024-04-22T23:45:53.026 INFO:tasks.workunit.client.0.smithi031.stdout:7/867: getdents d9/d15/d6d/d65 0 2024-04-22T23:45:53.031 INFO:tasks.workunit.client.0.smithi031.stdout:7/868: symlink d9/d15/d3a/d41/d12d/l12e 0 2024-04-22T23:45:53.031 INFO:tasks.workunit.client.0.smithi031.stdout:6/971: dread d3/d77/db5/d110/f113 [0,4194304] 0 2024-04-22T23:45:53.032 INFO:tasks.workunit.client.0.smithi031.stdout:6/972: read d3/d35/db1/f11f [220969,73988] 0 2024-04-22T23:45:53.037 INFO:tasks.workunit.client.0.smithi031.stdout:6/973: creat d3/d35/db1/de6/f14f x:0 0 0 2024-04-22T23:45:53.038 INFO:tasks.workunit.client.0.smithi031.stdout:7/869: dwrite d9/d15/dcf/fd2 [0,4194304] 0 2024-04-22T23:45:53.041 INFO:tasks.workunit.client.0.smithi031.stdout:6/974: mknod d3/d1f/d4c/df9/c150 0 2024-04-22T23:45:53.042 INFO:tasks.workunit.client.0.smithi031.stdout:7/870: symlink d9/d14/d1f/d45/da6/l12f 0 2024-04-22T23:45:53.043 INFO:tasks.workunit.client.0.smithi031.stdout:7/871: read d9/d14/d22/d28/fd5 [383206,8818] 0 2024-04-22T23:45:53.045 INFO:tasks.workunit.client.0.smithi031.stdout:6/975: creat d3/d35/d75/d7d/d135/f151 x:0 0 0 2024-04-22T23:45:53.058 INFO:tasks.workunit.client.0.smithi031.stdout:6/976: dread d3/d1f/f4f [0,4194304] 0 2024-04-22T23:45:53.059 INFO:tasks.workunit.client.0.smithi031.stdout:6/977: chown d3/d1f/da3/fd5 0 1 2024-04-22T23:45:53.062 INFO:tasks.workunit.client.0.smithi031.stdout:8/788: sync 2024-04-22T23:45:53.062 INFO:tasks.workunit.client.0.smithi031.stdout:9/845: sync 2024-04-22T23:45:53.063 INFO:tasks.workunit.client.0.smithi031.stdout:6/978: rename d3/d77/db5/d110/l12c to d3/d35/d75/d7d/d135/d14a/l152 0 2024-04-22T23:45:53.063 INFO:tasks.workunit.client.0.smithi031.stdout:9/846: fdatasync d2/d15/f7c 0 2024-04-22T23:45:53.066 INFO:tasks.workunit.client.0.smithi031.stdout:8/789: creat d5/da/d12/d2e/d4d/d9a/ff6 x:0 0 0 2024-04-22T23:45:53.069 INFO:tasks.workunit.client.0.smithi031.stdout:9/847: truncate d2/dd/d75/d64/f55 102960 0 2024-04-22T23:45:53.069 INFO:tasks.workunit.client.0.smithi031.stdout:6/979: dread d3/d2a/d85/fab [0,4194304] 0 2024-04-22T23:45:53.069 INFO:tasks.workunit.client.0.smithi031.stdout:9/848: chown d2/dd/d75/d64/f5f 1186 1 2024-04-22T23:45:53.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/980: fsync d3/d1f/d60/f8f 0 2024-04-22T23:45:53.114 INFO:tasks.workunit.client.0.smithi031.stdout:7/872: write d9/d14/d1f/d45/fa2 [2676781,8831] 0 2024-04-22T23:45:53.114 INFO:tasks.workunit.client.0.smithi031.stdout:6/981: mkdir d3/d2a/d3c/db6/d153 0 2024-04-22T23:45:53.116 INFO:tasks.workunit.client.0.smithi031.stdout:9/849: rename d2/dc/d3e/f56 to d2/dd/d75/d64/d5d/d109/f116 0 2024-04-22T23:45:53.116 INFO:tasks.workunit.client.0.smithi031.stdout:7/873: chown d9/d14/d1f/d7f/fc7 41351215 1 2024-04-22T23:45:53.117 INFO:tasks.workunit.client.0.smithi031.stdout:6/982: creat d3/da5/f154 x:0 0 0 2024-04-22T23:45:53.119 INFO:tasks.workunit.client.0.smithi031.stdout:7/874: creat d9/d14/d22/d28/f130 x:0 0 0 2024-04-22T23:45:53.119 INFO:tasks.workunit.client.0.smithi031.stdout:8/790: dwrite d5/da/d12/d1d/fca [0,4194304] 0 2024-04-22T23:45:53.120 INFO:tasks.workunit.client.0.smithi031.stdout:6/983: chown d3/d1f/d4c/d8e/l112 4 1 2024-04-22T23:45:53.123 INFO:tasks.workunit.client.0.smithi031.stdout:7/875: symlink d9/d15/dac/l131 0 2024-04-22T23:45:53.125 INFO:tasks.workunit.client.0.smithi031.stdout:8/791: fsync d5/d48/d61/d6b/d6c/f70 0 2024-04-22T23:45:53.128 INFO:tasks.workunit.client.0.smithi031.stdout:9/850: dwrite d2/dc/f43 [0,4194304] 0 2024-04-22T23:45:53.129 INFO:tasks.workunit.client.0.smithi031.stdout:9/851: readlink d2/d15/l52 0 2024-04-22T23:45:53.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/876: rename d9/d14/d22/d5c/de0 to d9/d14/d132 0 2024-04-22T23:45:53.132 INFO:tasks.workunit.client.0.smithi031.stdout:9/852: truncate d2/dd/d75/d4a/da0/fe9 592697 0 2024-04-22T23:45:53.135 INFO:tasks.workunit.client.0.smithi031.stdout:9/853: mkdir d2/dd/d75/d4a/da0/d117 0 2024-04-22T23:45:53.137 INFO:tasks.workunit.client.0.smithi031.stdout:7/877: creat d9/f133 x:0 0 0 2024-04-22T23:45:53.138 INFO:tasks.workunit.client.0.smithi031.stdout:7/878: chown d9/d15/d6d/d65/f90 4853292 1 2024-04-22T23:45:53.141 INFO:tasks.workunit.client.0.smithi031.stdout:9/854: truncate d2/dd/d75/d4a/fba 81566 0 2024-04-22T23:45:53.143 INFO:tasks.workunit.client.0.smithi031.stdout:6/984: dread d3/d2a/d3c/d40/f44 [0,4194304] 0 2024-04-22T23:45:53.144 INFO:tasks.workunit.client.0.smithi031.stdout:5/803: sync 2024-04-22T23:45:53.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/985: creat d3/d2a/f155 x:0 0 0 2024-04-22T23:45:53.149 INFO:tasks.workunit.client.0.smithi031.stdout:7/879: dwrite d9/d15/d6d/da9/f10f [0,4194304] 0 2024-04-22T23:45:53.150 INFO:tasks.workunit.client.0.smithi031.stdout:5/804: fdatasync db/d16/d2a/f8f 0 2024-04-22T23:45:53.153 INFO:tasks.workunit.client.0.smithi031.stdout:6/986: chown d3/d1f/f2f 573 1 2024-04-22T23:45:53.155 INFO:tasks.workunit.client.0.smithi031.stdout:5/805: symlink db/d8d/d9b/l10a 0 2024-04-22T23:45:53.156 INFO:tasks.workunit.client.0.smithi031.stdout:7/880: write d9/d14/f61 [4018100,29601] 0 2024-04-22T23:45:53.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/987: read - d3/da5/f154 zero size 2024-04-22T23:45:53.160 INFO:tasks.workunit.client.0.smithi031.stdout:8/792: dread d5/da/d12/d37/d34/d6f/f42 [0,4194304] 0 2024-04-22T23:45:53.160 INFO:tasks.workunit.client.0.smithi031.stdout:5/806: rmdir db/d16/d1d/d24/d58/d5d 39 2024-04-22T23:45:53.161 INFO:tasks.workunit.client.0.smithi031.stdout:7/881: creat d9/d15/d3f/ddc/f134 x:0 0 0 2024-04-22T23:45:53.161 INFO:tasks.workunit.client.0.smithi031.stdout:5/807: read - db/d16/ffd zero size 2024-04-22T23:45:53.162 INFO:tasks.workunit.client.0.smithi031.stdout:6/988: mknod d3/d77/d8d/d13e/c156 0 2024-04-22T23:45:53.163 INFO:tasks.workunit.client.0.smithi031.stdout:6/989: chown d3/d1f/d4c/d8e/l9c 72058280 1 2024-04-22T23:45:53.164 INFO:tasks.workunit.client.0.smithi031.stdout:8/793: symlink d5/da/d12/lf7 0 2024-04-22T23:45:53.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/855: dread d2/dc/d8b/d3d/d40/f5c [0,4194304] 0 2024-04-22T23:45:53.167 INFO:tasks.workunit.client.0.smithi031.stdout:7/882: link d9/d14/d22/d28/d7a/lc5 d9/d14/d1f/dc6/df4/l135 0 2024-04-22T23:45:53.168 INFO:tasks.workunit.client.0.smithi031.stdout:5/808: getdents db/d16/d2a/d36 0 2024-04-22T23:45:53.170 INFO:tasks.workunit.client.0.smithi031.stdout:5/809: write db/d16/d1d/d24/d32/fe3 [1113315,4733] 0 2024-04-22T23:45:53.170 INFO:tasks.workunit.client.0.smithi031.stdout:9/856: creat d2/dc/d3e/da1/f118 x:0 0 0 2024-04-22T23:45:53.173 INFO:tasks.workunit.client.0.smithi031.stdout:7/883: rmdir d9/d15/dcf 39 2024-04-22T23:45:53.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/794: dwrite d5/da/d12/d37/d34/d6f/d5b/f74 [0,4194304] 0 2024-04-22T23:45:53.174 INFO:tasks.workunit.client.0.smithi031.stdout:5/810: chown db/d16/d2a/d36/f38 5446974 1 2024-04-22T23:45:53.175 INFO:tasks.workunit.client.0.smithi031.stdout:8/795: stat d5/da/d12/d37/d34/fcf 0 2024-04-22T23:45:53.175 INFO:tasks.workunit.client.0.smithi031.stdout:6/990: rename d3/d35/d86/ca2 to d3/d2a/d3c/d40/d5b/c157 0 2024-04-22T23:45:53.177 INFO:tasks.workunit.client.0.smithi031.stdout:9/857: creat d2/d15/daf/f119 x:0 0 0 2024-04-22T23:45:53.181 INFO:tasks.workunit.client.0.smithi031.stdout:5/811: truncate db/d16/d1d/d24/d27/d9e/fae 2491303 0 2024-04-22T23:45:53.186 INFO:tasks.workunit.client.0.smithi031.stdout:6/991: mknod d3/d77/d139/c158 0 2024-04-22T23:45:53.186 INFO:tasks.workunit.client.0.smithi031.stdout:7/884: write d9/d15/d6d/dcc/fd0 [101209,86166] 0 2024-04-22T23:45:53.187 INFO:tasks.workunit.client.0.smithi031.stdout:6/992: fdatasync d3/d35/f94 0 2024-04-22T23:45:53.187 INFO:tasks.workunit.client.0.smithi031.stdout:5/812: creat db/d16/d2a/d36/d74/d82/f10b x:0 0 0 2024-04-22T23:45:53.188 INFO:tasks.workunit.client.0.smithi031.stdout:9/858: link d2/dd3/ce3 d2/dd/d75/d4a/da0/d117/c11a 0 2024-04-22T23:45:53.191 INFO:tasks.workunit.client.0.smithi031.stdout:5/813: mkdir db/d8d/df1/d10c 0 2024-04-22T23:45:53.193 INFO:tasks.workunit.client.0.smithi031.stdout:6/993: rename d3/d1f/ca1 to d3/d2a/d3c/db6/c159 0 2024-04-22T23:45:53.193 INFO:tasks.workunit.client.0.smithi031.stdout:9/859: dread d2/dc/f4c [0,4194304] 0 2024-04-22T23:45:53.195 INFO:tasks.workunit.client.0.smithi031.stdout:2/904: sync 2024-04-22T23:45:53.195 INFO:tasks.workunit.client.0.smithi031.stdout:4/953: sync 2024-04-22T23:45:53.195 INFO:tasks.workunit.client.0.smithi031.stdout:3/772: sync 2024-04-22T23:45:53.195 INFO:tasks.workunit.client.0.smithi031.stdout:0/877: sync 2024-04-22T23:45:53.197 INFO:tasks.workunit.client.0.smithi031.stdout:5/814: mkdir db/d16/d1d/d24/d10d 0 2024-04-22T23:45:53.198 INFO:tasks.workunit.client.0.smithi031.stdout:4/954: rmdir db/d137/d30 39 2024-04-22T23:45:53.200 INFO:tasks.workunit.client.0.smithi031.stdout:5/815: fsync f5 0 2024-04-22T23:45:53.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/885: write d9/d15/d3a/d41/f83 [2053923,91568] 0 2024-04-22T23:45:53.201 INFO:tasks.workunit.client.0.smithi031.stdout:9/860: symlink d2/dd/l11b 0 2024-04-22T23:45:53.203 INFO:tasks.workunit.client.0.smithi031.stdout:0/878: chown d0/d1/d2/db4/de4/ff8 5963 1 2024-04-22T23:45:53.204 INFO:tasks.workunit.client.0.smithi031.stdout:8/796: dwrite d5/da/f59 [0,4194304] 0 2024-04-22T23:45:53.207 INFO:tasks.workunit.client.0.smithi031.stdout:9/861: dread - d2/dd/d65/fbf zero size 2024-04-22T23:45:53.207 INFO:tasks.workunit.client.0.smithi031.stdout:5/816: write db/d16/d1d/d24/d58/f4f [1067330,3922] 0 2024-04-22T23:45:53.211 INFO:tasks.workunit.client.0.smithi031.stdout:6/994: link d3/d2a/d85/l138 d3/d1f/d4c/d8e/d13c/l15a 0 2024-04-22T23:45:53.211 INFO:tasks.workunit.client.0.smithi031.stdout:8/797: read d5/da/d12/d2e/d7e/fb4 [237871,33393] 0 2024-04-22T23:45:53.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/955: dread - db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/ffb zero size 2024-04-22T23:45:53.214 INFO:tasks.workunit.client.0.smithi031.stdout:3/773: truncate d4/d6/d11/d25/d44/d7f/f8f 557740 0 2024-04-22T23:45:53.214 INFO:tasks.workunit.client.0.smithi031.stdout:7/886: truncate d9/d15/f99 189995 0 2024-04-22T23:45:53.216 INFO:tasks.workunit.client.0.smithi031.stdout:9/862: fsync d2/dc/d3e/f41 0 2024-04-22T23:45:53.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/817: mknod db/d16/d1d/d24/d32/d41/c10e 0 2024-04-22T23:45:53.218 INFO:tasks.workunit.client.0.smithi031.stdout:6/995: symlink d3/d77/d139/l15b 0 2024-04-22T23:45:53.220 INFO:tasks.workunit.client.0.smithi031.stdout:3/774: chown d4/d54/d6d/l8a 325324 1 2024-04-22T23:45:53.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/798: fdatasync d5/da/d12/d37/d34/d6f/f4b 0 2024-04-22T23:45:53.223 INFO:tasks.workunit.client.0.smithi031.stdout:6/996: chown d3/d35/c81 2186 1 2024-04-22T23:45:53.224 INFO:tasks.workunit.client.0.smithi031.stdout:7/887: mknod d9/d14/d1f/dc6/c136 0 2024-04-22T23:45:53.224 INFO:tasks.workunit.client.0.smithi031.stdout:2/905: dwrite d6/dc/d48/d4b/f64 [0,4194304] 0 2024-04-22T23:45:53.225 INFO:tasks.workunit.client.0.smithi031.stdout:4/956: symlink db/l14a 0 2024-04-22T23:45:53.226 INFO:tasks.workunit.client.0.smithi031.stdout:0/879: dwrite d0/d13/d18/d1e/fef [0,4194304] 0 2024-04-22T23:45:53.227 INFO:tasks.workunit.client.0.smithi031.stdout:6/997: chown d3/d2a/d3c/d40/d4d 264871127 1 2024-04-22T23:45:53.230 INFO:tasks.workunit.client.0.smithi031.stdout:3/775: mknod d4/d6/d11/de3/c109 0 2024-04-22T23:45:53.230 INFO:tasks.workunit.client.0.smithi031.stdout:0/880: readlink d0/d13/d18/d1e/d25/d8e/dac/dda/l5d 0 2024-04-22T23:45:53.232 INFO:tasks.workunit.client.0.smithi031.stdout:9/863: rename d2/dc/d8b/d3d/d40/da4/db1/dc5/d10a to d2/dd/d75/d80/df8/d11c 0 2024-04-22T23:45:53.233 INFO:tasks.workunit.client.0.smithi031.stdout:2/906: stat d6/dc/d48/d4b/d6b/de8/da4/l111 0 2024-04-22T23:45:53.235 INFO:tasks.workunit.client.0.smithi031.stdout:9/864: dread d2/dd/d75/f5e [0,4194304] 0 2024-04-22T23:45:53.235 INFO:tasks.workunit.client.0.smithi031.stdout:3/776: dread d4/d6/d11/dc0/fcf [0,4194304] 0 2024-04-22T23:45:53.236 INFO:tasks.workunit.client.0.smithi031.stdout:7/888: mknod d9/d14/d1f/d45/da6/daa/d10b/c137 0 2024-04-22T23:45:53.239 INFO:tasks.workunit.client.0.smithi031.stdout:5/818: truncate db/d16/d1d/d24/d32/f39 2069107 0 2024-04-22T23:45:53.240 INFO:tasks.workunit.client.0.smithi031.stdout:8/799: symlink d5/da/d12/d2e/db0/dc3/de0/lf8 0 2024-04-22T23:45:53.241 INFO:tasks.workunit.client.0.smithi031.stdout:4/957: truncate db/d14/d2d/d49/d59/d6a/d63/d8b/ff7 920436 0 2024-04-22T23:45:53.243 INFO:tasks.workunit.client.0.smithi031.stdout:8/800: readlink d5/da/d83/d8d/lae 0 2024-04-22T23:45:53.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/882: sync 2024-04-22T23:45:53.245 INFO:tasks.workunit.client.0.smithi031.stdout:3/777: dwrite d4/d6/d11/d25/d44/d7f/d5f/f63 [0,4194304] 0 2024-04-22T23:45:53.245 INFO:tasks.workunit.client.0.smithi031.stdout:7/889: mknod d9/d14/dd6/c138 0 2024-04-22T23:45:53.246 INFO:tasks.workunit.client.0.smithi031.stdout:2/907: write d6/dc/d9a/d6c/db2/df0/f54 [3436497,80133] 0 2024-04-22T23:45:53.247 INFO:tasks.workunit.client.0.smithi031.stdout:6/998: mknod d3/d35/d86/c15c 0 2024-04-22T23:45:53.248 INFO:tasks.workunit.client.0.smithi031.stdout:4/958: symlink db/d137/da1/d135/l14b 0 2024-04-22T23:45:53.250 INFO:tasks.workunit.client.0.smithi031.stdout:8/801: mkdir d5/da/d12/d2e/d4d/df9 0 2024-04-22T23:45:53.253 INFO:tasks.workunit.client.0.smithi031.stdout:0/881: truncate d0/d13/d18/d1e/d25/d8e/da7/f55 1443544 0 2024-04-22T23:45:53.255 INFO:tasks.workunit.client.0.smithi031.stdout:2/908: chown d6/dc/d9a/d6c/db2/df0/ld9 0 1 2024-04-22T23:45:53.256 INFO:tasks.workunit.client.0.smithi031.stdout:7/890: rename d9/d14/d22/d5c/d5f/f98 to d9/d14/d1f/dc6/d103/f139 0 2024-04-22T23:45:53.258 INFO:tasks.workunit.client.0.smithi031.stdout:8/802: creat d5/da/d12/d37/d34/d6f/d5b/ffa x:0 0 0 2024-04-22T23:45:53.259 INFO:tasks.workunit.client.0.smithi031.stdout:3/778: truncate d4/d54/d2c/d89/d2e/d45/f60 37195 0 2024-04-22T23:45:53.263 INFO:tasks.workunit.client.0.smithi031.stdout:8/803: chown d5/da/d12/d95 30 1 2024-04-22T23:45:53.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/865: dread d2/dd/d75/d4a/f36 [0,4194304] 0 2024-04-22T23:45:53.264 INFO:tasks.workunit.client.0.smithi031.stdout:7/891: creat d9/d15/d6d/da9/f13a x:0 0 0 2024-04-22T23:45:53.267 INFO:tasks.workunit.client.0.smithi031.stdout:3/779: chown d4/d54/fe8 22 1 2024-04-22T23:45:53.270 INFO:tasks.workunit.client.0.smithi031.stdout:7/892: fdatasync d9/d15/fe1 0 2024-04-22T23:45:53.270 INFO:tasks.workunit.client.0.smithi031.stdout:1/883: dwrite d1/d3f/d54/dc6/d110/ff9 [0,4194304] 0 2024-04-22T23:45:53.270 INFO:tasks.workunit.client.0.smithi031.stdout:0/882: rename d0/db5/cc5 to d0/d13/d18/d1e/c11c 0 2024-04-22T23:45:53.273 INFO:tasks.workunit.client.0.smithi031.stdout:6/999: dwrite d3/f74 [0,4194304] 0 2024-04-22T23:45:53.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/959: dwrite db/d14/d1d/d7b/d7f/fe8 [0,4194304] 0 2024-04-22T23:45:53.274 INFO:tasks.workunit.client.0.smithi031.stdout:8/804: mkdir d5/da/d12/d2e/db0/dfb 0 2024-04-22T23:45:53.274 INFO:tasks.workunit.client.0.smithi031.stdout:2/909: dread d6/dce/fda [0,4194304] 0 2024-04-22T23:45:53.281 INFO:tasks.workunit.client.0.smithi031.stdout:1/884: mknod d1/d35/d8d/d102/dc1/c12a 0 2024-04-22T23:45:53.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/893: mkdir d9/d15/d3a/d41/d12d/d13b 0 2024-04-22T23:45:53.285 INFO:tasks.workunit.client.0.smithi031.stdout:8/805: rmdir d5/da/df/d33/dd4 39 2024-04-22T23:45:53.286 INFO:tasks.workunit.client.0.smithi031.stdout:7/894: stat d9/d15/d3a/d41/fa3 0 2024-04-22T23:45:53.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/960: truncate db/d14/d1d/fd7 539018 0 2024-04-22T23:45:53.289 INFO:tasks.workunit.client.0.smithi031.stdout:5/819: dread db/d16/d2a/d36/d74/f87 [0,4194304] 0 2024-04-22T23:45:53.290 INFO:tasks.workunit.client.0.smithi031.stdout:0/883: dwrite d0/d13/d18/d1e/f30 [0,4194304] 0 2024-04-22T23:45:53.291 INFO:tasks.workunit.client.0.smithi031.stdout:3/780: truncate d4/d6/d11/d25/d44/d7f/f7d 1534711 0 2024-04-22T23:45:53.292 INFO:tasks.workunit.client.0.smithi031.stdout:9/866: link d2/dc/d8b/d3d/d40/lbd d2/d15/l11d 0 2024-04-22T23:45:53.294 INFO:tasks.workunit.client.0.smithi031.stdout:8/806: creat d5/da/d12/d2e/db0/ffc x:0 0 0 2024-04-22T23:45:53.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/807: write d5/da/d12/fb6 [1136536,48502] 0 2024-04-22T23:45:53.298 INFO:tasks.workunit.client.0.smithi031.stdout:7/895: dread d9/d15/d6d/d65/f72 [0,4194304] 0 2024-04-22T23:45:53.299 INFO:tasks.workunit.client.0.smithi031.stdout:1/885: fsync d1/d35/d36/fbd 0 2024-04-22T23:45:53.300 INFO:tasks.workunit.client.0.smithi031.stdout:8/808: write d5/da/d12/d1d/d38/fc6 [2188324,80620] 0 2024-04-22T23:45:53.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/896: write d9/d15/f11a [47776,68600] 0 2024-04-22T23:45:53.300 INFO:tasks.workunit.client.0.smithi031.stdout:2/910: mkdir d6/dc/d17/d124 0 2024-04-22T23:45:53.301 INFO:tasks.workunit.client.0.smithi031.stdout:1/886: stat d1/db/ffa 0 2024-04-22T23:45:53.302 INFO:tasks.workunit.client.0.smithi031.stdout:1/887: stat d1/de 0 2024-04-22T23:45:53.303 INFO:tasks.workunit.client.0.smithi031.stdout:0/884: rmdir d0/d13/d18/d1e/d25/d8e/da7 39 2024-04-22T23:45:53.303 INFO:tasks.workunit.client.0.smithi031.stdout:2/911: readlink d6/dc/d48/d4b/d6b/l75 0 2024-04-22T23:45:53.306 INFO:tasks.workunit.client.0.smithi031.stdout:2/912: fdatasync d6/dc/d49/d83/d85/da9/fad 0 2024-04-22T23:45:53.308 INFO:tasks.workunit.client.0.smithi031.stdout:4/961: truncate db/d14/fad 1063447 0 2024-04-22T23:45:53.309 INFO:tasks.workunit.client.0.smithi031.stdout:8/809: dwrite d5/da/d12/f1c [0,4194304] 0 2024-04-22T23:45:53.309 INFO:tasks.workunit.client.0.smithi031.stdout:4/962: dread - db/d137/da1/da7/f145 zero size 2024-04-22T23:45:53.311 INFO:tasks.workunit.client.0.smithi031.stdout:7/897: creat d9/d14/d1f/d45/da6/daa/d10b/f13c x:0 0 0 2024-04-22T23:45:53.311 INFO:tasks.workunit.client.0.smithi031.stdout:4/963: chown db/d14/d1d/l75 38141 1 2024-04-22T23:45:53.312 INFO:tasks.workunit.client.0.smithi031.stdout:1/888: fsync d1/db/d16/fd1 0 2024-04-22T23:45:53.313 INFO:tasks.workunit.client.0.smithi031.stdout:0/885: truncate d0/d1/d2/fa 9116654 0 2024-04-22T23:45:53.314 INFO:tasks.workunit.client.0.smithi031.stdout:3/781: truncate d4/d6/d11/d3e/f5a 188704 0 2024-04-22T23:45:53.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/810: write d5/d48/d61/d6b/fa1 [2988908,79284] 0 2024-04-22T23:45:53.317 INFO:tasks.workunit.client.0.smithi031.stdout:4/964: creat db/d137/da1/f14c x:0 0 0 2024-04-22T23:45:53.320 INFO:tasks.workunit.client.0.smithi031.stdout:8/811: write d5/d48/d61/d6b/fd2 [342448,93764] 0 2024-04-22T23:45:53.320 INFO:tasks.workunit.client.0.smithi031.stdout:0/886: read - d0/d1/d2/d36/d11/d1a/d56/fa2 zero size 2024-04-22T23:45:53.320 INFO:tasks.workunit.client.0.smithi031.stdout:7/898: symlink d9/d15/d3a/d41/d12d/l13d 0 2024-04-22T23:45:53.322 INFO:tasks.workunit.client.0.smithi031.stdout:3/782: creat d4/d6/d11/dda/f10a x:0 0 0 2024-04-22T23:45:53.325 INFO:tasks.workunit.client.0.smithi031.stdout:4/965: creat db/d14/d1d/d7b/dc1/f14d x:0 0 0 2024-04-22T23:45:53.327 INFO:tasks.workunit.client.0.smithi031.stdout:0/887: mknod d0/d1/d2/d17/d102/c11d 0 2024-04-22T23:45:53.330 INFO:tasks.workunit.client.0.smithi031.stdout:3/783: rename d4/d54/d2c/d89/d2e/d45/dde to d4/d54/d88/da2/de0/d10b 0 2024-04-22T23:45:53.331 INFO:tasks.workunit.client.0.smithi031.stdout:4/966: creat db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/d132/f14e x:0 0 0 2024-04-22T23:45:53.332 INFO:tasks.workunit.client.0.smithi031.stdout:0/888: mknod d0/d13/d18/d1e/d25/dea/d105/c11e 0 2024-04-22T23:45:53.333 INFO:tasks.workunit.client.0.smithi031.stdout:4/967: write db/d14/d2d/d49/d59/d6a/d63/d86/f12e [596303,42975] 0 2024-04-22T23:45:53.334 INFO:tasks.workunit.client.0.smithi031.stdout:7/899: creat d9/d15/f13e x:0 0 0 2024-04-22T23:45:53.336 INFO:tasks.workunit.client.0.smithi031.stdout:9/867: write d2/dd/d75/d64/f5f [166580,123885] 0 2024-04-22T23:45:53.337 INFO:tasks.workunit.client.0.smithi031.stdout:4/968: fsync db/d14/f6e 0 2024-04-22T23:45:53.337 INFO:tasks.workunit.client.0.smithi031.stdout:1/889: dread d1/de/d1e/f64 [0,4194304] 0 2024-04-22T23:45:53.341 INFO:tasks.workunit.client.0.smithi031.stdout:1/890: write d1/d3f/d54/f11a [197638,10248] 0 2024-04-22T23:45:53.341 INFO:tasks.workunit.client.0.smithi031.stdout:7/900: rmdir d9/d14/d1f/d45 39 2024-04-22T23:45:53.341 INFO:tasks.workunit.client.0.smithi031.stdout:4/969: chown db/d14/d2d/d49/d59/d6a/fee 3528539 1 2024-04-22T23:45:53.341 INFO:tasks.workunit.client.0.smithi031.stdout:0/889: dread d0/d1/d2/d36/d11/d1c/d54/f72 [0,4194304] 0 2024-04-22T23:45:53.342 INFO:tasks.workunit.client.0.smithi031.stdout:1/891: readlink d1/d3f/d54/l6b 0 2024-04-22T23:45:53.342 INFO:tasks.workunit.client.0.smithi031.stdout:5/820: write db/d16/fa2 [771258,70632] 0 2024-04-22T23:45:53.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/901: mknod d9/d14/d22/d5c/d5f/c13f 0 2024-04-22T23:45:53.345 INFO:tasks.workunit.client.0.smithi031.stdout:9/868: creat d2/f11e x:0 0 0 2024-04-22T23:45:53.347 INFO:tasks.workunit.client.0.smithi031.stdout:4/970: chown db/d14/d1d/d7b/d7f/l9d 120 1 2024-04-22T23:45:53.348 INFO:tasks.workunit.client.0.smithi031.stdout:0/890: dread d0/d1/d2/db4/ff4 [0,4194304] 0 2024-04-22T23:45:53.351 INFO:tasks.workunit.client.0.smithi031.stdout:7/902: mknod d9/d14/d22/dbe/c140 0 2024-04-22T23:45:53.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/869: symlink d2/dd/d75/d80/df8/l11f 0 2024-04-22T23:45:53.354 INFO:tasks.workunit.client.0.smithi031.stdout:7/903: write d9/d14/d22/d5c/d5f/fd7 [1173269,61865] 0 2024-04-22T23:45:53.356 INFO:tasks.workunit.client.0.smithi031.stdout:0/891: dwrite d0/d13/d18/f48 [0,4194304] 0 2024-04-22T23:45:53.356 INFO:tasks.workunit.client.0.smithi031.stdout:9/870: chown d2/dd/d75/d4a/da0/cb9 0 1 2024-04-22T23:45:53.363 INFO:tasks.workunit.client.0.smithi031.stdout:9/871: symlink d2/d15/l120 0 2024-04-22T23:45:53.367 INFO:tasks.workunit.client.0.smithi031.stdout:1/892: getdents d1/d35/d8d/d102 0 2024-04-22T23:45:53.368 INFO:tasks.workunit.client.0.smithi031.stdout:3/784: truncate d4/d54/d2c/d89/f62 820439 0 2024-04-22T23:45:53.368 INFO:tasks.workunit.client.0.smithi031.stdout:0/892: mkdir d0/d13/d11f 0 2024-04-22T23:45:53.369 INFO:tasks.workunit.client.0.smithi031.stdout:0/893: chown d0/d13/fe3 107517088 1 2024-04-22T23:45:53.371 INFO:tasks.workunit.client.0.smithi031.stdout:4/971: getdents db/d14/d1d/d7b 0 2024-04-22T23:45:53.376 INFO:tasks.workunit.client.0.smithi031.stdout:3/785: rename d4/d6/d11/ccb to d4/d54/d88/da2/de0/c10c 0 2024-04-22T23:45:53.376 INFO:tasks.workunit.client.0.smithi031.stdout:5/821: dwrite db/d16/d1d/d24/d58/ffc [0,4194304] 0 2024-04-22T23:45:53.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/894: creat d0/d1/d2/d36/d11/d1a/dfc/dad/f120 x:0 0 0 2024-04-22T23:45:53.381 INFO:tasks.workunit.client.0.smithi031.stdout:1/893: symlink d1/d99/d7d/d118/l12b 0 2024-04-22T23:45:53.383 INFO:tasks.workunit.client.0.smithi031.stdout:7/904: write d9/f37 [1294257,64449] 0 2024-04-22T23:45:53.384 INFO:tasks.workunit.client.0.smithi031.stdout:4/972: mkdir db/d14/d2d/d49/d59/d6a/d63/d107/d40/d14f 0 2024-04-22T23:45:53.387 INFO:tasks.workunit.client.0.smithi031.stdout:7/905: fsync d9/d15/d6d/da9/f10f 0 2024-04-22T23:45:53.393 INFO:tasks.workunit.client.0.smithi031.stdout:7/906: truncate d9/d14/f18 1496870 0 2024-04-22T23:45:53.397 INFO:tasks.workunit.client.0.smithi031.stdout:5/822: write db/d16/d1d/d24/d27/d9e/fae [159770,14076] 0 2024-04-22T23:45:53.397 INFO:tasks.workunit.client.0.smithi031.stdout:9/872: dwrite d2/dd/d75/f5e [0,4194304] 0 2024-04-22T23:45:53.403 INFO:tasks.workunit.client.0.smithi031.stdout:3/786: dread d4/d54/fdc [0,4194304] 0 2024-04-22T23:45:53.405 INFO:tasks.workunit.client.0.smithi031.stdout:7/907: truncate d9/d14/d1f/fc2 5026926 0 2024-04-22T23:45:53.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/895: dwrite d0/f86 [0,4194304] 0 2024-04-22T23:45:53.410 INFO:tasks.workunit.client.0.smithi031.stdout:9/873: symlink d2/dd/d75/d80/df8/l121 0 2024-04-22T23:45:53.410 INFO:tasks.workunit.client.0.smithi031.stdout:4/973: dread db/d14/d1d/d7b/d7f/ff0 [0,4194304] 0 2024-04-22T23:45:53.411 INFO:tasks.workunit.client.0.smithi031.stdout:0/896: stat d0/d1/d2/d17/d102 0 2024-04-22T23:45:53.414 INFO:tasks.workunit.client.0.smithi031.stdout:7/908: creat d9/d14/d22/dbe/f141 x:0 0 0 2024-04-22T23:45:53.415 INFO:tasks.workunit.client.0.smithi031.stdout:5/823: mknod db/d16/d1d/d24/d58/d5d/dd6/c10f 0 2024-04-22T23:45:53.417 INFO:tasks.workunit.client.0.smithi031.stdout:1/894: dread d1/d99/d63/f6d [0,4194304] 0 2024-04-22T23:45:53.418 INFO:tasks.workunit.client.0.smithi031.stdout:5/824: chown db/d16/d1d/d24/d32/ff9 18978311 1 2024-04-22T23:45:53.419 INFO:tasks.workunit.client.0.smithi031.stdout:7/909: unlink d9/d14/dd6/l115 0 2024-04-22T23:45:53.420 INFO:tasks.workunit.client.0.smithi031.stdout:4/974: dwrite db/d137/d47/d80/fcf [0,4194304] 0 2024-04-22T23:45:53.421 INFO:tasks.workunit.client.0.smithi031.stdout:0/897: symlink d0/d13/d18/d1e/d25/d8e/l121 0 2024-04-22T23:45:53.422 INFO:tasks.workunit.client.0.smithi031.stdout:9/874: rmdir d2/dc/d6d/d105 0 2024-04-22T23:45:53.423 INFO:tasks.workunit.client.0.smithi031.stdout:3/787: rename d4/d6/d4c/d8c/la1 to d4/d6/d11/l10d 0 2024-04-22T23:45:53.425 INFO:tasks.workunit.client.0.smithi031.stdout:5/825: creat db/d16/d2a/d36/d74/d82/d86/f110 x:0 0 0 2024-04-22T23:45:53.425 INFO:tasks.workunit.client.0.smithi031.stdout:0/898: fdatasync d0/d13/f67 0 2024-04-22T23:45:53.426 INFO:tasks.workunit.client.0.smithi031.stdout:4/975: fsync db/d14/d2d/d49/d59/d6a/d63/d86/fc5 0 2024-04-22T23:45:53.427 INFO:tasks.workunit.client.0.smithi031.stdout:9/875: mkdir d2/dd/d75/df7/d122 0 2024-04-22T23:45:53.428 INFO:tasks.workunit.client.0.smithi031.stdout:7/910: mknod d9/d15/d3a/d41/d12d/d13b/c142 0 2024-04-22T23:45:53.428 INFO:tasks.workunit.client.0.smithi031.stdout:4/976: readlink db/d137/d47/lbd 0 2024-04-22T23:45:53.429 INFO:tasks.workunit.client.0.smithi031.stdout:3/788: chown d4/d54/d2c/d89/lc9 9514 1 2024-04-22T23:45:53.430 INFO:tasks.workunit.client.0.smithi031.stdout:5/826: chown db/d16/l97 766109 1 2024-04-22T23:45:53.430 INFO:tasks.workunit.client.0.smithi031.stdout:7/911: stat d9/d14/d22/d5c/d5f/fd7 0 2024-04-22T23:45:53.432 INFO:tasks.workunit.client.0.smithi031.stdout:1/895: getdents d1/d3f/d54/dc6/d110 0 2024-04-22T23:45:53.433 INFO:tasks.workunit.client.0.smithi031.stdout:9/876: creat d2/d15/daf/f123 x:0 0 0 2024-04-22T23:45:53.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/899: creat d0/d1/d2/d36/d11/d1c/d54/f122 x:0 0 0 2024-04-22T23:45:53.435 INFO:tasks.workunit.client.0.smithi031.stdout:4/977: creat db/d137/d48/f150 x:0 0 0 2024-04-22T23:45:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:3/789: symlink d4/d6/d4c/d8c/l10e 0 2024-04-22T23:45:53.437 INFO:tasks.workunit.client.0.smithi031.stdout:7/912: fdatasync d9/d14/d1f/d45/fa2 0 2024-04-22T23:45:53.442 INFO:tasks.workunit.client.0.smithi031.stdout:5/827: creat db/d16/d2a/d59/d90/ddd/f111 x:0 0 0 2024-04-22T23:45:53.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:53 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:53.445 INFO:tasks.workunit.client.0.smithi031.stdout:1/896: dread d1/d35/d8d/d102/f50 [0,4194304] 0 2024-04-22T23:45:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:1/897: write d1/d3f/d54/dba/d28/f2a [2450405,45765] 0 2024-04-22T23:45:53.447 INFO:tasks.workunit.client.0.smithi031.stdout:0/900: mkdir d0/d13/d18/d1e/d25/d8e/da7/dc9/d123 0 2024-04-22T23:45:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:4/978: mkdir db/d14/d2d/d49/d59/d83/d9c/d151 0 2024-04-22T23:45:53.449 INFO:tasks.workunit.client.0.smithi031.stdout:1/898: chown d1/d35/d8d/d102/f50 2882386 1 2024-04-22T23:45:53.451 INFO:tasks.workunit.client.0.smithi031.stdout:7/913: getdents d9/d14/dd6/d129 0 2024-04-22T23:45:53.455 INFO:tasks.workunit.client.0.smithi031.stdout:3/790: dwrite d4/d54/d2c/d89/f1b [0,4194304] 0 2024-04-22T23:45:53.456 INFO:tasks.workunit.client.0.smithi031.stdout:1/899: unlink d1/db/d16/d1f/d52/lf0 0 2024-04-22T23:45:53.459 INFO:tasks.workunit.client.0.smithi031.stdout:7/914: dread d9/d15/d6d/d65/f72 [0,4194304] 0 2024-04-22T23:45:53.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:53 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:53.462 INFO:tasks.workunit.client.0.smithi031.stdout:1/900: write d1/db/d16/da0/f115 [880102,70089] 0 2024-04-22T23:45:53.463 INFO:tasks.workunit.client.0.smithi031.stdout:7/915: write d9/d15/f60 [1027232,63503] 0 2024-04-22T23:45:53.464 INFO:tasks.workunit.client.0.smithi031.stdout:4/979: dwrite db/d14/d1d/fdc [0,4194304] 0 2024-04-22T23:45:53.465 INFO:tasks.workunit.client.0.smithi031.stdout:7/916: stat d9/d15/f109 0 2024-04-22T23:45:53.466 INFO:tasks.workunit.client.0.smithi031.stdout:4/980: dread - db/d137/d47/d80/f13b zero size 2024-04-22T23:45:53.470 INFO:tasks.workunit.client.0.smithi031.stdout:7/917: dread f3 [0,4194304] 0 2024-04-22T23:45:53.477 INFO:tasks.workunit.client.0.smithi031.stdout:4/981: dwrite db/d14/d2d/d49/d59/d6a/d63/fa5 [0,4194304] 0 2024-04-22T23:45:53.477 INFO:tasks.workunit.client.0.smithi031.stdout:7/918: dread f3 [4194304,4194304] 0 2024-04-22T23:45:53.480 INFO:tasks.workunit.client.0.smithi031.stdout:3/791: dread d4/d49/f94 [0,4194304] 0 2024-04-22T23:45:53.480 INFO:tasks.workunit.client.0.smithi031.stdout:4/982: symlink db/d14/d1d/l152 0 2024-04-22T23:45:53.486 INFO:tasks.workunit.client.0.smithi031.stdout:0/901: truncate d0/db5/df6/ffb 4788695 0 2024-04-22T23:45:53.486 INFO:tasks.workunit.client.0.smithi031.stdout:4/983: creat db/d14/d1d/d7b/f153 x:0 0 0 2024-04-22T23:45:53.486 INFO:tasks.workunit.client.0.smithi031.stdout:5/828: dread db/d16/d1d/d24/ddf/fe9 [0,4194304] 0 2024-04-22T23:45:53.491 INFO:tasks.workunit.client.0.smithi031.stdout:4/984: mknod db/d14/d2d/d49/d59/d6a/d63/d107/d40/d43/dfa/d132/c154 0 2024-04-22T23:45:53.491 INFO:tasks.workunit.client.0.smithi031.stdout:3/792: link d4/d54/d6d/d98/cb5 d4/d6/d11/d25/d44/c10f 0 2024-04-22T23:45:53.497 INFO:tasks.workunit.client.0.smithi031.stdout:5/829: dwrite db/f23 [0,4194304] 0 2024-04-22T23:45:53.498 INFO:tasks.workunit.client.0.smithi031.stdout:1/901: dread d1/d3f/d54/dba/d28/fa8 [0,4194304] 0 2024-04-22T23:45:53.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/793: creat d4/d6/d11/d25/d44/d7f/d5f/dee/f110 x:0 0 0 2024-04-22T23:45:53.499 INFO:tasks.workunit.client.0.smithi031.stdout:4/985: truncate db/d14/d2d/d49/d59/d6a/fbb 395069 0 2024-04-22T23:45:53.500 INFO:tasks.workunit.client.0.smithi031.stdout:5/830: symlink db/d16/d1d/d24/d32/d41/l112 0 2024-04-22T23:45:53.504 INFO:tasks.workunit.client.0.smithi031.stdout:3/794: mknod d4/d54/d88/da2/de0/c111 0 2024-04-22T23:45:53.506 INFO:tasks.workunit.client.0.smithi031.stdout:4/986: dread - db/d14/d2d/d49/d59/d6a/f64 zero size 2024-04-22T23:45:53.508 INFO:tasks.workunit.client.0.smithi031.stdout:5/831: write db/d16/d2a/d36/d74/d82/d86/ffe [847480,74771] 0 2024-04-22T23:45:53.509 INFO:tasks.workunit.client.0.smithi031.stdout:3/795: mknod d4/d6/c112 0 2024-04-22T23:45:53.510 INFO:tasks.workunit.client.0.smithi031.stdout:0/902: dwrite d0/fbe [4194304,4194304] 0 2024-04-22T23:45:53.512 INFO:tasks.workunit.client.0.smithi031.stdout:7/919: dread d9/d14/d22/f78 [0,4194304] 0 2024-04-22T23:45:53.514 INFO:tasks.workunit.client.0.smithi031.stdout:1/902: dwrite d1/de/d1e/d94/d125/ff6 [0,4194304] 0 2024-04-22T23:45:53.515 INFO:tasks.workunit.client.0.smithi031.stdout:1/903: read - d1/f114 zero size 2024-04-22T23:45:53.516 INFO:tasks.workunit.client.0.smithi031.stdout:3/796: symlink d4/d54/d85/l113 0 2024-04-22T23:45:53.518 INFO:tasks.workunit.client.0.smithi031.stdout:4/987: mknod db/d14/d2d/d49/d59/d6a/d63/d107/d129/dc6/dd2/c155 0 2024-04-22T23:45:53.522 INFO:tasks.workunit.client.0.smithi031.stdout:0/903: rename d0/d1/d2/db4/ff4 to d0/d13/d18/d1e/dd3/f124 0 2024-04-22T23:45:53.524 INFO:tasks.workunit.client.0.smithi031.stdout:0/904: write d0/d1/d2/f23 [1459049,129810] 0 2024-04-22T23:45:53.524 INFO:tasks.workunit.client.0.smithi031.stdout:4/988: dwrite db/d14/d2d/d49/d59/d6a/d63/dc9/f104 [0,4194304] 0 2024-04-22T23:45:53.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/904: link d1/de/d1e/d34/d41/f42 d1/de/d1e/d34/f12c 0 2024-04-22T23:45:53.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/905: mknod d0/d13/d18/d1e/d25/d8e/dac/dda/c125 0 2024-04-22T23:45:53.532 INFO:tasks.workunit.client.0.smithi031.stdout:4/989: creat db/d14/d2d/d49/d59/d83/d9c/d151/f156 x:0 0 0 2024-04-22T23:45:53.534 INFO:tasks.workunit.client.0.smithi031.stdout:1/905: fsync d1/d35/d36/d108/d4e/dd9/f10b 0 2024-04-22T23:45:53.536 INFO:tasks.workunit.client.0.smithi031.stdout:1/906: dread d1/d99/d63/f6d [4194304,4194304] 0 2024-04-22T23:45:53.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/906: truncate d0/d13/d18/d1e/d25/d8e/fa8 494606 0 2024-04-22T23:45:53.538 INFO:tasks.workunit.client.0.smithi031.stdout:0/907: write d0/d13/d18/f6d [2319290,106621] 0 2024-04-22T23:45:53.540 INFO:tasks.workunit.client.0.smithi031.stdout:7/920: write d9/d14/d22/d28/d7a/f7d [1990776,48364] 0 2024-04-22T23:45:53.542 INFO:tasks.workunit.client.0.smithi031.stdout:3/797: write d4/d6/d11/d25/d44/f8e [851843,33606] 0 2024-04-22T23:45:53.544 INFO:tasks.workunit.client.0.smithi031.stdout:4/990: getdents db/d14/d2d/d49/d59/d8a/d89 0 2024-04-22T23:45:53.548 INFO:tasks.workunit.client.0.smithi031.stdout:3/798: mknod d4/d54/d88/da2/c114 0 2024-04-22T23:45:53.548 INFO:tasks.workunit.client.0.smithi031.stdout:1/907: dwrite d1/d35/d8d/d102/f113 [0,4194304] 0 2024-04-22T23:45:53.549 INFO:tasks.workunit.client.0.smithi031.stdout:4/991: mknod db/d137/da1/c157 0 2024-04-22T23:45:53.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/921: getdents d9 0 2024-04-22T23:45:53.551 INFO:tasks.workunit.client.0.smithi031.stdout:0/908: dwrite d0/d1/d2/d36/f1d [0,4194304] 0 2024-04-22T23:45:53.552 INFO:tasks.workunit.client.0.smithi031.stdout:5/832: dread db/d8d/d9b/fcc [0,4194304] 0 2024-04-22T23:45:53.552 INFO:tasks.workunit.client.0.smithi031.stdout:4/992: truncate db/f134 962307 0 2024-04-22T23:45:53.554 INFO:tasks.workunit.client.0.smithi031.stdout:3/799: fsync d4/d54/f78 0 2024-04-22T23:45:53.554 INFO:tasks.workunit.client.0.smithi031.stdout:1/908: write d1/de/d1e/d34/d66/f121 [65434,105085] 0 2024-04-22T23:45:53.556 INFO:tasks.workunit.client.0.smithi031.stdout:5/833: chown db/d16/d1d/d24/d58/d5d/fbd 249873024 1 2024-04-22T23:45:53.558 INFO:tasks.workunit.client.0.smithi031.stdout:7/922: mkdir d9/d15/d3a/d41/d12d/d13b/d143 0 2024-04-22T23:45:53.559 INFO:tasks.workunit.client.0.smithi031.stdout:1/909: chown d1/d3f/d54/dc6/d110/c11b 28 1 2024-04-22T23:45:53.562 INFO:tasks.workunit.client.0.smithi031.stdout:3/800: mkdir d4/d6/d11/ddf/d115 0 2024-04-22T23:45:53.569 INFO:tasks.workunit.client.0.smithi031.stdout:1/910: rename d1/d35/l48 to d1/d35/dbb/l12d 0 2024-04-22T23:45:53.571 INFO:tasks.workunit.client.0.smithi031.stdout:7/923: dread d9/d14/d1f/d45/f12a [0,4194304] 0 2024-04-22T23:45:53.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/911: creat d1/d3f/d54/f12e x:0 0 0 2024-04-22T23:45:53.572 INFO:tasks.workunit.client.0.smithi031.stdout:1/912: fdatasync d1/d35/d8d/d102/d81/f122 0 2024-04-22T23:45:53.573 INFO:tasks.workunit.client.0.smithi031.stdout:3/801: link d4/d54/d2c/d89/d2e/d45/f60 d4/f116 0 2024-04-22T23:45:53.574 INFO:tasks.workunit.client.0.smithi031.stdout:3/802: stat d4/d54/d56/f87 0 2024-04-22T23:45:53.577 INFO:tasks.workunit.client.0.smithi031.stdout:7/924: getdents d9/d14/d1f/d45/da6/daa/d10b 0 2024-04-22T23:45:53.578 INFO:tasks.workunit.client.0.smithi031.stdout:3/803: truncate d4/d6/f7 498870 0 2024-04-22T23:45:53.581 INFO:tasks.workunit.client.0.smithi031.stdout:7/925: creat d9/d14/d1f/dc6/d103/f144 x:0 0 0 2024-04-22T23:45:53.581 INFO:tasks.workunit.client.0.smithi031.stdout:0/909: dread d0/d13/d18/d1e/d25/d8e/da7/f79 [0,4194304] 0 2024-04-22T23:45:53.584 INFO:tasks.workunit.client.0.smithi031.stdout:1/913: truncate d1/d3f/d54/dba/ff8 868709 0 2024-04-22T23:45:53.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/993: dwrite db/d14/ff3 [0,4194304] 0 2024-04-22T23:45:53.587 INFO:tasks.workunit.client.0.smithi031.stdout:7/926: creat d9/d14/d132/f145 x:0 0 0 2024-04-22T23:45:53.588 INFO:tasks.workunit.client.0.smithi031.stdout:5/834: dwrite db/d16/d2a/d59/fec [0,4194304] 0 2024-04-22T23:45:53.588 INFO:tasks.workunit.client.0.smithi031.stdout:3/804: rename d4/d6/d11/d25/d44/d7f/d5f/c95 to d4/d54/d6d/c117 0 2024-04-22T23:45:53.589 INFO:tasks.workunit.client.0.smithi031.stdout:4/994: write db/d14/ff3 [7078980,57229] 0 2024-04-22T23:45:53.590 INFO:tasks.workunit.client.0.smithi031.stdout:0/910: dread d0/db5/fcf [0,4194304] 0 2024-04-22T23:45:53.591 INFO:tasks.workunit.client.0.smithi031.stdout:5/835: stat db/d16/d1d/d24/d58/d5d/cc3 0 2024-04-22T23:45:53.599 INFO:tasks.workunit.client.0.smithi031.stdout:5/836: dwrite db/d16/fe1 [0,4194304] 0 2024-04-22T23:45:53.602 INFO:tasks.workunit.client.0.smithi031.stdout:7/927: rename d9/d14/d1f/d45/da6/lbc to d9/d14/dd6/d129/l146 0 2024-04-22T23:45:53.603 INFO:tasks.workunit.client.0.smithi031.stdout:4/995: chown db/d14/d1d/d7b/d7f/f32 36 1 2024-04-22T23:45:53.606 INFO:tasks.workunit.client.0.smithi031.stdout:7/928: read d9/ff [2987107,120682] 0 2024-04-22T23:45:53.612 INFO:tasks.workunit.client.0.smithi031.stdout:5/837: symlink db/d16/d1d/d24/d10d/l113 0 2024-04-22T23:45:53.613 INFO:tasks.workunit.client.0.smithi031.stdout:2/913: sync 2024-04-22T23:45:53.613 INFO:tasks.workunit.client.0.smithi031.stdout:7/929: mknod d9/d14/d22/d28/d125/c147 0 2024-04-22T23:45:53.614 INFO:tasks.workunit.client.0.smithi031.stdout:1/914: link d1/d35/d36/d108/d4b/d6f/dd8/ce1 d1/d35/d36/d108/d4b/c12f 0 2024-04-22T23:45:53.614 INFO:tasks.workunit.client.0.smithi031.stdout:5/838: write db/d16/d1d/d24/d58/d5d/f109 [562456,119366] 0 2024-04-22T23:45:53.615 INFO:tasks.workunit.client.0.smithi031.stdout:2/914: mknod d6/dc/d48/d8a/c125 0 2024-04-22T23:45:53.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/915: chown d6/dc/d49/da6 240 1 2024-04-22T23:45:53.617 INFO:tasks.workunit.client.0.smithi031.stdout:7/930: symlink d9/d14/d1f/d45/da6/daa/db0/l148 0 2024-04-22T23:45:53.619 INFO:tasks.workunit.client.0.smithi031.stdout:1/915: creat d1/d35/d36/d108/d4b/f130 x:0 0 0 2024-04-22T23:45:53.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/805: write d4/d6/d11/d3e/f5a [217184,104018] 0 2024-04-22T23:45:53.625 INFO:tasks.workunit.client.0.smithi031.stdout:5/839: creat db/d16/d1d/f114 x:0 0 0 2024-04-22T23:45:53.626 INFO:tasks.workunit.client.0.smithi031.stdout:1/916: creat d1/d3f/d54/dba/f131 x:0 0 0 2024-04-22T23:45:53.626 INFO:tasks.workunit.client.0.smithi031.stdout:0/911: truncate d0/d13/d18/f6d 257074 0 2024-04-22T23:45:53.628 INFO:tasks.workunit.client.0.smithi031.stdout:0/912: fsync d0/d13/d18/d1e/d25/d8e/dac/dda/fcc 0 2024-04-22T23:45:53.629 INFO:tasks.workunit.client.0.smithi031.stdout:7/931: symlink d9/d14/d1f/d45/de3/d11d/l149 0 2024-04-22T23:45:53.630 INFO:tasks.workunit.client.0.smithi031.stdout:7/932: dread - d9/d15/fe4 zero size 2024-04-22T23:45:53.631 INFO:tasks.workunit.client.0.smithi031.stdout:5/840: mkdir db/d16/d1d/d24/d32/d41/d115 0 2024-04-22T23:45:53.633 INFO:tasks.workunit.client.0.smithi031.stdout:4/996: dread db/d14/d2d/d49/d59/d6a/d63/d107/d129/dc6/dd2/fd9 [4194304,4194304] 0 2024-04-22T23:45:53.633 INFO:tasks.workunit.client.0.smithi031.stdout:1/917: rename d1/d3f/d54/d84 to d1/d3f/d54/d132 0 2024-04-22T23:45:53.636 INFO:tasks.workunit.client.0.smithi031.stdout:5/841: readlink db/d16/d1d/d24/ddf/l101 0 2024-04-22T23:45:53.638 INFO:tasks.workunit.client.0.smithi031.stdout:7/933: rename d9/d14/d22/d5c/f9a to d9/d14/d1f/d45/de3/dfd/f14a 0 2024-04-22T23:45:53.640 INFO:tasks.workunit.client.0.smithi031.stdout:0/913: mknod d0/d1/d2/d36/d11/c126 0 2024-04-22T23:45:53.642 INFO:tasks.workunit.client.0.smithi031.stdout:2/916: dread d6/dc/d9a/d6c/db2/df0/d94/fe0 [0,4194304] 0 2024-04-22T23:45:53.644 INFO:tasks.workunit.client.0.smithi031.stdout:1/918: dwrite d1/d3f/d54/dba/f131 [0,4194304] 0 2024-04-22T23:45:53.644 INFO:tasks.workunit.client.0.smithi031.stdout:3/806: write d4/d6/fca [2004361,56691] 0 2024-04-22T23:45:53.645 INFO:tasks.workunit.client.0.smithi031.stdout:2/917: chown d6/d45/f73 762129179 1 2024-04-22T23:45:53.646 INFO:tasks.workunit.client.0.smithi031.stdout:7/934: mknod d9/d14/d1f/d45/da6/daa/c14b 0 2024-04-22T23:45:53.647 INFO:tasks.workunit.client.0.smithi031.stdout:4/997: mkdir db/d14/d1d/d7b/dc1/d13a/d158 0 2024-04-22T23:45:53.651 INFO:tasks.workunit.client.0.smithi031.stdout:2/918: symlink d6/dc/d49/d83/d85/l126 0 2024-04-22T23:45:53.654 INFO:tasks.workunit.client.0.smithi031.stdout:4/998: dread - db/d14/d2d/d49/d59/d6a/d63/d107/d129/dc6/dd2/f10e zero size 2024-04-22T23:45:53.655 INFO:tasks.workunit.client.0.smithi031.stdout:1/919: mknod d1/de/d1e/d34/d66/d124/c133 0 2024-04-22T23:45:53.657 INFO:tasks.workunit.client.0.smithi031.stdout:7/935: getdents d9/d14/d1f/d45/de3/dfd 0 2024-04-22T23:45:53.657 INFO:tasks.workunit.client.0.smithi031.stdout:2/919: chown d6/dc/d9a/d6c/db2/df0/d94/db4/f112 117945774 1 2024-04-22T23:45:53.661 INFO:tasks.workunit.client.0.smithi031.stdout:5/842: dwrite db/d16/fc5 [0,4194304] 0 2024-04-22T23:45:53.667 INFO:tasks.workunit.client.0.smithi031.stdout:1/920: dread d1/d3f/d54/dc6/fef [0,4194304] 0 2024-04-22T23:45:53.669 INFO:tasks.workunit.client.0.smithi031.stdout:5/843: mkdir db/d16/d2a/d59/d90/d116 0 2024-04-22T23:45:53.672 INFO:tasks.workunit.client.0.smithi031.stdout:1/921: chown d1/de/d1e/d34/lda 1323176 1 2024-04-22T23:45:53.673 INFO:tasks.workunit.client.0.smithi031.stdout:9/877: sync 2024-04-22T23:45:53.673 INFO:tasks.workunit.client.0.smithi031.stdout:8/812: sync 2024-04-22T23:45:53.674 INFO:tasks.workunit.client.0.smithi031.stdout:3/807: dread d4/d6/da/f3d [0,4194304] 0 2024-04-22T23:45:53.676 INFO:tasks.workunit.client.0.smithi031.stdout:4/999: dwrite db/d14/d2d/d49/d59/d6a/d63/d107/d129/dde/f11d [0,4194304] 0 2024-04-22T23:45:53.677 INFO:tasks.workunit.client.0.smithi031.stdout:1/922: mkdir d1/d99/d7d/d134 0 2024-04-22T23:45:53.678 INFO:tasks.workunit.client.0.smithi031.stdout:3/808: fsync d4/d6/da/ff 0 2024-04-22T23:45:53.683 INFO:tasks.workunit.client.0.smithi031.stdout:7/936: dwrite d9/d14/dd6/fff [0,4194304] 0 2024-04-22T23:45:53.686 INFO:tasks.workunit.client.0.smithi031.stdout:3/809: symlink d4/d83/dd8/de6/l118 0 2024-04-22T23:45:53.687 INFO:tasks.workunit.client.0.smithi031.stdout:5/844: dread db/d16/d1d/f6a [0,4194304] 0 2024-04-22T23:45:53.690 INFO:tasks.workunit.client.0.smithi031.stdout:2/920: dread d6/dc/d9a/d6c/de4/fea [0,4194304] 0 2024-04-22T23:45:53.693 INFO:tasks.workunit.client.0.smithi031.stdout:5/845: creat db/d16/d2a/d36/d74/d82/f117 x:0 0 0 2024-04-22T23:45:53.695 INFO:tasks.workunit.client.0.smithi031.stdout:7/937: creat d9/d14/d1f/d45/da6/daa/f14c x:0 0 0 2024-04-22T23:45:53.697 INFO:tasks.workunit.client.0.smithi031.stdout:9/878: rename d2/dd/d75/d64/d5d/d109/f116 to d2/dd/d75/d64/d5d/d59/f124 0 2024-04-22T23:45:53.699 INFO:tasks.workunit.client.0.smithi031.stdout:9/879: stat d2/d15/daf/de5 0 2024-04-22T23:45:53.699 INFO:tasks.workunit.client.0.smithi031.stdout:2/921: fdatasync d6/dc/d42/fa0 0 2024-04-22T23:45:53.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/813: dwrite d5/da/d12/d37/d34/f4c [0,4194304] 0 2024-04-22T23:45:53.701 INFO:tasks.workunit.client.0.smithi031.stdout:1/923: link d1/d35/l123 d1/db/d16/d1f/d52/l135 0 2024-04-22T23:45:53.702 INFO:tasks.workunit.client.0.smithi031.stdout:7/938: symlink d9/d14/d1f/d45/da6/daa/db0/l14d 0 2024-04-22T23:45:53.702 INFO:tasks.workunit.client.0.smithi031.stdout:2/922: fsync d6/d45/f73 0 2024-04-22T23:45:53.705 INFO:tasks.workunit.client.0.smithi031.stdout:1/924: rename d1/d35/d36 to d1/d35/d36/d108/d4b/d6f/dd8/d136 22 2024-04-22T23:45:53.707 INFO:tasks.workunit.client.0.smithi031.stdout:1/925: stat d1/d35/d36/d108/d4e 0 2024-04-22T23:45:53.709 INFO:tasks.workunit.client.0.smithi031.stdout:3/810: truncate d4/d54/d6d/f9b 1639843 0 2024-04-22T23:45:53.710 INFO:tasks.workunit.client.0.smithi031.stdout:7/939: creat d9/d15/f14e x:0 0 0 2024-04-22T23:45:53.711 INFO:tasks.workunit.client.0.smithi031.stdout:3/811: rmdir d4/d83/dd8/de6 39 2024-04-22T23:45:53.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/940: symlink d9/d14/d1f/d45/de3/d11d/l14f 0 2024-04-22T23:45:53.717 INFO:tasks.workunit.client.0.smithi031.stdout:3/812: rename d4/d54/d2c/l3a to d4/d54/d88/da2/de0/d10b/l119 0 2024-04-22T23:45:53.717 INFO:tasks.workunit.client.0.smithi031.stdout:1/926: link d1/d35/d36/d108/d4b/c128 d1/d35/d8d/d102/d81/c137 0 2024-04-22T23:45:53.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/923: write d6/f9 [7680597,120097] 0 2024-04-22T23:45:53.723 INFO:tasks.workunit.client.0.smithi031.stdout:1/927: symlink d1/d35/d36/d108/d4b/d6f/l138 0 2024-04-22T23:45:53.723 INFO:tasks.workunit.client.0.smithi031.stdout:5/846: dread db/d16/d1d/d24/d32/f83 [0,4194304] 0 2024-04-22T23:45:53.724 INFO:tasks.workunit.client.0.smithi031.stdout:3/813: truncate d4/d83/f8b 3881461 0 2024-04-22T23:45:53.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/814: dread d5/f19 [0,4194304] 0 2024-04-22T23:45:53.726 INFO:tasks.workunit.client.0.smithi031.stdout:3/814: chown d4/d6/d11/ddf/d115 2543686 1 2024-04-22T23:45:53.727 INFO:tasks.workunit.client.0.smithi031.stdout:7/941: dwrite d9/d14/d22/d28/f130 [0,4194304] 0 2024-04-22T23:45:53.727 INFO:tasks.workunit.client.0.smithi031.stdout:2/924: creat d6/dc/d42/d24/d32/f127 x:0 0 0 2024-04-22T23:45:53.732 INFO:tasks.workunit.client.0.smithi031.stdout:7/942: chown d9/le6 2 1 2024-04-22T23:45:53.732 INFO:tasks.workunit.client.0.smithi031.stdout:8/815: rmdir d5/da/d12/d37/d34/d6f 39 2024-04-22T23:45:53.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/928: dwrite d1/d35/d36/d108/d4b/d6f/da6/fc7 [0,4194304] 0 2024-04-22T23:45:53.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/816: symlink d5/da/d12/d1d/d38/lfd 0 2024-04-22T23:45:53.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/925: rmdir d6/dc/d9a/d10e 0 2024-04-22T23:45:53.740 INFO:tasks.workunit.client.0.smithi031.stdout:5/847: write db/d16/fce [263980,56504] 0 2024-04-22T23:45:53.741 INFO:tasks.workunit.client.0.smithi031.stdout:3/815: creat d4/d6/d11/d25/d44/d7f/d5f/dee/df8/f11a x:0 0 0 2024-04-22T23:45:53.742 INFO:tasks.workunit.client.0.smithi031.stdout:2/926: truncate d6/d45/d50/fed 113181 0 2024-04-22T23:45:53.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/943: rename d9/d14/d1f/d45/de3/d11d/l149 to d9/d14/d22/d5c/d5f/l150 0 2024-04-22T23:45:53.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/929: mkdir d1/db/d16/da0/d139 0 2024-04-22T23:45:53.744 INFO:tasks.workunit.client.0.smithi031.stdout:7/944: write d9/d15/f114 [695096,22822] 0 2024-04-22T23:45:53.748 INFO:tasks.workunit.client.0.smithi031.stdout:3/816: mknod d4/d6/d11/d25/d44/d7f/d5f/dee/c11b 0 2024-04-22T23:45:53.750 INFO:tasks.workunit.client.0.smithi031.stdout:2/927: mkdir d6/dc/d48/dc1/d128 0 2024-04-22T23:45:53.751 INFO:tasks.workunit.client.0.smithi031.stdout:1/930: dread d1/d35/d8d/d102/f113 [0,4194304] 0 2024-04-22T23:45:53.752 INFO:tasks.workunit.client.0.smithi031.stdout:7/945: rename d9/d14/d22/d28/d125 to d9/d15/d3a/d41/d151 0 2024-04-22T23:45:53.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/928: dread - d6/dc/d48/d4b/d6b/fc7 zero size 2024-04-22T23:45:53.754 INFO:tasks.workunit.client.0.smithi031.stdout:1/931: fdatasync d1/de/d1e/d94/d125/ff6 0 2024-04-22T23:45:53.754 INFO:tasks.workunit.client.0.smithi031.stdout:7/946: fdatasync d9/d14/d1f/d45/da6/daa/d10b/f10e 0 2024-04-22T23:45:53.756 INFO:tasks.workunit.client.0.smithi031.stdout:3/817: dread d4/d6/d11/d25/d44/d7f/d5f/fb1 [0,4194304] 0 2024-04-22T23:45:53.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/929: mkdir d6/dc/d48/d4b/d6b/de8/d129 0 2024-04-22T23:45:53.760 INFO:tasks.workunit.client.0.smithi031.stdout:1/932: dread d1/d35/d8d/d102/f50 [0,4194304] 0 2024-04-22T23:45:53.760 INFO:tasks.workunit.client.0.smithi031.stdout:3/818: unlink d4/d54/d6d/fed 0 2024-04-22T23:45:53.762 INFO:tasks.workunit.client.0.smithi031.stdout:7/947: dwrite d9/d15/f60 [0,4194304] 0 2024-04-22T23:45:53.762 INFO:tasks.workunit.client.0.smithi031.stdout:5/848: write db/d16/d1d/d24/d58/db1/fb5 [1403952,104679] 0 2024-04-22T23:45:53.764 INFO:tasks.workunit.client.0.smithi031.stdout:2/930: link d6/dc/d48/d4b/d6b/fc7 d6/dc/d17/d124/f12a 0 2024-04-22T23:45:53.766 INFO:tasks.workunit.client.0.smithi031.stdout:3/819: write d4/d6/d11/d25/d44/d7f/d5f/ffa [511495,109284] 0 2024-04-22T23:45:53.769 INFO:tasks.workunit.client.0.smithi031.stdout:2/931: stat d6/dc/d17/c89 0 2024-04-22T23:45:53.771 INFO:tasks.workunit.client.0.smithi031.stdout:8/817: dwrite d5/da/df/f13 [0,4194304] 0 2024-04-22T23:45:53.772 INFO:tasks.workunit.client.0.smithi031.stdout:7/948: rename d9/d14/d1f/d7f/fc7 to d9/d15/d6d/d65/dea/f152 0 2024-04-22T23:45:53.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/820: symlink d4/d6/d11/de3/l11c 0 2024-04-22T23:45:53.774 INFO:tasks.workunit.client.0.smithi031.stdout:8/818: chown d5/da/d12/d2e/d7e/f51 807894 1 2024-04-22T23:45:53.774 INFO:tasks.workunit.client.0.smithi031.stdout:7/949: readlink d9/d14/d22/d5c/lf5 0 2024-04-22T23:45:53.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/932: rename d6/dc/d49/d83 to d6/dc/d42/d24/d12b 0 2024-04-22T23:45:53.779 INFO:tasks.workunit.client.0.smithi031.stdout:5/849: getdents db/d16/d2a/d59/d90/ddd 0 2024-04-22T23:45:53.780 INFO:tasks.workunit.client.0.smithi031.stdout:7/950: dwrite d9/d15/d6d/da9/f10f [0,4194304] 0 2024-04-22T23:45:53.782 INFO:tasks.workunit.client.0.smithi031.stdout:8/819: unlink d5/da/d83/d8d/fa4 0 2024-04-22T23:45:53.784 INFO:tasks.workunit.client.0.smithi031.stdout:1/933: truncate d1/db/ffa 2529757 0 2024-04-22T23:45:53.786 INFO:tasks.workunit.client.0.smithi031.stdout:2/933: creat d6/dc/d9a/d6c/db2/f12c x:0 0 0 2024-04-22T23:45:53.786 INFO:tasks.workunit.client.0.smithi031.stdout:2/934: stat d6/d45/d50/dfe 0 2024-04-22T23:45:53.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/850: mknod db/d16/d1d/d24/d58/db1/c118 0 2024-04-22T23:45:53.789 INFO:tasks.workunit.client.0.smithi031.stdout:7/951: mknod d9/d15/d3a/d41/d151/c153 0 2024-04-22T23:45:53.796 INFO:tasks.workunit.client.0.smithi031.stdout:8/820: rename d5/da/d12/d2e/db0/dfb to d5/da/d83/dfe 0 2024-04-22T23:45:53.796 INFO:tasks.workunit.client.0.smithi031.stdout:3/821: write d4/d54/f78 [846380,13158] 0 2024-04-22T23:45:53.799 INFO:tasks.workunit.client.0.smithi031.stdout:1/934: symlink d1/d35/d8d/l13a 0 2024-04-22T23:45:53.801 INFO:tasks.workunit.client.0.smithi031.stdout:2/935: mknod d6/dc/d42/d24/d12b/dc2/c12d 0 2024-04-22T23:45:53.802 INFO:tasks.workunit.client.0.smithi031.stdout:2/936: readlink d6/dc/d42/d24/d9c/lbc 0 2024-04-22T23:45:53.804 INFO:tasks.workunit.client.0.smithi031.stdout:3/822: creat d4/d6/d11/d25/d44/d7f/da5/f11d x:0 0 0 2024-04-22T23:45:53.807 INFO:tasks.workunit.client.0.smithi031.stdout:5/851: getdents db 0 2024-04-22T23:45:53.809 INFO:tasks.workunit.client.0.smithi031.stdout:8/821: link d5/da/d12/l66 d5/da/d12/d1d/d38/lff 0 2024-04-22T23:45:53.809 INFO:tasks.workunit.client.0.smithi031.stdout:1/935: dwrite d1/d3f/d54/f11a [0,4194304] 0 2024-04-22T23:45:53.813 INFO:tasks.workunit.client.0.smithi031.stdout:1/936: write d1/d35/d36/f9e [2522376,14443] 0 2024-04-22T23:45:53.815 INFO:tasks.workunit.client.0.smithi031.stdout:8/822: mknod d5/da/d83/dfe/c100 0 2024-04-22T23:45:53.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/823: chown d5/da/d12/d37/d4e/l8e 877766861 1 2024-04-22T23:45:53.818 INFO:tasks.workunit.client.0.smithi031.stdout:7/952: dread d9/d14/d1f/d45/da6/daa/d10b/f10e [0,4194304] 0 2024-04-22T23:45:53.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/852: truncate db/d16/f2b 1675810 0 2024-04-22T23:45:53.820 INFO:tasks.workunit.client.0.smithi031.stdout:7/953: write d9/d14/d132/f145 [790944,67980] 0 2024-04-22T23:45:53.821 INFO:tasks.workunit.client.0.smithi031.stdout:1/937: mknod d1/de/d1e/c13b 0 2024-04-22T23:45:53.823 INFO:tasks.workunit.client.0.smithi031.stdout:8/824: mkdir d5/da/d12/d37/d4e/d101 0 2024-04-22T23:45:53.824 INFO:tasks.workunit.client.0.smithi031.stdout:2/937: dread d6/dc/d17/d69/f7f [0,4194304] 0 2024-04-22T23:45:53.826 INFO:tasks.workunit.client.0.smithi031.stdout:1/938: symlink d1/d3f/d54/d132/l13c 0 2024-04-22T23:45:53.829 INFO:tasks.workunit.client.0.smithi031.stdout:7/954: dwrite d9/d15/d6d/d65/faf [0,4194304] 0 2024-04-22T23:45:53.829 INFO:tasks.workunit.client.0.smithi031.stdout:8/825: mknod d5/da/d12/d2e/d4d/df9/c102 0 2024-04-22T23:45:53.831 INFO:tasks.workunit.client.0.smithi031.stdout:1/939: creat d1/d99/d7d/d118/f13d x:0 0 0 2024-04-22T23:45:53.831 INFO:tasks.workunit.client.0.smithi031.stdout:3/823: truncate d4/d6/d22/feb 3030432 0 2024-04-22T23:45:53.835 INFO:tasks.workunit.client.0.smithi031.stdout:7/955: symlink d9/d14/d1f/d45/da6/daa/dbb/dd8/l154 0 2024-04-22T23:45:53.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/938: getdents d6/dc/d9a/d6c/db2 0 2024-04-22T23:45:53.836 INFO:tasks.workunit.client.0.smithi031.stdout:1/940: truncate d1/f6 1461502 0 2024-04-22T23:45:53.838 INFO:tasks.workunit.client.0.smithi031.stdout:3/824: symlink d4/d54/d6d/d98/l11e 0 2024-04-22T23:45:53.841 INFO:tasks.workunit.client.0.smithi031.stdout:2/939: chown d6/dc/d9a/d6c/db2/df0/d94/db4 20094754 1 2024-04-22T23:45:53.845 INFO:tasks.workunit.client.0.smithi031.stdout:5/853: dwrite db/d16/d1d/d24/d32/f83 [0,4194304] 0 2024-04-22T23:45:53.845 INFO:tasks.workunit.client.0.smithi031.stdout:3/825: chown d4/d6/d11/d25/d44/d7f/da5/fce 153304 1 2024-04-22T23:45:53.845 INFO:tasks.workunit.client.0.smithi031.stdout:8/826: dwrite d5/da/d12/d37/d34/d6f/f4b [4194304,4194304] 0 2024-04-22T23:45:53.846 INFO:tasks.workunit.client.0.smithi031.stdout:2/940: dread - d6/dc/d9a/d6c/db2/f12c zero size 2024-04-22T23:45:53.851 INFO:tasks.workunit.client.0.smithi031.stdout:8/827: readlink d5/l64 0 2024-04-22T23:45:53.851 INFO:tasks.workunit.client.0.smithi031.stdout:3/826: mknod d4/d54/d88/c11f 0 2024-04-22T23:45:53.854 INFO:tasks.workunit.client.0.smithi031.stdout:8/828: rename d5/da/d83/f8c to d5/da/d12/d95/f103 0 2024-04-22T23:45:53.856 INFO:tasks.workunit.client.0.smithi031.stdout:1/941: link d1/d3f/d54/dba/d28/fa8 d1/de/d1e/d34/d66/d124/f13e 0 2024-04-22T23:45:53.857 INFO:tasks.workunit.client.0.smithi031.stdout:2/941: symlink d6/dc/d48/dc1/d128/l12e 0 2024-04-22T23:45:53.858 INFO:tasks.workunit.client.0.smithi031.stdout:5/854: creat db/d16/d2a/d59/d90/da4/f119 x:0 0 0 2024-04-22T23:45:53.859 INFO:tasks.workunit.client.0.smithi031.stdout:3/827: mknod d4/d54/d2c/d89/d2e/c120 0 2024-04-22T23:45:53.861 INFO:tasks.workunit.client.0.smithi031.stdout:8/829: read d5/da/d12/d2e/d7e/fb4 [102817,76095] 0 2024-04-22T23:45:53.861 INFO:tasks.workunit.client.0.smithi031.stdout:3/828: write d4/d6/d11/d25/d44/d7f/da5/f102 [569021,4374] 0 2024-04-22T23:45:53.864 INFO:tasks.workunit.client.0.smithi031.stdout:7/956: dread d9/d14/d22/d5c/f80 [0,4194304] 0 2024-04-22T23:45:53.864 INFO:tasks.workunit.client.0.smithi031.stdout:8/830: creat d5/da/d83/f104 x:0 0 0 2024-04-22T23:45:53.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/829: symlink d4/d54/l121 0 2024-04-22T23:45:53.865 INFO:tasks.workunit.client.0.smithi031.stdout:2/942: dread d6/dc/d48/d4b/fb0 [0,4194304] 0 2024-04-22T23:45:53.867 INFO:tasks.workunit.client.0.smithi031.stdout:7/957: rmdir d9/d15 39 2024-04-22T23:45:53.870 INFO:tasks.workunit.client.0.smithi031.stdout:2/943: fsync d6/dc/d9a/d6c/d86/fb7 0 2024-04-22T23:45:53.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/958: dread d9/d15/f11a [0,4194304] 0 2024-04-22T23:45:53.875 INFO:tasks.workunit.client.0.smithi031.stdout:2/944: creat d6/dc/d17/d69/f12f x:0 0 0 2024-04-22T23:45:53.877 INFO:tasks.workunit.client.0.smithi031.stdout:2/945: write d6/dc/d48/d4b/d6b/de8/fa3 [1299052,39920] 0 2024-04-22T23:45:53.878 INFO:tasks.workunit.client.0.smithi031.stdout:5/855: rename db/d16/d2a/d59/d90 to db/d8d/d11a 0 2024-04-22T23:45:53.878 INFO:tasks.workunit.client.0.smithi031.stdout:8/831: rename d5/da/d12/d2e/db0 to d5/da/d12/d2e/db0/d105 22 2024-04-22T23:45:53.880 INFO:tasks.workunit.client.0.smithi031.stdout:2/946: read d6/dc/d42/d24/d12b/fbb [188231,82890] 0 2024-04-22T23:45:53.880 INFO:tasks.workunit.client.0.smithi031.stdout:8/832: truncate d5/f19 4970729 0 2024-04-22T23:45:53.882 INFO:tasks.workunit.client.0.smithi031.stdout:5/856: symlink db/d16/d2a/d7c/d89/l11b 0 2024-04-22T23:45:53.882 INFO:tasks.workunit.client.0.smithi031.stdout:8/833: dread - d5/da/d83/f104 zero size 2024-04-22T23:45:53.883 INFO:tasks.workunit.client.0.smithi031.stdout:3/830: dread d4/d6/f23 [0,4194304] 0 2024-04-22T23:45:53.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/942: dwrite d1/db/ffa [0,4194304] 0 2024-04-22T23:45:53.888 INFO:tasks.workunit.client.0.smithi031.stdout:8/834: mkdir d5/da/d12/d37/d4e/de7/d106 0 2024-04-22T23:45:53.890 INFO:tasks.workunit.client.0.smithi031.stdout:2/947: write d6/f16 [728715,108432] 0 2024-04-22T23:45:53.892 INFO:tasks.workunit.client.0.smithi031.stdout:8/835: write d5/da/f59 [102355,6828] 0 2024-04-22T23:45:53.892 INFO:tasks.workunit.client.0.smithi031.stdout:3/831: stat d4/d54/d6d/c117 0 2024-04-22T23:45:53.892 INFO:tasks.workunit.client.0.smithi031.stdout:1/943: mknod d1/d35/dbb/c13f 0 2024-04-22T23:45:53.894 INFO:tasks.workunit.client.0.smithi031.stdout:7/959: dwrite d9/d14/d22/d28/f4a [0,4194304] 0 2024-04-22T23:45:53.901 INFO:tasks.workunit.client.0.smithi031.stdout:5/857: getdents db/d16/d1d/d24/d7d 0 2024-04-22T23:45:53.902 INFO:tasks.workunit.client.0.smithi031.stdout:1/944: dwrite d1/de/d1e/d34/d66/f121 [0,4194304] 0 2024-04-22T23:45:53.904 INFO:tasks.workunit.client.0.smithi031.stdout:3/832: symlink d4/d6/d11/d25/d44/d7f/d5f/l122 0 2024-04-22T23:45:53.907 INFO:tasks.workunit.client.0.smithi031.stdout:7/960: mkdir d9/d14/d1f/d45/de3/d155 0 2024-04-22T23:45:53.908 INFO:tasks.workunit.client.0.smithi031.stdout:7/961: write d9/d15/d6d/d65/dea/f12c [26290,8302] 0 2024-04-22T23:45:53.911 INFO:tasks.workunit.client.0.smithi031.stdout:5/858: symlink db/d8d/df1/l11c 0 2024-04-22T23:45:53.917 INFO:tasks.workunit.client.0.smithi031.stdout:3/833: rename d4/d54/d2c/d89/lc9 to d4/d54/d2c/d89/d2e/d8d/l123 0 2024-04-22T23:45:53.920 INFO:tasks.workunit.client.0.smithi031.stdout:7/962: creat d9/d15/d6d/d65/dea/f156 x:0 0 0 2024-04-22T23:45:53.924 INFO:tasks.workunit.client.0.smithi031.stdout:3/834: dwrite d4/d6/d11/d25/d44/d7f/f69 [0,4194304] 0 2024-04-22T23:45:53.925 INFO:tasks.workunit.client.0.smithi031.stdout:5/859: read db/d16/d1d/d24/d58/f85 [316103,30349] 0 2024-04-22T23:45:53.926 INFO:tasks.workunit.client.0.smithi031.stdout:1/945: truncate d1/db/d16/d1f/d52/f5c 1911810 0 2024-04-22T23:45:53.927 INFO:tasks.workunit.client.0.smithi031.stdout:5/860: chown db/d16/d2a/d36/d74/d82/d86/f91 45939 1 2024-04-22T23:45:53.928 INFO:tasks.workunit.client.0.smithi031.stdout:8/836: link d5/da/d83/d8d/lae d5/da/d12/d1d/d38/db9/l107 0 2024-04-22T23:45:53.928 INFO:tasks.workunit.client.0.smithi031.stdout:2/948: dread d6/dc/d9a/d6c/f76 [0,4194304] 0 2024-04-22T23:45:53.929 INFO:tasks.workunit.client.0.smithi031.stdout:2/949: fsync d6/dc/d9a/d6c/db2/df0/d94/dab/f11e 0 2024-04-22T23:45:53.937 INFO:tasks.workunit.client.0.smithi031.stdout:1/946: creat d1/d99/d63/f140 x:0 0 0 2024-04-22T23:45:53.940 INFO:tasks.workunit.client.0.smithi031.stdout:3/835: dwrite d4/d6/d11/d25/d44/d7f/d5f/fb1 [4194304,4194304] 0 2024-04-22T23:45:53.940 INFO:tasks.workunit.client.0.smithi031.stdout:5/861: mkdir db/d16/d2a/d36/d11d 0 2024-04-22T23:45:53.941 INFO:tasks.workunit.client.0.smithi031.stdout:3/836: read - d4/d54/d6d/d98/fea zero size 2024-04-22T23:45:53.948 INFO:tasks.workunit.client.0.smithi031.stdout:0/914: sync 2024-04-22T23:45:53.948 INFO:tasks.workunit.client.0.smithi031.stdout:9/880: sync 2024-04-22T23:45:53.952 INFO:tasks.workunit.client.0.smithi031.stdout:8/837: creat d5/da/d12/d37/d4e/dc9/f108 x:0 0 0 2024-04-22T23:45:53.953 INFO:tasks.workunit.client.0.smithi031.stdout:2/950: mkdir d6/dc/d9a/d6c/d130 0 2024-04-22T23:45:53.955 INFO:tasks.workunit.client.0.smithi031.stdout:3/837: dread d4/d6/d11/d25/d44/d7f/f69 [0,4194304] 0 2024-04-22T23:45:53.956 INFO:tasks.workunit.client.0.smithi031.stdout:2/951: write d6/dc/d48/d4b/d6b/de8/fa3 [2072682,99724] 0 2024-04-22T23:45:53.957 INFO:tasks.workunit.client.0.smithi031.stdout:1/947: dread d1/de/d1e/d94/d125/ff6 [0,4194304] 0 2024-04-22T23:45:53.960 INFO:tasks.workunit.client.0.smithi031.stdout:2/952: truncate d6/dc/d9a/d6c/db2/df0/d94/db4/f112 159063 0 2024-04-22T23:45:53.969 INFO:tasks.workunit.client.0.smithi031.stdout:1/948: dread d1/d35/d36/fbd [0,4194304] 0 2024-04-22T23:45:54.027 INFO:tasks.workunit.client.0.smithi031.stdout:7/963: link d9/d14/d1f/dc6/c136 d9/d15/d3f/ddc/d106/c157 0 2024-04-22T23:45:54.027 INFO:tasks.workunit.client.0.smithi031.stdout:7/964: truncate d9/d15/f109 58426 0 2024-04-22T23:45:54.030 INFO:tasks.workunit.client.0.smithi031.stdout:9/881: creat d2/dc/d3e/da1/f125 x:0 0 0 2024-04-22T23:45:54.032 INFO:tasks.workunit.client.0.smithi031.stdout:0/915: mknod d0/d13/d18/d1e/d25/d8e/dac/dda/c127 0 2024-04-22T23:45:54.034 INFO:tasks.workunit.client.0.smithi031.stdout:8/838: mkdir d5/da/d12/d2e/db0/dc3/d109 0 2024-04-22T23:45:54.036 INFO:tasks.workunit.client.0.smithi031.stdout:9/882: dwrite d2/dd/d65/feb [0,4194304] 0 2024-04-22T23:45:54.036 INFO:tasks.workunit.client.0.smithi031.stdout:8/839: read d5/d48/d61/d6b/fa1 [1286906,56392] 0 2024-04-22T23:45:54.038 INFO:tasks.workunit.client.0.smithi031.stdout:9/883: truncate d2/dc/fb7 928044 0 2024-04-22T23:45:54.038 INFO:tasks.workunit.client.0.smithi031.stdout:3/838: creat d4/d54/d6d/f124 x:0 0 0 2024-04-22T23:45:54.045 INFO:tasks.workunit.client.0.smithi031.stdout:1/949: rename d1/d99/d63/ffc to d1/d3f/d54/dba/ddc/f141 0 2024-04-22T23:45:54.047 INFO:tasks.workunit.client.0.smithi031.stdout:7/965: write d9/d14/d22/fb9 [1730338,57792] 0 2024-04-22T23:45:54.051 INFO:tasks.workunit.client.0.smithi031.stdout:0/916: creat d0/d1/d2/d36/d11/d1a/dfc/dad/f128 x:0 0 0 2024-04-22T23:45:54.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/884: mknod d2/dd/d75/d64/d5d/d109/c126 0 2024-04-22T23:45:54.057 INFO:tasks.workunit.client.0.smithi031.stdout:3/839: fsync d4/d6/d11/d3e/fbe 0 2024-04-22T23:45:54.061 INFO:tasks.workunit.client.0.smithi031.stdout:5/862: creat db/d16/d2a/d36/d74/d82/f11e x:0 0 0 2024-04-22T23:45:54.061 INFO:tasks.workunit.client.0.smithi031.stdout:9/885: dwrite d2/dd/d75/d64/d5d/d59/fb0 [4194304,4194304] 0 2024-04-22T23:45:54.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/950: rename d1/cb9 to d1/db/d16/d1f/d52/d11d/c142 0 2024-04-22T23:45:54.068 INFO:tasks.workunit.client.0.smithi031.stdout:1/951: read - d1/d35/d36/d108/d4e/fcd zero size 2024-04-22T23:45:54.075 INFO:tasks.workunit.client.0.smithi031.stdout:8/840: mkdir d5/d48/d61/d6b/d6c/dd1/d10a 0 2024-04-22T23:45:54.077 INFO:tasks.workunit.client.0.smithi031.stdout:8/841: read d5/da/d12/d2e/f7d [1583972,16841] 0 2024-04-22T23:45:54.078 INFO:tasks.workunit.client.0.smithi031.stdout:8/842: readlink d5/da/d12/d37/d34/l35 0 2024-04-22T23:45:54.079 INFO:tasks.workunit.client.0.smithi031.stdout:8/843: chown d5/da/d12/d2e/d7e/fb4 36719250 1 2024-04-22T23:45:54.080 INFO:tasks.workunit.client.0.smithi031.stdout:8/844: write d5/da/d83/d8d/f91 [3536071,130964] 0 2024-04-22T23:45:54.082 INFO:tasks.workunit.client.0.smithi031.stdout:3/840: dread d4/d54/d2c/d89/d2e/d45/f60 [0,4194304] 0 2024-04-22T23:45:54.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/886: creat d2/dc/d8b/d3d/d40/da4/d104/f127 x:0 0 0 2024-04-22T23:45:54.087 INFO:tasks.workunit.client.0.smithi031.stdout:7/966: mkdir d9/d14/d22/d5c/d158 0 2024-04-22T23:45:54.098 INFO:tasks.workunit.client.0.smithi031.stdout:8/845: creat d5/da/d12/d2e/d4d/f10b x:0 0 0 2024-04-22T23:45:54.098 INFO:tasks.workunit.client.0.smithi031.stdout:3/841: unlink d4/d54/d6d/l8a 0 2024-04-22T23:45:54.099 INFO:tasks.workunit.client.0.smithi031.stdout:3/842: write d4/d6/d11/d25/d44/d7f/da5/f11d [675683,91198] 0 2024-04-22T23:45:54.100 INFO:tasks.workunit.client.0.smithi031.stdout:9/887: unlink d2/dd/d75/d80/f82 0 2024-04-22T23:45:54.108 INFO:tasks.workunit.client.0.smithi031.stdout:8/846: chown d5/da/d12/d37/f78 59 1 2024-04-22T23:45:54.109 INFO:tasks.workunit.client.0.smithi031.stdout:3/843: unlink d4/d6/d4c/c4e 0 2024-04-22T23:45:54.109 INFO:tasks.workunit.client.0.smithi031.stdout:9/888: rmdir d2/dd 39 2024-04-22T23:45:54.115 INFO:tasks.workunit.client.0.smithi031.stdout:8/847: link d5/da/d12/d2e/d4d/d9a/ff6 d5/da/d12/d37/d34/d6f/d85/dcd/f10c 0 2024-04-22T23:45:54.116 INFO:tasks.workunit.client.0.smithi031.stdout:9/889: creat d2/dd/d75/d64/d5d/d59/f128 x:0 0 0 2024-04-22T23:45:54.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/848: mknod d5/d48/d61/d6b/d6c/c10d 0 2024-04-22T23:45:54.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/849: readlink d5/da/d83/d8d/l53 0 2024-04-22T23:45:54.119 INFO:tasks.workunit.client.0.smithi031.stdout:9/890: creat d2/dc/d3e/da1/f129 x:0 0 0 2024-04-22T23:45:54.121 INFO:tasks.workunit.client.0.smithi031.stdout:8/850: creat d5/da/d12/d37/d34/d6f/d5b/f10e x:0 0 0 2024-04-22T23:45:54.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/851: fsync d5/da/d12/d37/d34/d6f/d5b/fde 0 2024-04-22T23:45:54.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/891: rename d2/dc/d8b/d3d/d40/da4/db1/dc5/def to d2/dd/d75/df7/d12a 0 2024-04-22T23:45:54.123 INFO:tasks.workunit.client.0.smithi031.stdout:8/852: write d5/da/d12/d37/d34/d6f/d5b/dcb/fd0 [150630,105971] 0 2024-04-22T23:45:54.124 INFO:tasks.workunit.client.0.smithi031.stdout:8/853: stat d5/da/d12/d37/d34/d6f/d5b/f84 0 2024-04-22T23:45:54.128 INFO:tasks.workunit.client.0.smithi031.stdout:8/854: dread d5/d48/f77 [0,4194304] 0 2024-04-22T23:45:54.130 INFO:tasks.workunit.client.0.smithi031.stdout:8/855: chown d5/da/d12/d1d/d38 1600 1 2024-04-22T23:45:54.132 INFO:tasks.workunit.client.0.smithi031.stdout:2/953: dwrite d6/dc/d42/f67 [4194304,4194304] 0 2024-04-22T23:45:54.133 INFO:tasks.workunit.client.0.smithi031.stdout:8/856: getdents d5/d48/d61/d6b/de1 0 2024-04-22T23:45:54.138 INFO:tasks.workunit.client.0.smithi031.stdout:2/954: link d6/dc/f1a d6/dc/d42/d24/d12b/d85/da9/f131 0 2024-04-22T23:45:54.141 INFO:tasks.workunit.client.0.smithi031.stdout:8/857: rename d5/d48/d61/d6b/d6c/dd1 to d5/da/df/d33/d10f 0 2024-04-22T23:45:54.143 INFO:tasks.workunit.client.0.smithi031.stdout:2/955: creat d6/dc/d57/f132 x:0 0 0 2024-04-22T23:45:54.144 INFO:tasks.workunit.client.0.smithi031.stdout:8/858: mknod d5/da/d12/d37/d34/d6f/d85/d9b/c110 0 2024-04-22T23:45:54.146 INFO:tasks.workunit.client.0.smithi031.stdout:8/859: truncate d5/da/d12/d2e/d4d/fbc 33765 0 2024-04-22T23:45:54.146 INFO:tasks.workunit.client.0.smithi031.stdout:2/956: mkdir d6/d45/d133 0 2024-04-22T23:45:54.149 INFO:tasks.workunit.client.0.smithi031.stdout:0/917: truncate d0/d1/d2/d36/d11/f14 3100949 0 2024-04-22T23:45:54.152 INFO:tasks.workunit.client.0.smithi031.stdout:8/860: dwrite d5/da/d12/d2e/d4d/f52 [0,4194304] 0 2024-04-22T23:45:54.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/861: creat d5/da/d12/d2e/d4d/d9a/db1/f111 x:0 0 0 2024-04-22T23:45:54.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/952: dwrite d1/db/d16/fcb [0,4194304] 0 2024-04-22T23:45:54.167 INFO:tasks.workunit.client.0.smithi031.stdout:2/957: write d6/dc/d57/d52/fdd [486726,129700] 0 2024-04-22T23:45:54.167 INFO:tasks.workunit.client.0.smithi031.stdout:8/862: rmdir d5/da/d12/d2e/d4d/d56/de8 0 2024-04-22T23:45:54.169 INFO:tasks.workunit.client.0.smithi031.stdout:7/967: dwrite d9/d14/d1f/fc2 [0,4194304] 0 2024-04-22T23:45:54.169 INFO:tasks.workunit.client.0.smithi031.stdout:3/844: dwrite d4/d54/d2c/d89/f91 [0,4194304] 0 2024-04-22T23:45:54.174 INFO:tasks.workunit.client.0.smithi031.stdout:3/845: fdatasync d4/d54/d6d/d98/fea 0 2024-04-22T23:45:54.175 INFO:tasks.workunit.client.0.smithi031.stdout:0/918: dread d0/d1/f6b [0,4194304] 0 2024-04-22T23:45:54.175 INFO:tasks.workunit.client.0.smithi031.stdout:7/968: write d9/d14/d1f/d7f/ffa [4855448,28446] 0 2024-04-22T23:45:54.176 INFO:tasks.workunit.client.0.smithi031.stdout:8/863: mkdir d5/da/d83/d112 0 2024-04-22T23:45:54.177 INFO:tasks.workunit.client.0.smithi031.stdout:2/958: write d6/dc/d9a/d6c/f76 [4533892,13763] 0 2024-04-22T23:45:54.183 INFO:tasks.workunit.client.0.smithi031.stdout:1/953: creat d1/d3f/d54/d132/d120/f143 x:0 0 0 2024-04-22T23:45:54.186 INFO:tasks.workunit.client.0.smithi031.stdout:8/864: dwrite d5/d48/d61/d6b/d6c/daa/fb8 [0,4194304] 0 2024-04-22T23:45:54.190 INFO:tasks.workunit.client.0.smithi031.stdout:3/846: rename d4/l14 to d4/d83/l125 0 2024-04-22T23:45:54.190 INFO:tasks.workunit.client.0.smithi031.stdout:0/919: write d0/db5/fcf [1633336,102620] 0 2024-04-22T23:45:54.193 INFO:tasks.workunit.client.0.smithi031.stdout:8/865: rename d5/da/d12/d37/d34/d6f to d5/da/d12/d37/d34/d6f/d113 22 2024-04-22T23:45:54.193 INFO:tasks.workunit.client.0.smithi031.stdout:3/847: fdatasync d4/d6/d4c/d8c/fa3 0 2024-04-22T23:45:54.193 INFO:tasks.workunit.client.0.smithi031.stdout:1/954: creat d1/db/d16/d1f/f144 x:0 0 0 2024-04-22T23:45:54.194 INFO:tasks.workunit.client.0.smithi031.stdout:0/920: read - d0/d1/d2/d36/d11/d1a/d56/f117 zero size 2024-04-22T23:45:54.195 INFO:tasks.workunit.client.0.smithi031.stdout:0/921: dread - d0/d1/d2/f7a zero size 2024-04-22T23:45:54.197 INFO:tasks.workunit.client.0.smithi031.stdout:3/848: write d4/d6/d11/d25/d44/d7f/d5f/ffa [716020,34844] 0 2024-04-22T23:45:54.198 INFO:tasks.workunit.client.0.smithi031.stdout:0/922: chown d0/d1/d2/d36/d11/d1a/dfc 45575661 1 2024-04-22T23:45:54.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/955: truncate d1/d99/d7d/d118/f13d 807598 0 2024-04-22T23:45:54.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/969: dread d9/d14/d1f/d45/da6/fd4 [0,4194304] 0 2024-04-22T23:45:54.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/866: link d5/da/d12/d37/d4e/l8e d5/da/d12/d1d/d38/l114 0 2024-04-22T23:45:54.201 INFO:tasks.workunit.client.0.smithi031.stdout:3/849: mknod d4/d6/d11/ddf/c126 0 2024-04-22T23:45:54.202 INFO:tasks.workunit.client.0.smithi031.stdout:1/956: symlink d1/de/d1e/d34/l145 0 2024-04-22T23:45:54.202 INFO:tasks.workunit.client.0.smithi031.stdout:0/923: chown d0/d1/d2/d36/d11/d1a/dfc/dad/dba/l114 679644 1 2024-04-22T23:45:54.204 INFO:tasks.workunit.client.0.smithi031.stdout:7/970: mkdir d9/d14/d1f/dc6/d103/d159 0 2024-04-22T23:45:54.204 INFO:tasks.workunit.client.0.smithi031.stdout:3/850: mknod d4/d6/d11/d25/c127 0 2024-04-22T23:45:54.205 INFO:tasks.workunit.client.0.smithi031.stdout:0/924: mknod d0/d13/d18/d1e/c129 0 2024-04-22T23:45:54.205 INFO:tasks.workunit.client.0.smithi031.stdout:7/971: readlink d9/d15/d3f/l96 0 2024-04-22T23:45:54.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/957: creat d1/d35/d8d/d102/f146 x:0 0 0 2024-04-22T23:45:54.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/867: truncate d5/da/d12/d2e/f7d 1969829 0 2024-04-22T23:45:54.208 INFO:tasks.workunit.client.0.smithi031.stdout:0/925: write d0/d13/d18/d1e/f6f [4618107,86084] 0 2024-04-22T23:45:54.208 INFO:tasks.workunit.client.0.smithi031.stdout:2/959: write d6/f8b [705489,50961] 0 2024-04-22T23:45:54.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/958: mknod d1/d3f/d54/dc6/c147 0 2024-04-22T23:45:54.213 INFO:tasks.workunit.client.0.smithi031.stdout:2/960: write d6/dc/d48/fe1 [4746734,102302] 0 2024-04-22T23:45:54.218 INFO:tasks.workunit.client.0.smithi031.stdout:1/959: read - d1/d99/d63/f140 zero size 2024-04-22T23:45:54.218 INFO:tasks.workunit.client.0.smithi031.stdout:0/926: dread - d0/d1/d2/d36/d11/f92 zero size 2024-04-22T23:45:54.222 INFO:tasks.workunit.client.0.smithi031.stdout:3/851: dwrite d4/d54/d2c/d89/d2e/ff6 [0,4194304] 0 2024-04-22T23:45:54.225 INFO:tasks.workunit.client.0.smithi031.stdout:8/868: dwrite d5/da/d12/fb6 [0,4194304] 0 2024-04-22T23:45:54.225 INFO:tasks.workunit.client.0.smithi031.stdout:3/852: chown d4/d54/d59/c5d 633824 1 2024-04-22T23:45:54.227 INFO:tasks.workunit.client.0.smithi031.stdout:2/961: fdatasync d6/dc/d48/d4b/d6b/de8/fc9 0 2024-04-22T23:45:54.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/972: dwrite d9/d14/d1f/d45/da6/daa/d10b/f10e [0,4194304] 0 2024-04-22T23:45:54.234 INFO:tasks.workunit.client.0.smithi031.stdout:0/927: rename d0/d13/d18/f6d to d0/f12a 0 2024-04-22T23:45:54.236 INFO:tasks.workunit.client.0.smithi031.stdout:7/973: mknod d9/d14/d1f/dc6/c15a 0 2024-04-22T23:45:54.239 INFO:tasks.workunit.client.0.smithi031.stdout:7/974: fdatasync d9/d14/d1f/d45/de3/f104 0 2024-04-22T23:45:54.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/960: dwrite d1/de/d1e/d34/f12c [0,4194304] 0 2024-04-22T23:45:54.250 INFO:tasks.workunit.client.0.smithi031.stdout:8/869: dwrite d5/da/d12/d2e/d7e/fb4 [0,4194304] 0 2024-04-22T23:45:54.253 INFO:tasks.workunit.client.0.smithi031.stdout:1/961: chown d1/d3f/d54/dba/d28/c49 270059 1 2024-04-22T23:45:54.253 INFO:tasks.workunit.client.0.smithi031.stdout:3/853: write d4/d54/d59/f6e [905430,37215] 0 2024-04-22T23:45:54.255 INFO:tasks.workunit.client.0.smithi031.stdout:2/962: truncate d6/dc/d42/d24/f34 2830920 0 2024-04-22T23:45:54.256 INFO:tasks.workunit.client.0.smithi031.stdout:0/928: truncate d0/d1/d2/d36/d11/d1a/ffa 1865306 0 2024-04-22T23:45:54.257 INFO:tasks.workunit.client.0.smithi031.stdout:3/854: creat d4/d6/da/f128 x:0 0 0 2024-04-22T23:45:54.258 INFO:tasks.workunit.client.0.smithi031.stdout:1/962: mkdir d1/db/d16/d1f/d52/d11d/d148 0 2024-04-22T23:45:54.263 INFO:tasks.workunit.client.0.smithi031.stdout:7/975: dread d9/f3b [0,4194304] 0 2024-04-22T23:45:54.266 INFO:tasks.workunit.client.0.smithi031.stdout:1/963: symlink d1/d3f/d54/dc6/d110/l149 0 2024-04-22T23:45:54.267 INFO:tasks.workunit.client.0.smithi031.stdout:7/976: dread - d9/d15/d3f/ddc/fc1 zero size 2024-04-22T23:45:54.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/929: dwrite d0/d13/d18/d1e/d25/d8e/dac/ff0 [0,4194304] 0 2024-04-22T23:45:54.275 INFO:tasks.workunit.client.0.smithi031.stdout:7/977: link d9/d15/dac/l131 d9/d14/d1f/d45/de3/d155/l15b 0 2024-04-22T23:45:54.276 INFO:tasks.workunit.client.0.smithi031.stdout:0/930: chown d0/d1/d2/d36/d11/d1c/f88 910945 1 2024-04-22T23:45:54.278 INFO:tasks.workunit.client.0.smithi031.stdout:7/978: symlink d9/d14/d1f/dc6/d103/d159/l15c 0 2024-04-22T23:45:54.281 INFO:tasks.workunit.client.0.smithi031.stdout:7/979: chown d9/fa 492576371 1 2024-04-22T23:45:54.286 INFO:tasks.workunit.client.0.smithi031.stdout:7/980: write d9/f4b [123195,58325] 0 2024-04-22T23:45:54.286 INFO:tasks.workunit.client.0.smithi031.stdout:8/870: truncate d5/da/d12/d37/d34/d6f/d5b/f84 711421 0 2024-04-22T23:45:54.288 INFO:tasks.workunit.client.0.smithi031.stdout:8/871: fdatasync d5/da/d12/d37/d34/d6f/d5b/dcb/fd0 0 2024-04-22T23:45:54.288 INFO:tasks.workunit.client.0.smithi031.stdout:1/964: truncate d1/d3f/d54/d132/f3e 3862473 0 2024-04-22T23:45:54.289 INFO:tasks.workunit.client.0.smithi031.stdout:7/981: write d9/d14/d1f/d45/da6/daa/d10b/f13c [273172,60017] 0 2024-04-22T23:45:54.291 INFO:tasks.workunit.client.0.smithi031.stdout:8/872: write d5/da/d12/d1d/d38/db9/fdc [1038820,36528] 0 2024-04-22T23:45:54.292 INFO:tasks.workunit.client.0.smithi031.stdout:1/965: symlink d1/de/d1e/d94/l14a 0 2024-04-22T23:45:54.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/982: creat d9/d14/d1f/d45/de3/f15d x:0 0 0 2024-04-22T23:45:54.294 INFO:tasks.workunit.client.0.smithi031.stdout:8/873: write d5/da/d12/f1c [820267,97596] 0 2024-04-22T23:45:54.294 INFO:tasks.workunit.client.0.smithi031.stdout:3/855: dread d4/f6a [0,4194304] 0 2024-04-22T23:45:54.299 INFO:tasks.workunit.client.0.smithi031.stdout:8/874: dread d5/d48/fba [0,4194304] 0 2024-04-22T23:45:54.299 INFO:tasks.workunit.client.0.smithi031.stdout:7/983: rename d9/d15/c32 to d9/d14/d132/c15e 0 2024-04-22T23:45:54.303 INFO:tasks.workunit.client.0.smithi031.stdout:7/984: dread - d9/d15/d6d/d65/dea/f121 zero size 2024-04-22T23:45:54.303 INFO:tasks.workunit.client.0.smithi031.stdout:0/931: dread d0/d1/d2/d36/d11/d1c/d54/f69 [0,4194304] 0 2024-04-22T23:45:54.303 INFO:tasks.workunit.client.0.smithi031.stdout:3/856: truncate d4/d54/d2c/d89/d2e/f39 1030463 0 2024-04-22T23:45:54.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/875: symlink d5/da/d12/d37/d4e/d101/l115 0 2024-04-22T23:45:54.307 INFO:tasks.workunit.client.0.smithi031.stdout:0/932: creat d0/d13/d18/d1e/d25/d8e/da7/dc9/d123/f12b x:0 0 0 2024-04-22T23:45:54.307 INFO:tasks.workunit.client.0.smithi031.stdout:3/857: truncate d4/d6/d11/d3e/fbe 270553 0 2024-04-22T23:45:54.307 INFO:tasks.workunit.client.0.smithi031.stdout:8/876: stat d5/da/d12/d1d/fca 0 2024-04-22T23:45:54.309 INFO:tasks.workunit.client.0.smithi031.stdout:7/985: dwrite d9/d15/f114 [0,4194304] 0 2024-04-22T23:45:54.311 INFO:tasks.workunit.client.0.smithi031.stdout:0/933: dread - d0/d1/d2/d36/d11/d1a/d56/f117 zero size 2024-04-22T23:45:54.314 INFO:tasks.workunit.client.0.smithi031.stdout:0/934: dread - d0/d1/d2/d36/d11/d1a/f113 zero size 2024-04-22T23:45:54.315 INFO:tasks.workunit.client.0.smithi031.stdout:1/966: dwrite d1/d35/d36/d108/fc4 [0,4194304] 0 2024-04-22T23:45:54.317 INFO:tasks.workunit.client.0.smithi031.stdout:3/858: rename d4/d54/d6d/d98/fea to d4/d54/d56/f129 0 2024-04-22T23:45:54.319 INFO:tasks.workunit.client.0.smithi031.stdout:7/986: creat d9/d15/dcf/f15f x:0 0 0 2024-04-22T23:45:54.322 INFO:tasks.workunit.client.0.smithi031.stdout:0/935: rename d0/db5/df6 to d0/d1/d2/d36/d11/d1a/d56/d12c 0 2024-04-22T23:45:54.323 INFO:tasks.workunit.client.0.smithi031.stdout:0/936: write d0/d1/d2/d36/d11/fe5 [524011,96012] 0 2024-04-22T23:45:54.324 INFO:tasks.workunit.client.0.smithi031.stdout:1/967: symlink d1/d3f/l14b 0 2024-04-22T23:45:54.326 INFO:tasks.workunit.client.0.smithi031.stdout:7/987: readlink d9/d14/d1f/l4d 0 2024-04-22T23:45:54.327 INFO:tasks.workunit.client.0.smithi031.stdout:3/859: creat d4/d6/d4c/ddb/f12a x:0 0 0 2024-04-22T23:45:54.328 INFO:tasks.workunit.client.0.smithi031.stdout:7/988: readlink d9/d14/d1f/d45/da6/daa/lba 0 2024-04-22T23:45:54.329 INFO:tasks.workunit.client.0.smithi031.stdout:0/937: truncate d0/d1/d2/d36/d11/d1a/d56/d12c/f112 767339 0 2024-04-22T23:45:54.331 INFO:tasks.workunit.client.0.smithi031.stdout:3/860: write d4/d6/d11/d25/d44/d7f/d5f/fb1 [458430,25822] 0 2024-04-22T23:45:54.333 INFO:tasks.workunit.client.0.smithi031.stdout:1/968: dwrite d1/db/d16/da0/f115 [0,4194304] 0 2024-04-22T23:45:54.334 INFO:tasks.workunit.client.0.smithi031.stdout:0/938: dread - d0/d1/d2/d36/d11/d1a/dfc/dad/f128 zero size 2024-04-22T23:45:54.334 INFO:tasks.workunit.client.0.smithi031.stdout:8/877: write d5/da/d12/d2e/f7d [2978715,52113] 0 2024-04-22T23:45:54.336 INFO:tasks.workunit.client.0.smithi031.stdout:7/989: write d9/f3e [4061126,1284] 0 2024-04-22T23:45:54.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/861: symlink d4/d54/d2c/d89/d2e/d8d/l12b 0 2024-04-22T23:45:54.338 INFO:tasks.workunit.client.0.smithi031.stdout:7/990: readlink d9/d14/d1f/d45/da6/l12f 0 2024-04-22T23:45:54.339 INFO:tasks.workunit.client.0.smithi031.stdout:7/991: fdatasync f6 0 2024-04-22T23:45:54.340 INFO:tasks.workunit.client.0.smithi031.stdout:0/939: mknod d0/d13/c12d 0 2024-04-22T23:45:54.341 INFO:tasks.workunit.client.0.smithi031.stdout:1/969: dread d1/de/d1e/d34/f12c [0,4194304] 0 2024-04-22T23:45:54.342 INFO:tasks.workunit.client.0.smithi031.stdout:8/878: mknod d5/d48/d61/d6b/d6c/c116 0 2024-04-22T23:45:54.343 INFO:tasks.workunit.client.0.smithi031.stdout:7/992: mkdir d9/d14/d1f/d7f/d160 0 2024-04-22T23:45:54.347 INFO:tasks.workunit.client.0.smithi031.stdout:8/879: creat d5/da/d12/d37/d4e/dc9/f117 x:0 0 0 2024-04-22T23:45:54.348 INFO:tasks.workunit.client.0.smithi031.stdout:7/993: read d9/d14/d1f/d45/da6/fd4 [19590,121037] 0 2024-04-22T23:45:54.352 INFO:tasks.workunit.client.0.smithi031.stdout:0/940: dwrite d0/fee [0,4194304] 0 2024-04-22T23:45:54.352 INFO:tasks.workunit.client.0.smithi031.stdout:1/970: dwrite d1/db/d16/f25 [0,4194304] 0 2024-04-22T23:45:54.358 INFO:tasks.workunit.client.0.smithi031.stdout:0/941: creat d0/d1/d2/dd8/f12e x:0 0 0 2024-04-22T23:45:54.359 INFO:tasks.workunit.client.0.smithi031.stdout:1/971: symlink d1/db/d16/da0/l14c 0 2024-04-22T23:45:54.367 INFO:tasks.workunit.client.0.smithi031.stdout:7/994: dwrite d9/d15/d6d/d65/f72 [0,4194304] 0 2024-04-22T23:45:54.370 INFO:tasks.workunit.client.0.smithi031.stdout:8/880: dwrite d5/da/df/f79 [0,4194304] 0 2024-04-22T23:45:54.370 INFO:tasks.workunit.client.0.smithi031.stdout:1/972: symlink d1/d35/d36/d108/l14d 0 2024-04-22T23:45:54.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/942: mkdir d0/d13/d18/d1e/d25/d12f 0 2024-04-22T23:45:54.373 INFO:tasks.workunit.client.0.smithi031.stdout:3/862: dwrite d4/d54/d2c/d89/f62 [0,4194304] 0 2024-04-22T23:45:54.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/943: chown d0/d1/d2/d36/d11/d1a/d3c/fd4 12 1 2024-04-22T23:45:54.379 INFO:tasks.workunit.client.0.smithi031.stdout:8/881: chown d5/da/d12/d2e/db0/dc3/de0/lf8 33527183 1 2024-04-22T23:45:54.380 INFO:tasks.workunit.client.0.smithi031.stdout:0/944: write d0/d1/d2/d17/f58 [3024342,43654] 0 2024-04-22T23:45:54.380 INFO:tasks.workunit.client.0.smithi031.stdout:1/973: rename d1/d3f/da1 to d1/d35/d36/d14e 0 2024-04-22T23:45:54.381 INFO:tasks.workunit.client.0.smithi031.stdout:7/995: symlink d9/d14/d22/d5c/d5f/l161 0 2024-04-22T23:45:54.383 INFO:tasks.workunit.client.0.smithi031.stdout:7/996: read - d9/d15/d6d/d65/dea/f156 zero size 2024-04-22T23:45:54.384 INFO:tasks.workunit.client.0.smithi031.stdout:0/945: creat d0/d13/d18/d1e/d25/dea/d105/f130 x:0 0 0 2024-04-22T23:45:54.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/974: symlink d1/db/d16/d1f/l14f 0 2024-04-22T23:45:54.387 INFO:tasks.workunit.client.0.smithi031.stdout:8/882: creat d5/da/df/d33/dd4/f118 x:0 0 0 2024-04-22T23:45:54.387 INFO:tasks.workunit.client.0.smithi031.stdout:0/946: creat d0/d1/d2/d36/d11/d1a/d56/f131 x:0 0 0 2024-04-22T23:45:54.391 INFO:tasks.workunit.client.0.smithi031.stdout:7/997: dwrite d9/d15/d3a/d41/f5a [4194304,4194304] 0 2024-04-22T23:45:54.392 INFO:tasks.workunit.client.0.smithi031.stdout:0/947: write d0/d1/d2/d36/d11/d1a/d3c/f3e [5023201,23488] 0 2024-04-22T23:45:54.393 INFO:tasks.workunit.client.0.smithi031.stdout:0/948: fsync d0/db5/fcf 0 2024-04-22T23:45:54.395 INFO:tasks.workunit.client.0.smithi031.stdout:8/883: link d5/da/d12/d1d/l22 d5/d48/d61/d6b/d6c/l119 0 2024-04-22T23:45:54.396 INFO:tasks.workunit.client.0.smithi031.stdout:0/949: truncate d0/f61 888041 0 2024-04-22T23:45:54.397 INFO:tasks.workunit.client.0.smithi031.stdout:8/884: write d5/da/d12/d37/d34/d6f/d5b/f74 [5165360,5434] 0 2024-04-22T23:45:54.398 INFO:tasks.workunit.client.0.smithi031.stdout:7/998: mkdir d9/d15/d3a/d41/d12d/d13b/d143/d162 0 2024-04-22T23:45:54.405 INFO:tasks.workunit.client.0.smithi031.stdout:7/999: write d9/d14/d1f/d45/de3/dfd/f14a [4194479,55606] 0 2024-04-22T23:45:54.405 INFO:tasks.workunit.client.0.smithi031.stdout:3/863: dread d4/d54/fdc [4194304,4194304] 0 2024-04-22T23:45:54.405 INFO:tasks.workunit.client.0.smithi031.stdout:8/885: fsync d5/da/d12/d37/d34/d6f/d5b/f84 0 2024-04-22T23:45:54.408 INFO:tasks.workunit.client.0.smithi031.stdout:0/950: dwrite d0/d1/d2/dd8/fd9 [0,4194304] 0 2024-04-22T23:45:54.408 INFO:tasks.workunit.client.0.smithi031.stdout:1/975: dread d1/de/d1e/f64 [0,4194304] 0 2024-04-22T23:45:54.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:54 smithi031 ceph-mon[26721]: pgmap v33: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 55 MiB/s rd, 160 MiB/s wr, 399 op/s 2024-04-22T23:45:54.409 INFO:tasks.workunit.client.0.smithi031.stdout:8/886: write d5/da/d12/d37/d34/f4c [3287775,52910] 0 2024-04-22T23:45:54.411 INFO:tasks.workunit.client.0.smithi031.stdout:3/864: creat d4/d6/d11/f12c x:0 0 0 2024-04-22T23:45:54.416 INFO:tasks.workunit.client.0.smithi031.stdout:0/951: mkdir d0/d13/d132 0 2024-04-22T23:45:54.419 INFO:tasks.workunit.client.0.smithi031.stdout:8/887: link d5/da/d83/d8d/c90 d5/da/d12/d2e/d4d/d56/c11a 0 2024-04-22T23:45:54.421 INFO:tasks.workunit.client.0.smithi031.stdout:0/952: mknod d0/d1/d2/d36/d11/d1a/dfc/dad/dba/de6/c133 0 2024-04-22T23:45:54.425 INFO:tasks.workunit.client.0.smithi031.stdout:0/953: mknod d0/d13/d18/d1e/d25/d8e/dac/dda/c134 0 2024-04-22T23:45:54.426 INFO:tasks.workunit.client.0.smithi031.stdout:8/888: dwrite d5/d48/d61/d6b/d6c/daa/fb8 [0,4194304] 0 2024-04-22T23:45:54.427 INFO:tasks.workunit.client.0.smithi031.stdout:8/889: truncate d5/f19 5752504 0 2024-04-22T23:45:54.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/954: creat d0/f135 x:0 0 0 2024-04-22T23:45:54.429 INFO:tasks.workunit.client.0.smithi031.stdout:8/890: fsync d5/da/d12/d37/d34/d6f/d5b/ffa 0 2024-04-22T23:45:54.431 INFO:tasks.workunit.client.0.smithi031.stdout:8/891: chown d5/da/d12/d37/d34/d6f/f40 63 1 2024-04-22T23:45:54.431 INFO:tasks.workunit.client.0.smithi031.stdout:0/955: mknod d0/d1/d2/d36/d11/c136 0 2024-04-22T23:45:54.432 INFO:tasks.workunit.client.0.smithi031.stdout:9/892: sync 2024-04-22T23:45:54.432 INFO:tasks.workunit.client.0.smithi031.stdout:2/963: sync 2024-04-22T23:45:54.432 INFO:tasks.workunit.client.0.smithi031.stdout:5/863: sync 2024-04-22T23:45:54.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/956: readlink d0/d1/d2/d36/l12 0 2024-04-22T23:45:54.435 INFO:tasks.workunit.client.0.smithi031.stdout:2/964: chown d6/dc/d9a/d6c/de4/fea 686 1 2024-04-22T23:45:54.439 INFO:tasks.workunit.client.0.smithi031.stdout:8/892: creat d5/da/d12/d2e/db2/f11b x:0 0 0 2024-04-22T23:45:54.440 INFO:tasks.workunit.client.0.smithi031.stdout:0/957: mkdir d0/d13/d18/dbf/dc3/d137 0 2024-04-22T23:45:54.449 INFO:tasks.workunit.client.0.smithi031.stdout:0/958: rmdir d0/d1/d2/d36 39 2024-04-22T23:45:54.449 INFO:tasks.workunit.client.0.smithi031.stdout:5/864: dwrite db/d8d/d11a/da4/ffa [4194304,4194304] 0 2024-04-22T23:45:54.452 INFO:tasks.workunit.client.0.smithi031.stdout:3/865: dread d4/d54/d2c/d89/d2e/f97 [0,4194304] 0 2024-04-22T23:45:54.454 INFO:tasks.workunit.client.0.smithi031.stdout:5/865: readlink db/d16/d1d/d24/d32/d62/laa 0 2024-04-22T23:45:54.454 INFO:tasks.workunit.client.0.smithi031.stdout:1/976: dread d1/db/d16/d1f/f26 [0,4194304] 0 2024-04-22T23:45:54.460 INFO:tasks.workunit.client.0.smithi031.stdout:1/977: write d1/d35/d36/d108/fc4 [4472291,12939] 0 2024-04-22T23:45:54.460 INFO:tasks.workunit.client.0.smithi031.stdout:8/893: rename d5/da/d12/d2e/d4d/l89 to d5/da/d12/d37/d34/d6f/d5b/l11c 0 2024-04-22T23:45:54.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:54 smithi044 ceph-mon[34463]: pgmap v33: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 55 MiB/s rd, 160 MiB/s wr, 399 op/s 2024-04-22T23:45:54.461 INFO:tasks.workunit.client.0.smithi031.stdout:0/959: creat d0/d1/d2/d36/d11/d1c/d54/f138 x:0 0 0 2024-04-22T23:45:54.462 INFO:tasks.workunit.client.0.smithi031.stdout:5/866: symlink db/d16/d2a/l11f 0 2024-04-22T23:45:54.463 INFO:tasks.workunit.client.0.smithi031.stdout:1/978: creat d1/de/d1e/d94/d125/d101/f150 x:0 0 0 2024-04-22T23:45:54.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/867: creat db/d16/d1d/d24/ddf/f120 x:0 0 0 2024-04-22T23:45:54.466 INFO:tasks.workunit.client.0.smithi031.stdout:1/979: write d1/de/d1e/d94/d125/d101/f150 [263148,62950] 0 2024-04-22T23:45:54.470 INFO:tasks.workunit.client.0.smithi031.stdout:1/980: mknod d1/d3f/d54/dba/c151 0 2024-04-22T23:45:54.471 INFO:tasks.workunit.client.0.smithi031.stdout:9/893: dread d2/f2e [0,4194304] 0 2024-04-22T23:45:54.474 INFO:tasks.workunit.client.0.smithi031.stdout:1/981: fdatasync d1/de/d1e/d34/d66/d124/f13e 0 2024-04-22T23:45:54.475 INFO:tasks.workunit.client.0.smithi031.stdout:1/982: stat d1/l55 0 2024-04-22T23:45:54.475 INFO:tasks.workunit.client.0.smithi031.stdout:5/868: dwrite db/d16/d2a/d7c/fa7 [0,4194304] 0 2024-04-22T23:45:54.477 INFO:tasks.workunit.client.0.smithi031.stdout:0/960: rename d0/d1/d2/d36/d11/d1c/d54 to d0/d1/d2/d36/d11/d1a/dfc/dad/dba/d139 0 2024-04-22T23:45:54.479 INFO:tasks.workunit.client.0.smithi031.stdout:9/894: creat d2/dc/f12b x:0 0 0 2024-04-22T23:45:54.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/983: rename d1/d35/d36/d108/d4e to d1/d35/d36/d14e/df3/d152 0 2024-04-22T23:45:54.481 INFO:tasks.workunit.client.0.smithi031.stdout:0/961: chown d0/d1/d2/cb7 4520 1 2024-04-22T23:45:54.481 INFO:tasks.workunit.client.0.smithi031.stdout:8/894: truncate d5/d48/d61/d6b/d6c/daa/fb8 516614 0 2024-04-22T23:45:54.481 INFO:tasks.workunit.client.0.smithi031.stdout:1/984: readlink d1/d35/d8d/d102/d81/l126 0 2024-04-22T23:45:54.482 INFO:tasks.workunit.client.0.smithi031.stdout:5/869: rename db/d16/d1d/d24/f76 to db/d16/d1d/d24/d32/d41/d115/f121 0 2024-04-22T23:45:54.483 INFO:tasks.workunit.client.0.smithi031.stdout:8/895: fsync d5/da/df/d33/fbd 0 2024-04-22T23:45:54.484 INFO:tasks.workunit.client.0.smithi031.stdout:5/870: fdatasync db/d16/d1d/f114 0 2024-04-22T23:45:54.484 INFO:tasks.workunit.client.0.smithi031.stdout:0/962: dread - d0/d1/d2/d36/d11/d1a/f100 zero size 2024-04-22T23:45:54.486 INFO:tasks.workunit.client.0.smithi031.stdout:5/871: readlink db/d8d/d9b/l10a 0 2024-04-22T23:45:54.488 INFO:tasks.workunit.client.0.smithi031.stdout:8/896: truncate d5/d48/d61/d6b/fd2 1025885 0 2024-04-22T23:45:54.490 INFO:tasks.workunit.client.0.smithi031.stdout:8/897: fdatasync d5/f76 0 2024-04-22T23:45:54.490 INFO:tasks.workunit.client.0.smithi031.stdout:9/895: dwrite d2/d15/d5a/fdc [0,4194304] 0 2024-04-22T23:45:54.493 INFO:tasks.workunit.client.0.smithi031.stdout:5/872: truncate db/d16/d1d/f6b 3982426 0 2024-04-22T23:45:54.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/898: fsync d5/f87 0 2024-04-22T23:45:54.498 INFO:tasks.workunit.client.0.smithi031.stdout:9/896: fdatasync d2/dc/d8b/d3d/d40/f5c 0 2024-04-22T23:45:54.499 INFO:tasks.workunit.client.0.smithi031.stdout:5/873: symlink db/d8d/l122 0 2024-04-22T23:45:54.501 INFO:tasks.workunit.client.0.smithi031.stdout:0/963: write d0/d13/d18/d1e/d25/d8e/fa8 [414302,110364] 0 2024-04-22T23:45:54.502 INFO:tasks.workunit.client.0.smithi031.stdout:8/899: rename c3 to d5/da/d12/d37/d34/d6f/d85/df1/c11d 0 2024-04-22T23:45:54.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/964: mkdir d0/d13/d18/dbf/d13a 0 2024-04-22T23:45:54.511 INFO:tasks.workunit.client.0.smithi031.stdout:9/897: getdents d2/dd/d75/d80 0 2024-04-22T23:45:54.512 INFO:tasks.workunit.client.0.smithi031.stdout:0/965: fdatasync d0/d1/d2/d36/d11/d1a/dfc/dad/dba/d139/f72 0 2024-04-22T23:45:54.513 INFO:tasks.workunit.client.0.smithi031.stdout:5/874: truncate db/d8d/d11a/da4/ffa 3726338 0 2024-04-22T23:45:54.515 INFO:tasks.workunit.client.0.smithi031.stdout:5/875: rmdir db/d16/d1d/d24/d32/d62 39 2024-04-22T23:45:54.517 INFO:tasks.workunit.client.0.smithi031.stdout:0/966: creat d0/d1/d2/d36/d11/d1a/dfc/f13b x:0 0 0 2024-04-22T23:45:54.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/967: mkdir d0/d1/d2/d17/d13c 0 2024-04-22T23:45:54.520 INFO:tasks.workunit.client.0.smithi031.stdout:5/876: link db/d16/d1d/d24/d27/l34 db/d16/d2a/d36/d74/d82/l123 0 2024-04-22T23:45:54.521 INFO:tasks.workunit.client.0.smithi031.stdout:8/900: dread d5/d48/d61/d6b/fa1 [0,4194304] 0 2024-04-22T23:45:54.522 INFO:tasks.workunit.client.0.smithi031.stdout:5/877: chown db/d16/d2a/d36/d74/d82/f117 14674 1 2024-04-22T23:45:54.522 INFO:tasks.workunit.client.0.smithi031.stdout:0/968: creat d0/d13/f13d x:0 0 0 2024-04-22T23:45:54.526 INFO:tasks.workunit.client.0.smithi031.stdout:0/969: creat d0/d13/d18/dbf/dc3/f13e x:0 0 0 2024-04-22T23:45:54.530 INFO:tasks.workunit.client.0.smithi031.stdout:8/901: dwrite d5/da/d12/d2e/d4d/f10b [0,4194304] 0 2024-04-22T23:45:54.533 INFO:tasks.workunit.client.0.smithi031.stdout:5/878: write db/d16/d1d/d24/d58/d5d/f92 [431082,55995] 0 2024-04-22T23:45:54.534 INFO:tasks.workunit.client.0.smithi031.stdout:8/902: write d5/f19 [3072043,113338] 0 2024-04-22T23:45:54.537 INFO:tasks.workunit.client.0.smithi031.stdout:5/879: getdents db/d8d/d11a/d116 0 2024-04-22T23:45:54.539 INFO:tasks.workunit.client.0.smithi031.stdout:0/970: dwrite d0/d1/d3f/fb0 [0,4194304] 0 2024-04-22T23:45:54.548 INFO:tasks.workunit.client.0.smithi031.stdout:8/903: write d5/da/d12/d95/f103 [689658,28804] 0 2024-04-22T23:45:54.549 INFO:tasks.workunit.client.0.smithi031.stdout:5/880: dread db/d16/f8b [0,4194304] 0 2024-04-22T23:45:54.551 INFO:tasks.workunit.client.0.smithi031.stdout:5/881: rmdir db/d8d/d11a/da4 39 2024-04-22T23:45:54.553 INFO:tasks.workunit.client.0.smithi031.stdout:8/904: rename d5/fac to d5/da/d12/d2e/d4d/d9a/db1/f11e 0 2024-04-22T23:45:54.554 INFO:tasks.workunit.client.0.smithi031.stdout:5/882: mknod db/d16/d1d/d24/c124 0 2024-04-22T23:45:54.555 INFO:tasks.workunit.client.0.smithi031.stdout:8/905: mkdir d5/da/d12/d2e/db0/d11f 0 2024-04-22T23:45:54.557 INFO:tasks.workunit.client.0.smithi031.stdout:8/906: truncate d5/da/d12/d37/d4e/dc9/f108 344684 0 2024-04-22T23:45:54.558 INFO:tasks.workunit.client.0.smithi031.stdout:5/883: rename db/d16/d2a/d36/d74/d82/d86 to db/d8d/df1/d125 0 2024-04-22T23:45:54.559 INFO:tasks.workunit.client.0.smithi031.stdout:5/884: read - db/d16/d2a/d36/d74/d82/f117 zero size 2024-04-22T23:45:54.560 INFO:tasks.workunit.client.0.smithi031.stdout:8/907: write d5/da/d12/d2e/d4d/d56/f88 [5072771,47791] 0 2024-04-22T23:45:54.561 INFO:tasks.workunit.client.0.smithi031.stdout:5/885: creat db/d8d/d9b/f126 x:0 0 0 2024-04-22T23:45:54.562 INFO:tasks.workunit.client.0.smithi031.stdout:5/886: chown db/d8d/d11a/l7a 3475 1 2024-04-22T23:45:54.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/971: dread d0/d13/f67 [0,4194304] 0 2024-04-22T23:45:54.567 INFO:tasks.workunit.client.0.smithi031.stdout:0/972: link d0/d1/d3f/f93 d0/d1/d2/d36/d11/d1a/dfc/dad/dba/f13f 0 2024-04-22T23:45:54.569 INFO:tasks.workunit.client.0.smithi031.stdout:5/887: dwrite db/d16/d2a/d36/fdb [0,4194304] 0 2024-04-22T23:45:54.569 INFO:tasks.workunit.client.0.smithi031.stdout:0/973: write d0/d13/d18/d1e/f6f [1014357,36071] 0 2024-04-22T23:45:54.571 INFO:tasks.workunit.client.0.smithi031.stdout:0/974: chown d0/d1/d2/d36/d11/c3b 187 1 2024-04-22T23:45:54.573 INFO:tasks.workunit.client.0.smithi031.stdout:0/975: fsync d0/d1/d2/d36/d11/d1a/dfc/dad/dba/d139/f138 0 2024-04-22T23:45:54.575 INFO:tasks.workunit.client.0.smithi031.stdout:0/976: write d0/d1/d2/d36/d11/d1a/dfc/f13b [727339,12986] 0 2024-04-22T23:45:54.581 INFO:tasks.workunit.client.0.smithi031.stdout:5/888: dwrite db/d16/f65 [0,4194304] 0 2024-04-22T23:45:54.582 INFO:tasks.workunit.client.0.smithi031.stdout:5/889: chown db/d8d/df1/d125/f95 6 1 2024-04-22T23:45:54.583 INFO:tasks.workunit.client.0.smithi031.stdout:5/890: fsync db/d16/fc5 0 2024-04-22T23:45:54.590 INFO:tasks.workunit.client.0.smithi031.stdout:5/891: dwrite db/d16/d2a/d59/da8/fac [0,4194304] 0 2024-04-22T23:45:54.668 INFO:tasks.workunit.client.0.smithi031.stdout:2/965: sync 2024-04-22T23:45:54.670 INFO:tasks.workunit.client.0.smithi031.stdout:2/966: write d6/dc/d9a/fb9 [4761785,127603] 0 2024-04-22T23:45:54.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/967: creat d6/dc/d57/f134 x:0 0 0 2024-04-22T23:45:54.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/968: creat d6/dc/d42/d24/f135 x:0 0 0 2024-04-22T23:45:54.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/969: symlink d6/dc/d57/d52/db5/l136 0 2024-04-22T23:45:54.678 INFO:tasks.workunit.client.0.smithi031.stdout:2/970: fdatasync d6/dc/d9a/d6c/db2/df0/fd8 0 2024-04-22T23:45:54.680 INFO:tasks.workunit.client.0.smithi031.stdout:2/971: mkdir d6/dc/d48/d4b/d137 0 2024-04-22T23:45:54.681 INFO:tasks.workunit.client.0.smithi031.stdout:2/972: mknod d6/dc/d9a/d6c/d86/c138 0 2024-04-22T23:45:54.683 INFO:tasks.workunit.client.0.smithi031.stdout:2/973: mknod d6/dc/d48/d4b/d6b/c139 0 2024-04-22T23:45:54.684 INFO:tasks.workunit.client.0.smithi031.stdout:2/974: fdatasync d6/dc/d42/fa0 0 2024-04-22T23:45:54.686 INFO:tasks.workunit.client.0.smithi031.stdout:2/975: chown d6/dc/d9a/d6c/db2/df0/d94/dab/d11a 7684168 1 2024-04-22T23:45:54.696 INFO:tasks.workunit.client.0.smithi031.stdout:2/976: dread d6/dc/d42/d24/d12b/d85/da9/fad [0,4194304] 0 2024-04-22T23:45:54.702 INFO:tasks.workunit.client.0.smithi031.stdout:2/977: dwrite d6/dc/d48/d4b/d6b/de8/de6/f104 [0,4194304] 0 2024-04-22T23:45:54.703 INFO:tasks.workunit.client.0.smithi031.stdout:2/978: fdatasync d6/dc/d48/d8a/fef 0 2024-04-22T23:45:54.705 INFO:tasks.workunit.client.0.smithi031.stdout:2/979: symlink d6/dc/d48/d8a/d92/l13a 0 2024-04-22T23:45:54.709 INFO:tasks.workunit.client.0.smithi031.stdout:2/980: rename d6/f8b to d6/dc/d57/d52/db5/f13b 0 2024-04-22T23:45:54.710 INFO:tasks.workunit.client.0.smithi031.stdout:2/981: dread d6/d45/d50/fed [0,4194304] 0 2024-04-22T23:45:54.712 INFO:tasks.workunit.client.0.smithi031.stdout:2/982: write d6/dc/d42/d24/f39 [4315505,65140] 0 2024-04-22T23:45:54.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/983: read d6/dc/d48/d4b/d6b/de8/de6/f104 [3135736,50353] 0 2024-04-22T23:45:54.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/984: read - d6/dc/d9a/d6c/d86/fb7 zero size 2024-04-22T23:45:54.717 INFO:tasks.workunit.client.0.smithi031.stdout:2/985: creat d6/d45/f13c x:0 0 0 2024-04-22T23:45:54.720 INFO:tasks.workunit.client.0.smithi031.stdout:2/986: rename d6/dc/d9a/d6c/db2/f12c to d6/dc/d9a/d6c/db2/df0/d94/f13d 0 2024-04-22T23:45:54.724 INFO:tasks.workunit.client.0.smithi031.stdout:2/987: mkdir d6/dc/d9a/d6c/db2/df0/d94/dab/d11a/d13e 0 2024-04-22T23:45:54.726 INFO:tasks.workunit.client.0.smithi031.stdout:2/988: mknod d6/d45/d50/c13f 0 2024-04-22T23:45:54.730 INFO:tasks.workunit.client.0.smithi031.stdout:2/989: dwrite d6/f16 [0,4194304] 0 2024-04-22T23:45:54.731 INFO:tasks.workunit.client.0.smithi031.stdout:2/990: readlink d6/dc/d48/d8a/d92/l13a 0 2024-04-22T23:45:54.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/991: readlink d6/dc/d9a/d6c/db2/df0/ld9 0 2024-04-22T23:45:54.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/992: dwrite d6/dc/f4e [0,4194304] 0 2024-04-22T23:45:54.740 INFO:tasks.workunit.client.0.smithi031.stdout:2/993: read d6/dc/d17/d69/f7f [2260700,91177] 0 2024-04-22T23:45:54.748 INFO:tasks.workunit.client.0.smithi031.stdout:2/994: dwrite d6/dc/d57/f132 [0,4194304] 0 2024-04-22T23:45:54.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/995: link d6/dc/d57/c3b d6/d45/d50/c140 0 2024-04-22T23:45:54.757 INFO:tasks.workunit.client.0.smithi031.stdout:2/996: rmdir d6/dc/d42/d24/d12b/d114 0 2024-04-22T23:45:54.768 INFO:tasks.workunit.client.0.smithi031.stdout:2/997: dread d6/dc/d42/d24/d12b/d85/da9/d10a/fd3 [0,4194304] 0 2024-04-22T23:45:54.770 INFO:tasks.workunit.client.0.smithi031.stdout:2/998: mknod d6/d45/d50/c141 0 2024-04-22T23:45:54.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/999: rename d6/d45/d50 to d6/d45/d50/d142 22 2024-04-22T23:45:54.856 INFO:tasks.workunit.client.0.smithi031.stdout:3/866: sync 2024-04-22T23:45:54.868 INFO:tasks.workunit.client.0.smithi031.stdout:3/867: dread d4/d54/d2c/d89/f1b [0,4194304] 0 2024-04-22T23:45:54.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/868: mkdir d4/d54/d85/dc1/d12d 0 2024-04-22T23:45:54.875 INFO:tasks.workunit.client.0.smithi031.stdout:3/869: unlink d4/d83/dd8/de6/ff5 0 2024-04-22T23:45:54.879 INFO:tasks.workunit.client.0.smithi031.stdout:3/870: rename d4/d54/d56/daf to d4/d6/d11/d25/d44/d7f/d5f/d12e 0 2024-04-22T23:45:54.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/871: symlink d4/d54/d6d/d98/l12f 0 2024-04-22T23:45:54.882 INFO:tasks.workunit.client.0.smithi031.stdout:3/872: write d4/d54/fdc [7535609,121715] 0 2024-04-22T23:45:54.884 INFO:tasks.workunit.client.0.smithi031.stdout:3/873: chown d4/d6/d22/le7 17098705 1 2024-04-22T23:45:54.892 INFO:tasks.workunit.client.0.smithi031.stdout:3/874: truncate d4/d54/fdc 8146947 0 2024-04-22T23:45:54.900 INFO:tasks.workunit.client.0.smithi031.stdout:3/875: dwrite d4/d83/fab [0,4194304] 0 2024-04-22T23:45:54.901 INFO:tasks.workunit.client.0.smithi031.stdout:3/876: chown d4/d6/d22 1063763667 1 2024-04-22T23:45:54.903 INFO:tasks.workunit.client.0.smithi031.stdout:3/877: rmdir d4/d54/d88/da2/de0/d10b 39 2024-04-22T23:45:54.904 INFO:tasks.workunit.client.0.smithi031.stdout:3/878: stat d4/d6/d4c/ddb/f12a 0 2024-04-22T23:45:54.910 INFO:tasks.workunit.client.0.smithi031.stdout:3/879: dwrite d4/d6/d11/d3e/f5a [0,4194304] 0 2024-04-22T23:45:54.919 INFO:tasks.workunit.client.0.smithi031.stdout:3/880: truncate d4/d83/f8b 4325231 0 2024-04-22T23:45:54.919 INFO:tasks.workunit.client.0.smithi031.stdout:3/881: chown d4/d6/d11/d25/d44/l81 689811 1 2024-04-22T23:45:54.921 INFO:tasks.workunit.client.0.smithi031.stdout:3/882: mknod d4/d54/d59/c130 0 2024-04-22T23:45:54.925 INFO:tasks.workunit.client.0.smithi031.stdout:3/883: truncate d4/d6/d22/feb 1771768 0 2024-04-22T23:45:54.927 INFO:tasks.workunit.client.0.smithi031.stdout:3/884: mkdir d4/d6/d11/dc0/d131 0 2024-04-22T23:45:54.933 INFO:tasks.workunit.client.0.smithi031.stdout:3/885: getdents d4/d54/d56 0 2024-04-22T23:45:54.984 INFO:tasks.workunit.client.0.smithi031.stdout:9/898: sync 2024-04-22T23:45:54.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/985: sync 2024-04-22T23:45:54.985 INFO:tasks.workunit.client.0.smithi031.stdout:9/899: chown d2/fb8 6059 1 2024-04-22T23:45:54.986 INFO:tasks.workunit.client.0.smithi031.stdout:9/900: chown d2/d15/d5a/c96 2347079 1 2024-04-22T23:45:54.993 INFO:tasks.workunit.client.0.smithi031.stdout:3/886: write d4/d54/d6d/f82 [959887,16966] 0 2024-04-22T23:45:55.027 INFO:tasks.workunit.client.0.smithi031.stdout:1/986: write d1/d3f/d54/dc6/fef [409499,75288] 0 2024-04-22T23:45:55.029 INFO:tasks.workunit.client.0.smithi031.stdout:9/901: symlink d2/dc/d3e/da1/l12c 0 2024-04-22T23:45:55.036 INFO:tasks.workunit.client.0.smithi031.stdout:9/902: dread d2/dc/f25 [0,4194304] 0 2024-04-22T23:45:55.062 INFO:tasks.workunit.client.0.smithi031.stdout:9/903: truncate d2/dd/d75/f5e 3789086 0 2024-04-22T23:45:55.063 INFO:tasks.workunit.client.0.smithi031.stdout:9/904: chown d2/f2e 0 1 2024-04-22T23:45:55.067 INFO:tasks.workunit.client.0.smithi031.stdout:9/905: link d2/dc/d8b/d3d/d40/da4/lbb d2/dd/d75/d4a/da0/dd0/l12d 0 2024-04-22T23:45:55.068 INFO:tasks.workunit.client.0.smithi031.stdout:9/906: write d2/f2e [2595915,11644] 0 2024-04-22T23:45:55.166 INFO:tasks.workunit.client.0.smithi031.stdout:8/908: sync 2024-04-22T23:45:55.167 INFO:tasks.workunit.client.0.smithi031.stdout:0/977: sync 2024-04-22T23:45:55.174 INFO:tasks.workunit.client.0.smithi031.stdout:0/978: creat d0/d1/d2/f140 x:0 0 0 2024-04-22T23:45:55.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/979: creat d0/d1/d2/d36/d11/d1c/f141 x:0 0 0 2024-04-22T23:45:55.183 INFO:tasks.workunit.client.0.smithi031.stdout:8/909: dread d5/da/d12/d1d/d38/f3f [0,4194304] 0 2024-04-22T23:45:55.196 INFO:tasks.workunit.client.0.smithi031.stdout:0/980: creat d0/d13/d18/d1e/d25/d8e/dac/f142 x:0 0 0 2024-04-22T23:45:55.197 INFO:tasks.workunit.client.0.smithi031.stdout:8/910: dread d5/d48/d61/d6b/d6c/f70 [0,4194304] 0 2024-04-22T23:45:55.198 INFO:tasks.workunit.client.0.smithi031.stdout:0/981: write d0/d13/d18/d1e/d25/d8e/da7/dc9/d123/f12b [803477,46251] 0 2024-04-22T23:45:55.202 INFO:tasks.workunit.client.0.smithi031.stdout:5/892: sync 2024-04-22T23:45:55.205 INFO:tasks.workunit.client.0.smithi031.stdout:8/911: getdents d5/d48/d61 0 2024-04-22T23:45:55.207 INFO:tasks.workunit.client.0.smithi031.stdout:5/893: creat db/d16/d1d/d24/d27/f127 x:0 0 0 2024-04-22T23:45:55.208 INFO:tasks.workunit.client.0.smithi031.stdout:8/912: dread - d5/da/d12/d37/fa6 zero size 2024-04-22T23:45:55.210 INFO:tasks.workunit.client.0.smithi031.stdout:5/894: creat db/d8d/d11a/d116/f128 x:0 0 0 2024-04-22T23:45:55.213 INFO:tasks.workunit.client.0.smithi031.stdout:5/895: creat db/d8d/df1/f129 x:0 0 0 2024-04-22T23:45:55.219 INFO:tasks.workunit.client.0.smithi031.stdout:5/896: rename db/d16/d1d/d24/d7d/leb to db/d16/d1d/d24/d32/d62/l12a 0 2024-04-22T23:45:55.222 INFO:tasks.workunit.client.0.smithi031.stdout:5/897: fsync db/d16/f54 0 2024-04-22T23:45:55.223 INFO:tasks.workunit.client.0.smithi031.stdout:8/913: dread d5/da/d12/d37/d4e/fc4 [0,4194304] 0 2024-04-22T23:45:55.257 INFO:tasks.workunit.client.0.smithi031.stdout:0/982: write d0/d1/f6b [3329717,109008] 0 2024-04-22T23:45:55.267 INFO:tasks.workunit.client.0.smithi031.stdout:3/887: sync 2024-04-22T23:45:55.267 INFO:tasks.workunit.client.0.smithi031.stdout:1/987: sync 2024-04-22T23:45:55.268 INFO:tasks.workunit.client.0.smithi031.stdout:8/914: dwrite d5/da/d12/d37/d34/d6f/d5b/f67 [0,4194304] 0 2024-04-22T23:45:55.269 INFO:tasks.workunit.client.0.smithi031.stdout:9/907: symlink d2/dc/l12e 0 2024-04-22T23:45:55.271 INFO:tasks.workunit.client.0.smithi031.stdout:1/988: fdatasync d1/db/d16/fd1 0 2024-04-22T23:45:55.272 INFO:tasks.workunit.client.0.smithi031.stdout:5/898: mkdir db/d16/d1d/d24/d12b 0 2024-04-22T23:45:55.273 INFO:tasks.workunit.client.0.smithi031.stdout:9/908: mkdir d2/dd/d75/df7/d12f 0 2024-04-22T23:45:55.273 INFO:tasks.workunit.client.0.smithi031.stdout:0/983: write d0/fbe [2501597,69333] 0 2024-04-22T23:45:55.273 INFO:tasks.workunit.client.0.smithi031.stdout:1/989: read d1/d3f/d54/dba/f131 [256252,93881] 0 2024-04-22T23:45:55.282 INFO:tasks.workunit.client.0.smithi031.stdout:9/909: unlink d2/dd/d75/d64/d5d/f89 0 2024-04-22T23:45:55.284 INFO:tasks.workunit.client.0.smithi031.stdout:5/899: mkdir db/d8d/d11a/d12c 0 2024-04-22T23:45:55.284 INFO:tasks.workunit.client.0.smithi031.stdout:1/990: getdents d1/de/d107 0 2024-04-22T23:45:55.285 INFO:tasks.workunit.client.0.smithi031.stdout:8/915: getdents d5/da/d12/d95 0 2024-04-22T23:45:55.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/888: dwrite d4/d6/d11/d25/d44/d7f/da5/fce [0,4194304] 0 2024-04-22T23:45:55.289 INFO:tasks.workunit.client.0.smithi031.stdout:8/916: truncate d5/da/d12/d2e/d4d/fbc 664612 0 2024-04-22T23:45:55.290 INFO:tasks.workunit.client.0.smithi031.stdout:1/991: creat d1/de/d1e/d94/d125/d101/f153 x:0 0 0 2024-04-22T23:45:55.292 INFO:tasks.workunit.client.0.smithi031.stdout:3/889: creat d4/d54/d2c/d89/d2e/d8d/d108/f132 x:0 0 0 2024-04-22T23:45:55.293 INFO:tasks.workunit.client.0.smithi031.stdout:8/917: symlink d5/da/d12/d37/d34/d6f/d85/d9b/l120 0 2024-04-22T23:45:55.296 INFO:tasks.workunit.client.0.smithi031.stdout:0/984: write d0/d1/d2/db4/de4/ff8 [1063036,75564] 0 2024-04-22T23:45:55.296 INFO:tasks.workunit.client.0.smithi031.stdout:3/890: mkdir d4/d6/d11/d25/d133 0 2024-04-22T23:45:55.297 INFO:tasks.workunit.client.0.smithi031.stdout:0/985: chown d0/d1/d2/d17/c1b 5 1 2024-04-22T23:45:55.297 INFO:tasks.workunit.client.0.smithi031.stdout:0/986: readlink d0/d1/d3f/lce 0 2024-04-22T23:45:55.300 INFO:tasks.workunit.client.0.smithi031.stdout:0/987: write d0/d13/d18/d1e/d25/d8e/dac/ff0 [1130384,94947] 0 2024-04-22T23:45:55.300 INFO:tasks.workunit.client.0.smithi031.stdout:3/891: rmdir d4 39 2024-04-22T23:45:55.302 INFO:tasks.workunit.client.0.smithi031.stdout:1/992: link d1/db/d16/d1f/l14f d1/d99/d7d/l154 0 2024-04-22T23:45:55.305 INFO:tasks.workunit.client.0.smithi031.stdout:3/892: chown d4/d54/l121 40969105 1 2024-04-22T23:45:55.307 INFO:tasks.workunit.client.0.smithi031.stdout:0/988: creat d0/d13/d18/d1e/d25/d8e/dac/f143 x:0 0 0 2024-04-22T23:45:55.308 INFO:tasks.workunit.client.0.smithi031.stdout:9/910: dwrite d2/d15/d5a/fa3 [0,4194304] 0 2024-04-22T23:45:55.310 INFO:tasks.workunit.client.0.smithi031.stdout:1/993: creat d1/db/d16/d1f/d52/d11d/d148/f155 x:0 0 0 2024-04-22T23:45:55.310 INFO:tasks.workunit.client.0.smithi031.stdout:5/900: dwrite db/d16/d1d/d24/ddf/fe9 [0,4194304] 0 2024-04-22T23:45:55.313 INFO:tasks.workunit.client.0.smithi031.stdout:0/989: write d0/d1/d2/d36/d11/d1a/f100 [162164,89510] 0 2024-04-22T23:45:55.315 INFO:tasks.workunit.client.0.smithi031.stdout:9/911: fdatasync d2/dd/d75/d64/d5d/d59/f78 0 2024-04-22T23:45:55.318 INFO:tasks.workunit.client.0.smithi031.stdout:0/990: write d0/f4f [51973,28354] 0 2024-04-22T23:45:55.318 INFO:tasks.workunit.client.0.smithi031.stdout:5/901: stat db/d16/d1d/d24/d32/d41/d104 0 2024-04-22T23:45:55.322 INFO:tasks.workunit.client.0.smithi031.stdout:0/991: write d0/fee [4148835,78071] 0 2024-04-22T23:45:55.325 INFO:tasks.workunit.client.0.smithi031.stdout:3/893: dwrite d4/d6/d11/d25/d44/f8e [0,4194304] 0 2024-04-22T23:45:55.326 INFO:tasks.workunit.client.0.smithi031.stdout:8/918: dwrite d5/da/df/d33/f98 [0,4194304] 0 2024-04-22T23:45:55.331 INFO:tasks.workunit.client.0.smithi031.stdout:5/902: fsync db/d16/d1d/d24/d32/d62/f67 0 2024-04-22T23:45:55.331 INFO:tasks.workunit.client.0.smithi031.stdout:3/894: dread - d4/d54/d56/f129 zero size 2024-04-22T23:45:55.331 INFO:tasks.workunit.client.0.smithi031.stdout:8/919: rmdir d5/d48/d61/d6b/d6c 39 2024-04-22T23:45:55.333 INFO:tasks.workunit.client.0.smithi031.stdout:5/903: write db/d16/d2a/d59/da8/fac [3983707,116338] 0 2024-04-22T23:45:55.333 INFO:tasks.workunit.client.0.smithi031.stdout:0/992: symlink d0/d13/d18/dbf/d13a/l144 0 2024-04-22T23:45:55.335 INFO:tasks.workunit.client.0.smithi031.stdout:3/895: mknod d4/d54/d56/dae/c134 0 2024-04-22T23:45:55.336 INFO:tasks.workunit.client.0.smithi031.stdout:1/994: dwrite d1/de/d1e/d94/d125/d101/f153 [0,4194304] 0 2024-04-22T23:45:55.342 INFO:tasks.workunit.client.0.smithi031.stdout:8/920: dwrite d5/da/d12/d37/d4e/fb5 [0,4194304] 0 2024-04-22T23:45:55.343 INFO:tasks.workunit.client.0.smithi031.stdout:8/921: stat d5/da/d12/d37/d4e/fc4 0 2024-04-22T23:45:55.345 INFO:tasks.workunit.client.0.smithi031.stdout:8/922: mknod d5/da/d12/d37/d4e/d101/c121 0 2024-04-22T23:45:55.430 INFO:tasks.workunit.client.0.smithi031.stdout:5/904: dread db/d16/d2a/d36/d74/fbc [0,4194304] 0 2024-04-22T23:45:55.437 INFO:tasks.workunit.client.0.smithi031.stdout:8/923: dwrite d5/da/d12/d37/d34/d6f/d5b/f84 [0,4194304] 0 2024-04-22T23:45:55.439 INFO:tasks.workunit.client.0.smithi031.stdout:8/924: fdatasync d5/da/d12/d2e/db0/ffc 0 2024-04-22T23:45:55.444 INFO:tasks.workunit.client.0.smithi031.stdout:8/925: read d5/da/d12/f44 [21934,101013] 0 2024-04-22T23:45:55.446 INFO:tasks.workunit.client.0.smithi031.stdout:3/896: dread d4/d54/d2c/d89/f91 [0,4194304] 0 2024-04-22T23:45:55.446 INFO:tasks.workunit.client.0.smithi031.stdout:5/905: write db/d16/d1d/d24/d32/d41/fd0 [468452,98008] 0 2024-04-22T23:45:55.449 INFO:tasks.workunit.client.0.smithi031.stdout:5/906: chown db/f29 223887074 1 2024-04-22T23:45:55.450 INFO:tasks.workunit.client.0.smithi031.stdout:1/995: dread d1/d35/d36/d108/feb [0,4194304] 0 2024-04-22T23:45:55.450 INFO:tasks.workunit.client.0.smithi031.stdout:3/897: chown d4/d54/d88/da2/cb6 45430 1 2024-04-22T23:45:55.451 INFO:tasks.workunit.client.0.smithi031.stdout:3/898: chown d4/d54/d85/dc1 194777974 1 2024-04-22T23:45:55.451 INFO:tasks.workunit.client.0.smithi031.stdout:8/926: link d5/d48/d61/d6b/fd2 d5/da/d12/d37/d34/d6f/d85/f122 0 2024-04-22T23:45:55.453 INFO:tasks.workunit.client.0.smithi031.stdout:1/996: mknod d1/d35/d8d/d102/c156 0 2024-04-22T23:45:55.459 INFO:tasks.workunit.client.0.smithi031.stdout:8/927: link d5/da/d12/d2e/d4d/d9a/db1/f11e d5/da/d83/f123 0 2024-04-22T23:45:55.460 INFO:tasks.workunit.client.0.smithi031.stdout:8/928: fdatasync d5/da/d12/d37/fa6 0 2024-04-22T23:45:55.465 INFO:tasks.workunit.client.0.smithi031.stdout:8/929: link d5/da/d12/d37/d4e/fc4 d5/da/d12/d37/d34/d6f/d85/f124 0 2024-04-22T23:45:55.466 INFO:tasks.workunit.client.0.smithi031.stdout:1/997: write d1/d3f/fd4 [607819,110809] 0 2024-04-22T23:45:55.468 INFO:tasks.workunit.client.0.smithi031.stdout:5/907: dwrite db/d16/d1d/d24/d27/d9e/fa1 [0,4194304] 0 2024-04-22T23:45:55.472 INFO:tasks.workunit.client.0.smithi031.stdout:3/899: dwrite d4/d6/d11/d25/d44/d7f/d5f/fe5 [0,4194304] 0 2024-04-22T23:45:55.476 INFO:tasks.workunit.client.0.smithi031.stdout:1/998: creat d1/d35/d36/d108/d4b/d6f/da6/f157 x:0 0 0 2024-04-22T23:45:55.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/908: symlink db/d16/d1d/d24/d27/d9e/l12d 0 2024-04-22T23:45:55.480 INFO:tasks.workunit.client.0.smithi031.stdout:1/999: symlink d1/d35/d8d/d102/dc1/l158 0 2024-04-22T23:45:55.480 INFO:tasks.workunit.client.0.smithi031.stdout:8/930: dwrite d5/da/d12/d37/d34/d6f/d5b/dcb/fd0 [0,4194304] 0 2024-04-22T23:45:55.484 INFO:tasks.workunit.client.0.smithi031.stdout:8/931: truncate d5/da/d12/d37/d4e/dc9/f108 434137 0 2024-04-22T23:45:55.485 INFO:tasks.workunit.client.0.smithi031.stdout:5/909: link db/d8d/d11a/l7a db/d16/d2a/d36/d11d/l12e 0 2024-04-22T23:45:55.487 INFO:tasks.workunit.client.0.smithi031.stdout:8/932: creat d5/da/df/f125 x:0 0 0 2024-04-22T23:45:55.490 INFO:tasks.workunit.client.0.smithi031.stdout:8/933: symlink d5/da/d12/d37/d34/d6f/d85/df1/l126 0 2024-04-22T23:45:55.491 INFO:tasks.workunit.client.0.smithi031.stdout:8/934: stat d5/da/d12/d2e/db2/f11b 0 2024-04-22T23:45:55.507 INFO:tasks.workunit.client.0.smithi031.stdout:5/910: dread db/d16/d1d/f20 [0,4194304] 0 2024-04-22T23:45:55.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/911: write db/d16/d1d/d24/d27/f127 [148232,22786] 0 2024-04-22T23:45:55.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/912: mknod db/d16/d2a/d36/d11d/c12f 0 2024-04-22T23:45:55.511 INFO:tasks.workunit.client.0.smithi031.stdout:8/935: dread d5/da/d12/d2e/d4d/d56/fd3 [0,4194304] 0 2024-04-22T23:45:55.512 INFO:tasks.workunit.client.0.smithi031.stdout:8/936: chown d5/d48/l86 0 1 2024-04-22T23:45:55.514 INFO:tasks.workunit.client.0.smithi031.stdout:5/913: fsync db/d16/d1d/f6b 0 2024-04-22T23:45:55.515 INFO:tasks.workunit.client.0.smithi031.stdout:8/937: creat d5/da/d12/d2e/db2/f127 x:0 0 0 2024-04-22T23:45:55.517 INFO:tasks.workunit.client.0.smithi031.stdout:8/938: creat d5/da/d83/f128 x:0 0 0 2024-04-22T23:45:55.521 INFO:tasks.workunit.client.0.smithi031.stdout:8/939: dwrite d5/da/d12/d1d/d38/fc6 [0,4194304] 0 2024-04-22T23:45:55.547 INFO:tasks.workunit.client.0.smithi031.stdout:8/940: dwrite d5/d48/fba [0,4194304] 0 2024-04-22T23:45:55.548 INFO:tasks.workunit.client.0.smithi031.stdout:8/941: chown d5/da/d12/d37/d34/f5e 9 1 2024-04-22T23:45:55.550 INFO:tasks.workunit.client.0.smithi031.stdout:8/942: creat d5/da/d12/d37/d4e/dc9/f129 x:0 0 0 2024-04-22T23:45:55.554 INFO:tasks.workunit.client.0.smithi031.stdout:8/943: dread d5/da/d12/f44 [0,4194304] 0 2024-04-22T23:45:55.556 INFO:tasks.workunit.client.0.smithi031.stdout:8/944: creat d5/da/d83/dfe/f12a x:0 0 0 2024-04-22T23:45:55.559 INFO:tasks.workunit.client.0.smithi031.stdout:8/945: creat d5/d48/d61/d6b/d6c/daa/f12b x:0 0 0 2024-04-22T23:45:55.561 INFO:tasks.workunit.client.0.smithi031.stdout:8/946: symlink d5/da/d12/d2e/db0/dc3/l12c 0 2024-04-22T23:45:55.562 INFO:tasks.workunit.client.0.smithi031.stdout:8/947: rmdir d5/d48/d61/d6b/d6c 39 2024-04-22T23:45:55.563 INFO:tasks.workunit.client.0.smithi031.stdout:8/948: dread - d5/da/d12/d2e/db2/f127 zero size 2024-04-22T23:45:55.565 INFO:tasks.workunit.client.0.smithi031.stdout:8/949: read d5/da/d12/d2e/d7e/fb4 [2479257,47625] 0 2024-04-22T23:45:55.567 INFO:tasks.workunit.client.0.smithi031.stdout:8/950: truncate d5/da/d12/d37/d34/d6f/d5b/fde 1328 0 2024-04-22T23:45:55.571 INFO:tasks.workunit.client.0.smithi031.stdout:8/951: truncate d5/da/df/f13 4085797 0 2024-04-22T23:45:55.576 INFO:tasks.workunit.client.0.smithi031.stdout:8/952: dwrite d5/da/d12/d2e/d7e/fb4 [0,4194304] 0 2024-04-22T23:45:55.578 INFO:tasks.workunit.client.0.smithi031.stdout:8/953: write d5/da/d12/d37/d34/d6f/d5b/f84 [3243356,29401] 0 2024-04-22T23:45:55.585 INFO:tasks.workunit.client.0.smithi031.stdout:8/954: getdents d5/da/d12/d2e/db0/dc3 0 2024-04-22T23:45:55.587 INFO:tasks.workunit.client.0.smithi031.stdout:0/993: sync 2024-04-22T23:45:55.587 INFO:tasks.workunit.client.0.smithi031.stdout:3/900: sync 2024-04-22T23:45:55.587 INFO:tasks.workunit.client.0.smithi031.stdout:9/912: sync 2024-04-22T23:45:55.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/994: chown d0/d13/d18/dbf/dc3/c116 1266354602 1 2024-04-22T23:45:55.589 INFO:tasks.workunit.client.0.smithi031.stdout:9/913: dread - d2/dd/d75/d80/df8/d11c/f110 zero size 2024-04-22T23:45:55.590 INFO:tasks.workunit.client.0.smithi031.stdout:0/995: write d0/d13/d18/d1e/dd3/fe9 [736247,36862] 0 2024-04-22T23:45:55.590 INFO:tasks.workunit.client.0.smithi031.stdout:3/901: mkdir d4/d6/d11/d135 0 2024-04-22T23:45:55.594 INFO:tasks.workunit.client.0.smithi031.stdout:0/996: fsync d0/d13/d18/d1e/dd3/f124 0 2024-04-22T23:45:55.596 INFO:tasks.workunit.client.0.smithi031.stdout:8/955: truncate d5/da/df/d33/f65 750629 0 2024-04-22T23:45:55.596 INFO:tasks.workunit.client.0.smithi031.stdout:8/956: fdatasync d5/da/d12/d37/d4e/dc9/f108 0 2024-04-22T23:45:55.597 INFO:tasks.workunit.client.0.smithi031.stdout:3/902: rmdir d4/d54/d88/de1 0 2024-04-22T23:45:55.663 INFO:tasks.workunit.client.0.smithi031.stdout:0/997: symlink d0/d1/d2/d36/d11/d1c/l145 0 2024-04-22T23:45:55.665 INFO:tasks.workunit.client.0.smithi031.stdout:3/903: creat d4/d6/d11/d25/d44/d7f/f136 x:0 0 0 2024-04-22T23:45:55.668 INFO:tasks.workunit.client.0.smithi031.stdout:3/904: creat d4/d6/d11/d3e/f137 x:0 0 0 2024-04-22T23:45:55.669 INFO:tasks.workunit.client.0.smithi031.stdout:0/998: dwrite d0/d1/d2/d36/d11/d1a/d3c/f4d [4194304,4194304] 0 2024-04-22T23:45:55.671 INFO:tasks.workunit.client.0.smithi031.stdout:0/999: dread - d0/d1/f45 zero size 2024-04-22T23:45:55.672 INFO:tasks.workunit.client.0.smithi031.stdout:3/905: mknod d4/d6/d11/ddf/dc3/c138 0 2024-04-22T23:45:55.676 INFO:tasks.workunit.client.0.smithi031.stdout:8/957: truncate d5/da/d12/d2e/d4d/d56/f71 3098560 0 2024-04-22T23:45:55.677 INFO:tasks.workunit.client.0.smithi031.stdout:3/906: symlink d4/d54/d2c/d89/d2e/d45/l139 0 2024-04-22T23:45:55.679 INFO:tasks.workunit.client.0.smithi031.stdout:3/907: write d4/d6/d11/d25/d44/d7f/d5f/dee/df8/f11a [582342,67679] 0 2024-04-22T23:45:55.680 INFO:tasks.workunit.client.0.smithi031.stdout:8/958: mknod d5/da/d12/d37/c12d 0 2024-04-22T23:45:55.683 INFO:tasks.workunit.client.0.smithi031.stdout:3/908: mknod d4/d54/d2c/d89/c13a 0 2024-04-22T23:45:55.684 INFO:tasks.workunit.client.0.smithi031.stdout:3/909: chown d4/d6/da/f13 206 1 2024-04-22T23:45:55.685 INFO:tasks.workunit.client.0.smithi031.stdout:5/914: sync 2024-04-22T23:45:55.696 INFO:tasks.workunit.client.0.smithi031.stdout:3/910: dwrite d4/d6/da/fb9 [0,4194304] 0 2024-04-22T23:45:55.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/915: dread db/d16/d1d/d24/d27/fb0 [0,4194304] 0 2024-04-22T23:45:55.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/959: dread d5/da/d12/d1d/fca [0,4194304] 0 2024-04-22T23:45:55.705 INFO:tasks.workunit.client.0.smithi031.stdout:8/960: stat d5/da/d12/d37/d4e/d101/c121 0 2024-04-22T23:45:55.705 INFO:tasks.workunit.client.0.smithi031.stdout:3/911: truncate d4/d6/d11/d25/d44/d7f/f8f 403256 0 2024-04-22T23:45:55.707 INFO:tasks.workunit.client.0.smithi031.stdout:8/961: creat d5/da/d83/d8d/f12e x:0 0 0 2024-04-22T23:45:55.707 INFO:tasks.workunit.client.0.smithi031.stdout:8/962: stat d5/da/d12/d2e/d4d/f10b 0 2024-04-22T23:45:55.710 INFO:tasks.workunit.client.0.smithi031.stdout:8/963: mknod d5/da/d12/d37/d34/d6f/d85/df1/c12f 0 2024-04-22T23:45:55.711 INFO:tasks.workunit.client.0.smithi031.stdout:5/916: dread db/d16/d1d/d24/d58/d5d/f109 [0,4194304] 0 2024-04-22T23:45:55.712 INFO:tasks.workunit.client.0.smithi031.stdout:8/964: mknod d5/da/d12/d37/c130 0 2024-04-22T23:45:55.713 INFO:tasks.workunit.client.0.smithi031.stdout:3/912: dread d4/d6/d11/d25/d44/f71 [0,4194304] 0 2024-04-22T23:45:55.714 INFO:tasks.workunit.client.0.smithi031.stdout:5/917: mkdir db/d16/d130 0 2024-04-22T23:45:55.714 INFO:tasks.workunit.client.0.smithi031.stdout:8/965: write d5/da/d12/d37/d34/fcf [171240,40125] 0 2024-04-22T23:45:55.716 INFO:tasks.workunit.client.0.smithi031.stdout:3/913: creat d4/d54/d2c/d89/f13b x:0 0 0 2024-04-22T23:45:55.717 INFO:tasks.workunit.client.0.smithi031.stdout:8/966: truncate d5/da/df/d33/f65 1564574 0 2024-04-22T23:45:55.718 INFO:tasks.workunit.client.0.smithi031.stdout:8/967: write d5/da/d12/d37/d34/fcf [508596,16173] 0 2024-04-22T23:45:55.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/918: link db/d16/d2a/f7f db/f131 0 2024-04-22T23:45:55.721 INFO:tasks.workunit.client.0.smithi031.stdout:8/968: unlink d5/da/d12/d37/d4e/fc4 0 2024-04-22T23:45:55.721 INFO:tasks.workunit.client.0.smithi031.stdout:8/969: readlink d5/da/d12/d2e/d4d/d9a/la3 0 2024-04-22T23:45:55.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/970: mkdir d5/da/d12/d2e/d7e/d131 0 2024-04-22T23:45:55.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/971: creat d5/da/d83/d112/f132 x:0 0 0 2024-04-22T23:45:55.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/972: readlink d5/da/d12/d2e/d4d/d56/l92 0 2024-04-22T23:45:55.731 INFO:tasks.workunit.client.0.smithi031.stdout:8/973: rename d5/da/df to d5/da/d12/d1d/d38/db9/d133 0 2024-04-22T23:45:55.733 INFO:tasks.workunit.client.0.smithi031.stdout:8/974: write d5/da/d12/d37/d34/f4c [4686273,91709] 0 2024-04-22T23:45:55.737 INFO:tasks.workunit.client.0.smithi031.stdout:8/975: creat d5/da/d12/d2e/db0/dc3/de0/f134 x:0 0 0 2024-04-22T23:45:55.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/976: dread d5/da/d12/f96 [0,4194304] 0 2024-04-22T23:45:55.740 INFO:tasks.workunit.client.0.smithi031.stdout:5/919: getdents db/d16/d1d/d24/d27 0 2024-04-22T23:45:55.741 INFO:tasks.workunit.client.0.smithi031.stdout:5/920: write db/d16/d2a/d36/fdb [505385,92577] 0 2024-04-22T23:45:55.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/977: dwrite d5/da/d12/d2e/d4d/d9a/db1/f111 [0,4194304] 0 2024-04-22T23:45:55.748 INFO:tasks.workunit.client.0.smithi031.stdout:5/921: rename db/d16/d1d/d24/ddf/fe9 to db/d16/d1d/d24/d27/d9e/d102/f132 0 2024-04-22T23:45:55.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/922: write db/d16/d2a/d36/d74/fe8 [903914,105466] 0 2024-04-22T23:45:55.754 INFO:tasks.workunit.client.0.smithi031.stdout:5/923: dread - db/d8d/df1/d125/f9d zero size 2024-04-22T23:45:55.758 INFO:tasks.workunit.client.0.smithi031.stdout:5/924: rename db/d16/cd2 to db/d16/d2a/c133 0 2024-04-22T23:45:55.763 INFO:tasks.workunit.client.0.smithi031.stdout:5/925: write db/d16/d2a/d36/d74/f87 [4781349,89715] 0 2024-04-22T23:45:55.780 INFO:tasks.workunit.client.0.smithi031.stdout:5/926: readlink db/d16/d1d/d24/d58/d5d/laf 0 2024-04-22T23:45:55.785 INFO:tasks.workunit.client.0.smithi031.stdout:5/927: dwrite db/d16/d1d/d24/d32/fe3 [0,4194304] 0 2024-04-22T23:45:55.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/928: chown db/d8d/df1/cf2 17528771 1 2024-04-22T23:45:55.791 INFO:tasks.workunit.client.0.smithi031.stdout:5/929: creat db/f134 x:0 0 0 2024-04-22T23:45:55.802 INFO:tasks.workunit.client.0.smithi031.stdout:5/930: dwrite db/d16/d1d/d24/fc6 [0,4194304] 0 2024-04-22T23:45:55.808 INFO:tasks.workunit.client.0.smithi031.stdout:5/931: dwrite db/d16/d1d/d24/d58/f51 [0,4194304] 0 2024-04-22T23:45:55.822 INFO:tasks.workunit.client.0.smithi031.stdout:9/914: sync 2024-04-22T23:45:55.824 INFO:tasks.workunit.client.0.smithi031.stdout:5/932: dread db/d16/f2b [0,4194304] 0 2024-04-22T23:45:55.825 INFO:tasks.workunit.client.0.smithi031.stdout:5/933: fsync db/d16/d2a/d36/d74/d82/f11e 0 2024-04-22T23:45:55.825 INFO:tasks.workunit.client.0.smithi031.stdout:9/915: rename d2/dc/d3e/f4f to d2/dc/d8b/f130 0 2024-04-22T23:45:55.827 INFO:tasks.workunit.client.0.smithi031.stdout:9/916: unlink d2/dd/d65/ce6 0 2024-04-22T23:45:55.829 INFO:tasks.workunit.client.0.smithi031.stdout:5/934: rmdir db/d8d/d11a/da4 39 2024-04-22T23:45:55.830 INFO:tasks.workunit.client.0.smithi031.stdout:5/935: write db/d16/d2a/d59/da8/fac [3246969,19099] 0 2024-04-22T23:45:55.873 INFO:tasks.workunit.client.0.smithi031.stdout:3/914: sync 2024-04-22T23:45:55.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/915: dread d4/d54/d6d/f9b [0,4194304] 0 2024-04-22T23:45:55.883 INFO:tasks.workunit.client.0.smithi031.stdout:3/916: mknod d4/d6/d4c/d8c/d93/c13c 0 2024-04-22T23:45:55.884 INFO:tasks.workunit.client.0.smithi031.stdout:3/917: chown d4/d54/d2c 137466048 1 2024-04-22T23:45:55.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/978: sync 2024-04-22T23:45:55.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/979: write d5/dd8/fe6 [709263,81566] 0 2024-04-22T23:45:55.939 INFO:tasks.workunit.client.0.smithi031.stdout:8/980: unlink d5/da/d12/d37/d4e/dc9/f129 0 2024-04-22T23:45:55.940 INFO:tasks.workunit.client.0.smithi031.stdout:8/981: dread - d5/da/d83/d8d/f12e zero size 2024-04-22T23:45:56.122 INFO:tasks.workunit.client.0.smithi031.stdout:5/936: sync 2024-04-22T23:45:56.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/917: sync 2024-04-22T23:45:56.123 INFO:tasks.workunit.client.0.smithi031.stdout:3/918: sync 2024-04-22T23:45:56.123 INFO:tasks.workunit.client.0.smithi031.stdout:3/919: write d4/d83/fab [1991636,114837] 0 2024-04-22T23:45:56.127 INFO:tasks.workunit.client.0.smithi031.stdout:3/920: chown d4/d6/d11/d25/d44/d7f/d5f/d12e/l105 130094062 1 2024-04-22T23:45:56.128 INFO:tasks.workunit.client.0.smithi031.stdout:5/937: creat db/d16/d2a/f135 x:0 0 0 2024-04-22T23:45:56.132 INFO:tasks.workunit.client.0.smithi031.stdout:5/938: symlink db/d8d/d11a/da4/l136 0 2024-04-22T23:45:56.132 INFO:tasks.workunit.client.0.smithi031.stdout:3/921: link d4/d54/d2c/d89/d2e/c4b d4/d54/d88/da2/c13d 0 2024-04-22T23:45:56.134 INFO:tasks.workunit.client.0.smithi031.stdout:5/939: symlink db/d16/d2a/d7c/l137 0 2024-04-22T23:45:56.135 INFO:tasks.workunit.client.0.smithi031.stdout:5/940: stat db/d16/d2a/d59/lea 0 2024-04-22T23:45:56.135 INFO:tasks.workunit.client.0.smithi031.stdout:3/922: fsync d4/d54/d6d/f9b 0 2024-04-22T23:45:56.135 INFO:tasks.workunit.client.0.smithi031.stdout:5/941: chown db/d16/d2a/d36 32550 1 2024-04-22T23:45:56.137 INFO:tasks.workunit.client.0.smithi031.stdout:5/942: unlink db/d8d/df1/l11c 0 2024-04-22T23:45:56.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/943: symlink db/d8d/d11a/d116/l138 0 2024-04-22T23:45:56.145 INFO:tasks.workunit.client.0.smithi031.stdout:3/923: rename d4/d6/d11/d25/d44/c10f to d4/d54/d88/da2/de0/d10b/c13e 0 2024-04-22T23:45:56.146 INFO:tasks.workunit.client.0.smithi031.stdout:3/924: mkdir d4/d6/d11/d25/d44/d7f/d5f/dee/d13f 0 2024-04-22T23:45:56.146 INFO:tasks.workunit.client.0.smithi031.stdout:3/925: readlink d4/d6/d22/l70 0 2024-04-22T23:45:56.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/926: mknod d4/d54/d88/da2/c140 0 2024-04-22T23:45:56.148 INFO:tasks.workunit.client.0.smithi031.stdout:5/944: dwrite db/f3f [0,4194304] 0 2024-04-22T23:45:56.149 INFO:tasks.workunit.client.0.smithi031.stdout:5/945: chown db/d16/f8b 105913 1 2024-04-22T23:45:56.153 INFO:tasks.workunit.client.0.smithi031.stdout:8/982: sync 2024-04-22T23:45:56.154 INFO:tasks.workunit.client.0.smithi031.stdout:5/946: rmdir db/d16/d2a/d7c/d89 39 2024-04-22T23:45:56.157 INFO:tasks.workunit.client.0.smithi031.stdout:9/918: dwrite d2/dc/d6d/fa6 [0,4194304] 0 2024-04-22T23:45:56.159 INFO:tasks.workunit.client.0.smithi031.stdout:5/947: write db/d16/d2a/d36/f107 [180959,112663] 0 2024-04-22T23:45:56.160 INFO:tasks.workunit.client.0.smithi031.stdout:9/919: dread d2/f6e [0,4194304] 0 2024-04-22T23:45:56.166 INFO:tasks.workunit.client.0.smithi031.stdout:9/920: truncate d2/dd/d75/f7e 5001830 0 2024-04-22T23:45:56.166 INFO:tasks.workunit.client.0.smithi031.stdout:3/927: dwrite d4/d6/d11/d25/d44/d7f/d5f/fa9 [0,4194304] 0 2024-04-22T23:45:56.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/921: read d2/dc/f4c [616674,28222] 0 2024-04-22T23:45:56.168 INFO:tasks.workunit.client.0.smithi031.stdout:5/948: rmdir db/d16/d1d/d24/d32/d41/d104 0 2024-04-22T23:45:56.168 INFO:tasks.workunit.client.0.smithi031.stdout:5/949: chown db/d16/d2a/d36/d74 5 1 2024-04-22T23:45:56.169 INFO:tasks.workunit.client.0.smithi031.stdout:3/928: mknod d4/d49/c141 0 2024-04-22T23:45:56.170 INFO:tasks.workunit.client.0.smithi031.stdout:5/950: fdatasync db/d8d/df1/f129 0 2024-04-22T23:45:56.170 INFO:tasks.workunit.client.0.smithi031.stdout:9/922: mknod d2/dc/d8b/d3d/d40/da4/db1/dc5/c131 0 2024-04-22T23:45:56.174 INFO:tasks.workunit.client.0.smithi031.stdout:3/929: unlink d4/d54/d88/da2/de0/c111 0 2024-04-22T23:45:56.176 INFO:tasks.workunit.client.0.smithi031.stdout:3/930: symlink d4/d54/d56/l142 0 2024-04-22T23:45:56.178 INFO:tasks.workunit.client.0.smithi031.stdout:3/931: creat d4/d6/d11/ddf/dc3/f143 x:0 0 0 2024-04-22T23:45:56.179 INFO:tasks.workunit.client.0.smithi031.stdout:5/951: getdents db/d16/d2a/d36/d74/d82 0 2024-04-22T23:45:56.180 INFO:tasks.workunit.client.0.smithi031.stdout:3/932: readlink d4/d54/d2c/d89/d2e/d8d/l12b 0 2024-04-22T23:45:56.181 INFO:tasks.workunit.client.0.smithi031.stdout:5/952: symlink db/d16/d1d/d24/d32/ded/l139 0 2024-04-22T23:45:56.183 INFO:tasks.workunit.client.0.smithi031.stdout:5/953: fsync db/d16/d2a/f7f 0 2024-04-22T23:45:56.187 INFO:tasks.workunit.client.0.smithi031.stdout:3/933: rename d4/d6/da/f3d to d4/f144 0 2024-04-22T23:45:56.191 INFO:tasks.workunit.client.0.smithi031.stdout:3/934: dread d4/d6/d11/d25/d44/d7f/da5/f11d [0,4194304] 0 2024-04-22T23:45:56.192 INFO:tasks.workunit.client.0.smithi031.stdout:9/923: write d2/dc/d8b/d3d/fdd [18374,130783] 0 2024-04-22T23:45:56.194 INFO:tasks.workunit.client.0.smithi031.stdout:9/924: dread - d2/dc/d8b/d3d/d40/da4/d104/f127 zero size 2024-04-22T23:45:56.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:56 smithi031 ceph-mon[26721]: pgmap v34: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 92 MiB/s rd, 234 MiB/s wr, 592 op/s 2024-04-22T23:45:56.196 INFO:tasks.workunit.client.0.smithi031.stdout:3/935: rename d4/d6/d4c/d8c/d93 to d4/d54/d85/dc1/d145 0 2024-04-22T23:45:56.196 INFO:tasks.workunit.client.0.smithi031.stdout:5/954: rmdir db/d16/d1d/d24/d32/ded 39 2024-04-22T23:45:56.198 INFO:tasks.workunit.client.0.smithi031.stdout:5/955: write db/d16/d2a/d36/d74/d82/f117 [227902,50677] 0 2024-04-22T23:45:56.198 INFO:tasks.workunit.client.0.smithi031.stdout:9/925: fdatasync d2/dd/f2b 0 2024-04-22T23:45:56.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/926: rmdir d2/dd/d75/d80 39 2024-04-22T23:45:56.203 INFO:tasks.workunit.client.0.smithi031.stdout:3/936: dwrite d4/d6/d11/ddf/dc3/f143 [0,4194304] 0 2024-04-22T23:45:56.204 INFO:tasks.workunit.client.0.smithi031.stdout:3/937: chown d4/d6/d11/d25/d44/d7f/d5f/f63 354 1 2024-04-22T23:45:56.206 INFO:tasks.workunit.client.0.smithi031.stdout:3/938: mknod d4/d6/d11/d25/d44/d7f/c146 0 2024-04-22T23:45:56.207 INFO:tasks.workunit.client.0.smithi031.stdout:3/939: chown d4/d6/d11/ddf 105668914 1 2024-04-22T23:45:56.207 INFO:tasks.workunit.client.0.smithi031.stdout:5/956: write db/d16/d1d/f47 [401389,13917] 0 2024-04-22T23:45:56.209 INFO:tasks.workunit.client.0.smithi031.stdout:5/957: stat db/f23 0 2024-04-22T23:45:56.210 INFO:tasks.workunit.client.0.smithi031.stdout:3/940: truncate d4/d54/d6d/f9b 994795 0 2024-04-22T23:45:56.213 INFO:tasks.workunit.client.0.smithi031.stdout:5/958: rename db/d16/d1d/fe5 to db/d16/d1d/d24/d58/d5d/dd6/f13a 0 2024-04-22T23:45:56.214 INFO:tasks.workunit.client.0.smithi031.stdout:3/941: symlink d4/d54/d88/da2/l147 0 2024-04-22T23:45:56.215 INFO:tasks.workunit.client.0.smithi031.stdout:3/942: mkdir d4/d6/da/d148 0 2024-04-22T23:45:56.219 INFO:tasks.workunit.client.0.smithi031.stdout:5/959: dwrite db/d16/d1d/d24/d27/d9e/fae [0,4194304] 0 2024-04-22T23:45:56.222 INFO:tasks.workunit.client.0.smithi031.stdout:3/943: rename d4/d6/d4c/ddb to d4/d54/d88/da2/de0/d149 0 2024-04-22T23:45:56.224 INFO:tasks.workunit.client.0.smithi031.stdout:5/960: mknod db/d16/d1d/d24/d27/c13b 0 2024-04-22T23:45:56.225 INFO:tasks.workunit.client.0.smithi031.stdout:5/961: write db/d16/fce [564360,87843] 0 2024-04-22T23:45:56.227 INFO:tasks.workunit.client.0.smithi031.stdout:3/944: mkdir d4/d6/d11/d25/d44/d7f/d5f/d14a 0 2024-04-22T23:45:56.230 INFO:tasks.workunit.client.0.smithi031.stdout:3/945: creat d4/d54/d6d/f14b x:0 0 0 2024-04-22T23:45:56.231 INFO:tasks.workunit.client.0.smithi031.stdout:3/946: chown d4/d6/d22/l70 1743 1 2024-04-22T23:45:56.234 INFO:tasks.workunit.client.0.smithi031.stdout:5/962: dread db/d16/d1d/f93 [0,4194304] 0 2024-04-22T23:45:56.246 INFO:tasks.workunit.client.0.smithi031.stdout:3/947: dread d4/d54/d2c/d89/f62 [0,4194304] 0 2024-04-22T23:45:56.248 INFO:tasks.workunit.client.0.smithi031.stdout:3/948: creat d4/d54/d85/f14c x:0 0 0 2024-04-22T23:45:56.250 INFO:tasks.workunit.client.0.smithi031.stdout:5/963: dread db/d16/d1d/d24/d27/f99 [0,4194304] 0 2024-04-22T23:45:56.251 INFO:tasks.workunit.client.0.smithi031.stdout:3/949: mkdir d4/d6/d11/d25/d44/d7f/da5/d14d 0 2024-04-22T23:45:56.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/964: dread db/d16/d1d/d24/d32/fe3 [0,4194304] 0 2024-04-22T23:45:56.255 INFO:tasks.workunit.client.0.smithi031.stdout:3/950: rename d4/d83/dd8/de6 to d4/d6/da/d148/d14e 0 2024-04-22T23:45:56.259 INFO:tasks.workunit.client.0.smithi031.stdout:3/951: creat d4/d6/d11/de3/f14f x:0 0 0 2024-04-22T23:45:56.260 INFO:tasks.workunit.client.0.smithi031.stdout:5/965: creat db/d16/d1d/d24/d12b/f13c x:0 0 0 2024-04-22T23:45:56.260 INFO:tasks.workunit.client.0.smithi031.stdout:5/966: stat db/d8d/d11a/da4/l136 0 2024-04-22T23:45:56.262 INFO:tasks.workunit.client.0.smithi031.stdout:3/952: creat d4/d54/d88/da2/de0/d10b/f150 x:0 0 0 2024-04-22T23:45:56.264 INFO:tasks.workunit.client.0.smithi031.stdout:5/967: creat db/d16/f13d x:0 0 0 2024-04-22T23:45:56.266 INFO:tasks.workunit.client.0.smithi031.stdout:3/953: unlink d4/d54/d88/da2/de0/ff7 0 2024-04-22T23:45:56.266 INFO:tasks.workunit.client.0.smithi031.stdout:5/968: fdatasync db/d16/d2a/d36/d74/fbc 0 2024-04-22T23:45:56.268 INFO:tasks.workunit.client.0.smithi031.stdout:3/954: creat d4/d6/d11/dda/f151 x:0 0 0 2024-04-22T23:45:56.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/955: dread d4/d54/d6d/f9b [0,4194304] 0 2024-04-22T23:45:56.275 INFO:tasks.workunit.client.0.smithi031.stdout:3/956: rename d4/d54/d85/dc1/cec to d4/d6/d11/dda/c152 0 2024-04-22T23:45:56.275 INFO:tasks.workunit.client.0.smithi031.stdout:5/969: truncate db/d16/d1d/d24/d58/db1/fb5 441422 0 2024-04-22T23:45:56.277 INFO:tasks.workunit.client.0.smithi031.stdout:3/957: fdatasync d4/d6/d11/d25/d44/d7f/fbc 0 2024-04-22T23:45:56.281 INFO:tasks.workunit.client.0.smithi031.stdout:5/970: write db/d16/d2a/f8f [341733,129145] 0 2024-04-22T23:45:56.285 INFO:tasks.workunit.client.0.smithi031.stdout:5/971: link db/d16/d2a/f2c db/d8d/d11a/da4/f13e 0 2024-04-22T23:45:56.288 INFO:tasks.workunit.client.0.smithi031.stdout:5/972: link db/d16/fa2 db/f13f 0 2024-04-22T23:45:56.289 INFO:tasks.workunit.client.0.smithi031.stdout:5/973: dread - db/d16/d1d/d24/d32/fe6 zero size 2024-04-22T23:45:56.290 INFO:tasks.workunit.client.0.smithi031.stdout:5/974: stat db/d8d/df1/d125/fc8 0 2024-04-22T23:45:56.292 INFO:tasks.workunit.client.0.smithi031.stdout:5/975: read db/d8d/d11a/f8e [688337,5210] 0 2024-04-22T23:45:56.330 INFO:tasks.workunit.client.0.smithi031.stdout:5/976: dwrite db/f1e [0,4194304] 0 2024-04-22T23:45:56.333 INFO:tasks.workunit.client.0.smithi031.stdout:5/977: mkdir db/d16/d1d/d24/d58/db1/d140 0 2024-04-22T23:45:56.336 INFO:tasks.workunit.client.0.smithi031.stdout:5/978: creat db/d16/d130/f141 x:0 0 0 2024-04-22T23:45:56.368 INFO:tasks.workunit.client.0.smithi031.stdout:8/983: sync 2024-04-22T23:45:56.371 INFO:tasks.workunit.client.0.smithi031.stdout:8/984: rename d5/da/d12/d1d/fca to d5/da/d12/d2e/d4d/d9a/db1/df2/f135 0 2024-04-22T23:45:56.375 INFO:tasks.workunit.client.0.smithi031.stdout:8/985: rename d5/da/d12/d37/d34/d6f/fdd to d5/da/d12/d2e/d4d/d9a/db1/dbb/f136 0 2024-04-22T23:45:56.380 INFO:tasks.workunit.client.0.smithi031.stdout:8/986: write d5/d48/d61/d6b/fa1 [3593523,36738] 0 2024-04-22T23:45:56.385 INFO:tasks.workunit.client.0.smithi031.stdout:8/987: rename d5/da/d12/d1d/d38/db9/d133/d33/dd4 to d5/da/d12/d2e/db0/d11f/d137 0 2024-04-22T23:45:56.390 INFO:tasks.workunit.client.0.smithi031.stdout:8/988: dwrite d5/da/d12/d2e/db0/d11f/d137/f118 [0,4194304] 0 2024-04-22T23:45:56.392 INFO:tasks.workunit.client.0.smithi031.stdout:8/989: creat d5/da/d12/d2e/db0/f138 x:0 0 0 2024-04-22T23:45:56.397 INFO:tasks.workunit.client.0.smithi031.stdout:8/990: link d5/da/d12/d1d/d38/db9/d133/c18 d5/da/d12/d2e/d4d/c139 0 2024-04-22T23:45:56.400 INFO:tasks.workunit.client.0.smithi031.stdout:8/991: unlink d5/da/d12/d2e/d4d/d9a/db1/dbb/led 0 2024-04-22T23:45:56.402 INFO:tasks.workunit.client.0.smithi031.stdout:8/992: symlink d5/da/d83/d112/l13a 0 2024-04-22T23:45:56.412 INFO:tasks.workunit.client.0.smithi031.stdout:8/993: dwrite d5/da/d12/d2e/d4d/d9a/db1/f11e [0,4194304] 0 2024-04-22T23:45:56.417 INFO:tasks.workunit.client.0.smithi031.stdout:8/994: write d5/da/d12/d37/f78 [1040842,74792] 0 2024-04-22T23:45:56.418 INFO:tasks.workunit.client.0.smithi031.stdout:8/995: chown d5/da/d12/d1d 189379497 1 2024-04-22T23:45:56.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/996: creat d5/d48/d61/d6b/de1/f13b x:0 0 0 2024-04-22T23:45:56.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/997: fsync d5/d48/fba 0 2024-04-22T23:45:56.422 INFO:tasks.workunit.client.0.smithi031.stdout:8/998: readlink d5/da/d12/l66 0 2024-04-22T23:45:56.423 INFO:tasks.workunit.client.0.smithi031.stdout:8/999: mkdir d5/da/d12/d2e/d4d/d9a/d13c 0 2024-04-22T23:45:56.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:56 smithi044 ceph-mon[34463]: pgmap v34: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 92 MiB/s rd, 234 MiB/s wr, 592 op/s 2024-04-22T23:45:56.464 INFO:tasks.workunit.client.0.smithi031.stdout:3/958: sync 2024-04-22T23:45:56.464 INFO:tasks.workunit.client.0.smithi031.stdout:9/927: sync 2024-04-22T23:45:56.466 INFO:tasks.workunit.client.0.smithi031.stdout:9/928: dread d2/dc/d8b/d3d/fdd [0,4194304] 0 2024-04-22T23:45:56.467 INFO:tasks.workunit.client.0.smithi031.stdout:3/959: rename d4/d6/d11/de3/c109 to d4/d54/d88/da2/c153 0 2024-04-22T23:45:56.468 INFO:tasks.workunit.client.0.smithi031.stdout:9/929: mkdir d2/dc/d132 0 2024-04-22T23:45:56.469 INFO:tasks.workunit.client.0.smithi031.stdout:3/960: creat d4/d6/d11/d25/d44/f154 x:0 0 0 2024-04-22T23:45:56.471 INFO:tasks.workunit.client.0.smithi031.stdout:3/961: creat d4/d6/d11/d25/d44/d7f/f155 x:0 0 0 2024-04-22T23:45:56.476 INFO:tasks.workunit.client.0.smithi031.stdout:9/930: write d2/dd/d75/d64/f55 [188809,32977] 0 2024-04-22T23:45:56.476 INFO:tasks.workunit.client.0.smithi031.stdout:3/962: write d4/d54/d2c/d89/d2e/f39 [628469,35787] 0 2024-04-22T23:45:56.481 INFO:tasks.workunit.client.0.smithi031.stdout:3/963: fsync d4/d54/d2c/d89/d2e/d45/f6c 0 2024-04-22T23:45:56.481 INFO:tasks.workunit.client.0.smithi031.stdout:9/931: write d2/dd/d75/d64/d5d/f57 [4247434,6063] 0 2024-04-22T23:45:56.484 INFO:tasks.workunit.client.0.smithi031.stdout:3/964: unlink d4/d54/d2c/d89/d2e/c104 0 2024-04-22T23:45:56.485 INFO:tasks.workunit.client.0.smithi031.stdout:3/965: fsync d4/d54/d2c/d89/d2e/ff6 0 2024-04-22T23:45:56.489 INFO:tasks.workunit.client.0.smithi031.stdout:3/966: write d4/d54/d2c/d89/d2e/d45/f60 [1021349,116059] 0 2024-04-22T23:45:56.491 INFO:tasks.workunit.client.0.smithi031.stdout:3/967: symlink d4/d49/l156 0 2024-04-22T23:45:56.492 INFO:tasks.workunit.client.0.smithi031.stdout:3/968: fsync d4/d54/d2c/d89/d2e/ff6 0 2024-04-22T23:45:56.495 INFO:tasks.workunit.client.0.smithi031.stdout:3/969: creat d4/d83/f157 x:0 0 0 2024-04-22T23:45:56.495 INFO:tasks.workunit.client.0.smithi031.stdout:9/932: dread d2/dc/d8b/d3d/d40/dfd/ffe [0,4194304] 0 2024-04-22T23:45:56.496 INFO:tasks.workunit.client.0.smithi031.stdout:3/970: rmdir d4/d54/d56/dae 39 2024-04-22T23:45:56.497 INFO:tasks.workunit.client.0.smithi031.stdout:9/933: write d2/dc/f43 [3737940,63827] 0 2024-04-22T23:45:56.499 INFO:tasks.workunit.client.0.smithi031.stdout:9/934: mkdir d2/dd/d133 0 2024-04-22T23:45:56.503 INFO:tasks.workunit.client.0.smithi031.stdout:3/971: dwrite d4/d54/d2c/d89/f13b [0,4194304] 0 2024-04-22T23:45:56.504 INFO:tasks.workunit.client.0.smithi031.stdout:9/935: rename d2/dd/d75/d64/d5d/d59/f128 to d2/dc/d8b/d3d/d40/da4/f134 0 2024-04-22T23:45:56.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/936: rmdir d2/d15/d5a 39 2024-04-22T23:45:56.509 INFO:tasks.workunit.client.0.smithi031.stdout:9/937: rmdir d2/dc/d8b/d3d/df9 39 2024-04-22T23:45:56.513 INFO:tasks.workunit.client.0.smithi031.stdout:3/972: truncate d4/d54/d2c/d89/f62 3547674 0 2024-04-22T23:45:56.515 INFO:tasks.workunit.client.0.smithi031.stdout:5/979: sync 2024-04-22T23:45:56.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/938: dwrite d2/dd/d75/d64/d5d/d59/fb0 [8388608,4194304] 0 2024-04-22T23:45:56.516 INFO:tasks.workunit.client.0.smithi031.stdout:3/973: chown d4/d54/d6d/d98/cb5 7587 1 2024-04-22T23:45:56.518 INFO:tasks.workunit.client.0.smithi031.stdout:5/980: creat db/d16/d1d/d24/d10d/f142 x:0 0 0 2024-04-22T23:45:56.522 INFO:tasks.workunit.client.0.smithi031.stdout:5/981: symlink db/d16/d1d/d24/ddf/df5/l143 0 2024-04-22T23:45:56.522 INFO:tasks.workunit.client.0.smithi031.stdout:3/974: getdents d4/d6 0 2024-04-22T23:45:56.523 INFO:tasks.workunit.client.0.smithi031.stdout:9/939: rename d2/dd/d75/f7e to d2/dc/d8b/d3d/df9/f135 0 2024-04-22T23:45:56.523 INFO:tasks.workunit.client.0.smithi031.stdout:3/975: chown d4/d54/d2c/d89/d2e 221 1 2024-04-22T23:45:56.524 INFO:tasks.workunit.client.0.smithi031.stdout:5/982: dread - db/d16/d2a/d36/d74/fa3 zero size 2024-04-22T23:45:56.525 INFO:tasks.workunit.client.0.smithi031.stdout:3/976: chown d4/d6/d11/l10d 50693 1 2024-04-22T23:45:56.525 INFO:tasks.workunit.client.0.smithi031.stdout:5/983: chown db/d16/d1d/d24/d32/ff9 1152 1 2024-04-22T23:45:56.527 INFO:tasks.workunit.client.0.smithi031.stdout:9/940: link d2/dd/d65/l7b d2/dd/l136 0 2024-04-22T23:45:56.530 INFO:tasks.workunit.client.0.smithi031.stdout:3/977: creat d4/d83/dd8/f158 x:0 0 0 2024-04-22T23:45:56.531 INFO:tasks.workunit.client.0.smithi031.stdout:3/978: write d4/d6/da/f128 [146807,104184] 0 2024-04-22T23:45:56.532 INFO:tasks.workunit.client.0.smithi031.stdout:9/941: mknod d2/dc/d8b/c137 0 2024-04-22T23:45:56.532 INFO:tasks.workunit.client.0.smithi031.stdout:3/979: stat d4/d6/d11/dda 0 2024-04-22T23:45:56.539 INFO:tasks.workunit.client.0.smithi031.stdout:9/942: truncate d2/dd/d75/d4a/f36 2210756 0 2024-04-22T23:45:56.539 INFO:tasks.workunit.client.0.smithi031.stdout:5/984: dwrite db/f42 [0,4194304] 0 2024-04-22T23:45:56.541 INFO:tasks.workunit.client.0.smithi031.stdout:3/980: fsync d4/d54/d2c/d89/f62 0 2024-04-22T23:45:56.542 INFO:tasks.workunit.client.0.smithi031.stdout:5/985: rmdir db/d8d/d11a 39 2024-04-22T23:45:56.545 INFO:tasks.workunit.client.0.smithi031.stdout:9/943: dwrite d2/dd/d75/d64/d5d/d59/f78 [0,4194304] 0 2024-04-22T23:45:56.550 INFO:tasks.workunit.client.0.smithi031.stdout:3/981: write d4/d54/d56/f75 [5370785,39956] 0 2024-04-22T23:45:56.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/944: truncate d2/dc/d8b/d3d/d40/da4/db1/dc5/fda 454384 0 2024-04-22T23:45:56.558 INFO:tasks.workunit.client.0.smithi031.stdout:3/982: creat d4/d6/d11/ddf/d115/f159 x:0 0 0 2024-04-22T23:45:56.558 INFO:tasks.workunit.client.0.smithi031.stdout:9/945: creat d2/d15/daf/de5/f138 x:0 0 0 2024-04-22T23:45:56.558 INFO:tasks.workunit.client.0.smithi031.stdout:9/946: fdatasync d2/dc/faa 0 2024-04-22T23:45:56.559 INFO:tasks.workunit.client.0.smithi031.stdout:5/986: dwrite db/f81 [0,4194304] 0 2024-04-22T23:45:56.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/987: chown db/d16/d2a/d36/d74/fe8 9 1 2024-04-22T23:45:56.561 INFO:tasks.workunit.client.0.smithi031.stdout:9/947: truncate d2/f6e 1522820 0 2024-04-22T23:45:56.561 INFO:tasks.workunit.client.0.smithi031.stdout:9/948: fdatasync d2/f11e 0 2024-04-22T23:45:56.563 INFO:tasks.workunit.client.0.smithi031.stdout:9/949: write d2/dd/d65/feb [3364184,75306] 0 2024-04-22T23:45:56.568 INFO:tasks.workunit.client.0.smithi031.stdout:5/988: dwrite db/d16/d1d/d24/d10d/f142 [0,4194304] 0 2024-04-22T23:45:56.571 INFO:tasks.workunit.client.0.smithi031.stdout:5/989: write db/d16/d1d/d24/d27/f57 [5237302,36523] 0 2024-04-22T23:45:56.575 INFO:tasks.workunit.client.0.smithi031.stdout:9/950: dread d2/dc/f4c [0,4194304] 0 2024-04-22T23:45:56.575 INFO:tasks.workunit.client.0.smithi031.stdout:9/951: fsync d2/f2e 0 2024-04-22T23:45:56.577 INFO:tasks.workunit.client.0.smithi031.stdout:5/990: unlink db/d16/d1d/d24/d32/ded/f100 0 2024-04-22T23:45:56.582 INFO:tasks.workunit.client.0.smithi031.stdout:9/952: link d2/dc/d3e/da1/f118 d2/dd/d75/df7/d122/f139 0 2024-04-22T23:45:56.583 INFO:tasks.workunit.client.0.smithi031.stdout:5/991: truncate db/d16/f15 2136780 0 2024-04-22T23:45:56.584 INFO:tasks.workunit.client.0.smithi031.stdout:9/953: truncate d2/dd/d75/d64/d5d/d59/d8e/f102 4263893 0 2024-04-22T23:45:56.584 INFO:tasks.workunit.client.0.smithi031.stdout:9/954: dread - d2/dc/f12b zero size 2024-04-22T23:45:56.585 INFO:tasks.workunit.client.0.smithi031.stdout:5/992: creat db/d16/d2a/dca/f144 x:0 0 0 2024-04-22T23:45:56.587 INFO:tasks.workunit.client.0.smithi031.stdout:9/955: unlink d2/dc/d8b/d3d/d40/da4/fb3 0 2024-04-22T23:45:56.589 INFO:tasks.workunit.client.0.smithi031.stdout:9/956: creat d2/dd/d65/f13a x:0 0 0 2024-04-22T23:45:56.590 INFO:tasks.workunit.client.0.smithi031.stdout:5/993: dwrite db/d16/d2a/d59/f75 [0,4194304] 0 2024-04-22T23:45:56.593 INFO:tasks.workunit.client.0.smithi031.stdout:9/957: getdents d2/dc/d6d/df1 0 2024-04-22T23:45:56.594 INFO:tasks.workunit.client.0.smithi031.stdout:5/994: symlink db/d8d/d11a/ddd/l145 0 2024-04-22T23:45:56.595 INFO:tasks.workunit.client.0.smithi031.stdout:9/958: symlink d2/dc/d6d/l13b 0 2024-04-22T23:45:56.596 INFO:tasks.workunit.client.0.smithi031.stdout:5/995: chown db/d16/d1d/d24/cde 9508 1 2024-04-22T23:45:56.596 INFO:tasks.workunit.client.0.smithi031.stdout:5/996: stat db/d16/d1d/d24/d32/ded 0 2024-04-22T23:45:56.599 INFO:tasks.workunit.client.0.smithi031.stdout:5/997: mknod db/d16/d2a/c146 0 2024-04-22T23:45:56.601 INFO:tasks.workunit.client.0.smithi031.stdout:5/998: unlink db/f25 0 2024-04-22T23:45:56.604 INFO:tasks.workunit.client.0.smithi031.stdout:5/999: rename db/d16/d1d/d24/d58/f85 to db/d16/d1d/d24/d27/d9e/d102/f147 0 2024-04-22T23:45:56.612 INFO:tasks.workunit.client.0.smithi031.stdout:9/959: write d2/d15/f54 [117687,89539] 0 2024-04-22T23:45:56.615 INFO:tasks.workunit.client.0.smithi031.stdout:9/960: mkdir d2/dd/d75/d4a/da0/d13c 0 2024-04-22T23:45:56.616 INFO:tasks.workunit.client.0.smithi031.stdout:9/961: chown d2/d15/d5a/f8d 429 1 2024-04-22T23:45:56.617 INFO:tasks.workunit.client.0.smithi031.stdout:9/962: readlink d2/dc/d8b/d3d/d40/da4/db1/dc5/l10b 0 2024-04-22T23:45:56.622 INFO:tasks.workunit.client.0.smithi031.stdout:9/963: dwrite d2/dd/d65/f13a [0,4194304] 0 2024-04-22T23:45:56.624 INFO:tasks.workunit.client.0.smithi031.stdout:9/964: chown d2/dd/d75/d80/cd9 364195942 1 2024-04-22T23:45:56.630 INFO:tasks.workunit.client.0.smithi031.stdout:9/965: link d2/dc/f1e d2/dc/d8b/d3d/f13d 0 2024-04-22T23:45:56.635 INFO:tasks.workunit.client.0.smithi031.stdout:9/966: write d2/dd/d75/d4a/f61 [2578475,42744] 0 2024-04-22T23:45:56.636 INFO:tasks.workunit.client.0.smithi031.stdout:9/967: chown d2/dc/f50 97 1 2024-04-22T23:45:56.637 INFO:tasks.workunit.client.0.smithi031.stdout:9/968: write d2/dc/d8b/d3d/d40/f39 [1640972,87438] 0 2024-04-22T23:45:56.640 INFO:tasks.workunit.client.0.smithi031.stdout:9/969: rename d2/dc/f43 to d2/dd/d75/d64/d5d/d59/f13e 0 2024-04-22T23:45:56.641 INFO:tasks.workunit.client.0.smithi031.stdout:9/970: write d2/dc/f17 [1218950,3468] 0 2024-04-22T23:45:56.643 INFO:tasks.workunit.client.0.smithi031.stdout:9/971: dread d2/dd/d75/d4a/da0/fe9 [0,4194304] 0 2024-04-22T23:45:56.646 INFO:tasks.workunit.client.0.smithi031.stdout:9/972: creat d2/dd/d75/d64/f13f x:0 0 0 2024-04-22T23:45:56.651 INFO:tasks.workunit.client.0.smithi031.stdout:9/973: write d2/dc/d8b/d3d/d40/dfd/ffe [5118595,14842] 0 2024-04-22T23:45:56.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/983: sync 2024-04-22T23:45:56.657 INFO:tasks.workunit.client.0.smithi031.stdout:9/974: dwrite d2/dd/d75/d64/d5d/d59/fb0 [0,4194304] 0 2024-04-22T23:45:56.657 INFO:tasks.workunit.client.0.smithi031.stdout:9/975: stat d2/dc/f83 0 2024-04-22T23:45:56.658 INFO:tasks.workunit.client.0.smithi031.stdout:3/984: symlink d4/d49/l15a 0 2024-04-22T23:45:56.665 INFO:tasks.workunit.client.0.smithi031.stdout:9/976: dwrite d2/dd/d75/d64/d5d/d59/fb0 [8388608,4194304] 0 2024-04-22T23:45:56.666 INFO:tasks.workunit.client.0.smithi031.stdout:9/977: readlink d2/dc/d8b/d3d/d40/da4/db1/le8 0 2024-04-22T23:45:56.669 INFO:tasks.workunit.client.0.smithi031.stdout:9/978: dread d2/dc/d8b/fdf [0,4194304] 0 2024-04-22T23:45:56.782 INFO:tasks.workunit.client.0.smithi031.stdout:9/979: write d2/dc/d8b/d3d/df9/f135 [3394275,128030] 0 2024-04-22T23:45:56.786 INFO:tasks.workunit.client.0.smithi031.stdout:9/980: write d2/dc/f83 [1135260,58741] 0 2024-04-22T23:45:56.802 INFO:tasks.workunit.client.0.smithi031.stdout:9/981: dread d2/dc/d3e/f41 [0,4194304] 0 2024-04-22T23:45:56.804 INFO:tasks.workunit.client.0.smithi031.stdout:9/982: mknod d2/dd/d75/d80/df8/c140 0 2024-04-22T23:45:56.804 INFO:tasks.workunit.client.0.smithi031.stdout:9/983: chown d2/dd/d75/d64/d5d/d59 44852023 1 2024-04-22T23:45:56.806 INFO:tasks.workunit.client.0.smithi031.stdout:9/984: mkdir d2/dc/d8b/d3d/d40/dfd/d141 0 2024-04-22T23:45:56.808 INFO:tasks.workunit.client.0.smithi031.stdout:9/985: creat d2/dd/d75/d4a/da0/d13c/f142 x:0 0 0 2024-04-22T23:45:56.811 INFO:tasks.workunit.client.0.smithi031.stdout:9/986: rename d2/dc/d8b/d3d/d40/da4/db1 to d2/dd/d75/df7/d122/d143 0 2024-04-22T23:45:56.812 INFO:tasks.workunit.client.0.smithi031.stdout:9/987: chown d2/d15/daf/cb4 1167 1 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:1/2: sync 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:4/14: sync 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/2: sync 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/3: rmdir - no directory 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/4: truncate - no filename 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:7/4: sync 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/5: dread - no filename 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/6: dread - no filename 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/7: rmdir - no directory 2024-04-22T23:45:56.912 INFO:tasks.workunit.client.1.smithi044.stdout:8/8: write - no filename 2024-04-22T23:45:56.913 INFO:tasks.workunit.client.1.smithi044.stdout:7/5: chown f1 1 1 2024-04-22T23:45:56.913 INFO:tasks.workunit.client.1.smithi044.stdout:7/6: readlink l0 0 2024-04-22T23:45:56.913 INFO:tasks.workunit.client.1.smithi044.stdout:7/7: chown f1 1905 1 2024-04-22T23:45:56.915 INFO:tasks.workunit.client.0.smithi031.stdout:3/985: sync 2024-04-22T23:45:56.916 INFO:tasks.workunit.client.0.smithi031.stdout:3/986: chown d4/d6/d11/d25/d44/d7f/d5f/fe5 0 1 2024-04-22T23:45:56.917 INFO:tasks.workunit.client.1.smithi044.stdout:8/9: creat f0 x:0 0 0 2024-04-22T23:45:56.918 INFO:tasks.workunit.client.0.smithi031.stdout:3/987: read - d4/d6/d11/d3e/fdd zero size 2024-04-22T23:45:56.919 INFO:tasks.workunit.client.1.smithi044.stdout:8/10: mknod c1 0 2024-04-22T23:45:56.920 INFO:tasks.workunit.client.1.smithi044.stdout:7/8: dwrite f1 [0,4194304] 0 2024-04-22T23:45:56.921 INFO:tasks.workunit.client.1.smithi044.stdout:8/11: write f0 [1026899,15808] 0 2024-04-22T23:45:56.921 INFO:tasks.workunit.client.1.smithi044.stdout:7/9: chown f1 77655502 1 2024-04-22T23:45:56.924 INFO:tasks.workunit.client.0.smithi031.stdout:3/988: creat d4/d54/d2c/d89/d2e/d8d/d108/f15b x:0 0 0 2024-04-22T23:45:56.927 INFO:tasks.workunit.client.1.smithi044.stdout:8/12: rename c1 to c2 0 2024-04-22T23:45:56.928 INFO:tasks.workunit.client.1.smithi044.stdout:8/13: fsync f0 0 2024-04-22T23:45:56.929 INFO:tasks.workunit.client.1.smithi044.stdout:7/10: creat f2 x:0 0 0 2024-04-22T23:45:56.930 INFO:tasks.workunit.client.1.smithi044.stdout:8/14: fdatasync f0 0 2024-04-22T23:45:56.930 INFO:tasks.workunit.client.0.smithi031.stdout:3/989: creat d4/d54/d2c/d89/d2e/f15c x:0 0 0 2024-04-22T23:45:56.931 INFO:tasks.workunit.client.1.smithi044.stdout:4/15: dwrite f1 [0,4194304] 0 2024-04-22T23:45:56.932 INFO:tasks.workunit.client.1.smithi044.stdout:8/15: fsync f0 0 2024-04-22T23:45:56.932 INFO:tasks.workunit.client.1.smithi044.stdout:8/16: readlink - no filename 2024-04-22T23:45:56.932 INFO:tasks.workunit.client.0.smithi031.stdout:3/990: chown d4/d54/d56/dae 7377 1 2024-04-22T23:45:56.935 INFO:tasks.workunit.client.1.smithi044.stdout:8/17: mknod c3 0 2024-04-22T23:45:56.936 INFO:tasks.workunit.client.1.smithi044.stdout:7/11: dread f1 [0,4194304] 0 2024-04-22T23:45:56.936 INFO:tasks.workunit.client.0.smithi031.stdout:3/991: creat d4/d6/d11/ddf/d115/f15d x:0 0 0 2024-04-22T23:45:56.938 INFO:tasks.workunit.client.1.smithi044.stdout:4/16: creat d5/f6 x:0 0 0 2024-04-22T23:45:56.939 INFO:tasks.workunit.client.1.smithi044.stdout:8/18: mknod c4 0 2024-04-22T23:45:56.942 INFO:tasks.workunit.client.0.smithi031.stdout:3/992: mkdir d4/d54/d6d/d98/d15e 0 2024-04-22T23:45:56.943 INFO:tasks.workunit.client.0.smithi031.stdout:3/993: chown d4/d54/d2c/d89/d2e/l90 3 1 2024-04-22T23:45:56.945 INFO:tasks.workunit.client.0.smithi031.stdout:3/994: creat d4/d49/d72/f15f x:0 0 0 2024-04-22T23:45:56.947 INFO:tasks.workunit.client.0.smithi031.stdout:3/995: truncate d4/d54/d2c/d89/d2e/f15c 1036209 0 2024-04-22T23:45:56.952 INFO:tasks.workunit.client.1.smithi044.stdout:8/19: dwrite f0 [0,4194304] 0 2024-04-22T23:45:56.954 INFO:tasks.workunit.client.1.smithi044.stdout:7/12: dwrite f1 [0,4194304] 0 2024-04-22T23:45:56.954 INFO:tasks.workunit.client.1.smithi044.stdout:8/20: chown f0 1953 1 2024-04-22T23:45:56.957 INFO:tasks.workunit.client.0.smithi031.stdout:3/996: write d4/d6/d11/d25/d44/f8e [489062,79500] 0 2024-04-22T23:45:56.957 INFO:tasks.workunit.client.1.smithi044.stdout:4/17: dwrite f1 [4194304,4194304] 0 2024-04-22T23:45:56.958 INFO:tasks.workunit.client.1.smithi044.stdout:7/13: creat f3 x:0 0 0 2024-04-22T23:45:56.960 INFO:tasks.workunit.client.1.smithi044.stdout:7/14: chown l0 333214400 1 2024-04-22T23:45:56.961 INFO:tasks.workunit.client.1.smithi044.stdout:8/21: creat f5 x:0 0 0 2024-04-22T23:45:56.963 INFO:tasks.workunit.client.1.smithi044.stdout:8/22: write f5 [325849,72065] 0 2024-04-22T23:45:56.964 INFO:tasks.workunit.client.1.smithi044.stdout:4/18: mknod d5/c7 0 2024-04-22T23:45:56.970 INFO:tasks.workunit.client.1.smithi044.stdout:7/15: dread f1 [0,4194304] 0 2024-04-22T23:45:56.970 INFO:tasks.workunit.client.1.smithi044.stdout:4/19: dread f1 [4194304,4194304] 0 2024-04-22T23:45:56.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/23: creat f6 x:0 0 0 2024-04-22T23:45:56.973 INFO:tasks.workunit.client.1.smithi044.stdout:7/16: dread f1 [0,4194304] 0 2024-04-22T23:45:56.975 INFO:tasks.workunit.client.0.smithi031.stdout:9/988: sync 2024-04-22T23:45:56.980 INFO:tasks.workunit.client.1.smithi044.stdout:7/17: symlink l4 0 2024-04-22T23:45:56.981 INFO:tasks.workunit.client.1.smithi044.stdout:4/20: dwrite f1 [0,4194304] 0 2024-04-22T23:45:56.982 INFO:tasks.workunit.client.1.smithi044.stdout:4/21: chown f1 2081073363 1 2024-04-22T23:45:56.986 INFO:tasks.workunit.client.1.smithi044.stdout:7/18: rename f3 to f5 0 2024-04-22T23:45:56.987 INFO:tasks.workunit.client.1.smithi044.stdout:6/31: sync 2024-04-22T23:45:56.987 INFO:tasks.workunit.client.1.smithi044.stdout:0/16: sync 2024-04-22T23:45:56.987 INFO:tasks.workunit.client.1.smithi044.stdout:2/20: sync 2024-04-22T23:45:56.987 INFO:tasks.workunit.client.1.smithi044.stdout:3/32: sync 2024-04-22T23:45:56.988 INFO:tasks.workunit.client.1.smithi044.stdout:9/63: sync 2024-04-22T23:45:56.988 INFO:tasks.workunit.client.1.smithi044.stdout:5/12: sync 2024-04-22T23:45:56.988 INFO:tasks.workunit.client.1.smithi044.stdout:4/22: dread f1 [0,4194304] 0 2024-04-22T23:45:56.989 INFO:tasks.workunit.client.1.smithi044.stdout:2/21: stat f3 0 2024-04-22T23:45:56.994 INFO:tasks.workunit.client.1.smithi044.stdout:0/17: write d0/d2/f5 [849399,12100] 0 2024-04-22T23:45:56.995 INFO:tasks.workunit.client.1.smithi044.stdout:0/18: chown d0 228 1 2024-04-22T23:45:56.999 INFO:tasks.workunit.client.1.smithi044.stdout:0/19: write d0/d2/f5 [1345097,4672] 0 2024-04-22T23:45:56.999 INFO:tasks.workunit.client.1.smithi044.stdout:5/13: rename f2 to f5 0 2024-04-22T23:45:57.000 INFO:tasks.workunit.client.1.smithi044.stdout:7/19: dwrite f2 [0,4194304] 0 2024-04-22T23:45:57.001 INFO:tasks.workunit.client.1.smithi044.stdout:8/24: link c2 c7 0 2024-04-22T23:45:57.003 INFO:tasks.workunit.client.1.smithi044.stdout:9/64: dwrite d2/f3 [0,4194304] 0 2024-04-22T23:45:57.004 INFO:tasks.workunit.client.1.smithi044.stdout:8/25: truncate f5 406768 0 2024-04-22T23:45:57.009 INFO:tasks.workunit.client.1.smithi044.stdout:4/23: mknod d5/c8 0 2024-04-22T23:45:57.009 INFO:tasks.workunit.client.1.smithi044.stdout:0/20: dread d0/f4 [0,4194304] 0 2024-04-22T23:45:57.010 INFO:tasks.workunit.client.1.smithi044.stdout:4/24: chown l4 0 1 2024-04-22T23:45:57.011 INFO:tasks.workunit.client.1.smithi044.stdout:2/22: creat f4 x:0 0 0 2024-04-22T23:45:57.012 INFO:tasks.workunit.client.1.smithi044.stdout:4/25: dread - d5/f6 zero size 2024-04-22T23:45:57.015 INFO:tasks.workunit.client.1.smithi044.stdout:4/26: dread - d5/f6 zero size 2024-04-22T23:45:57.015 INFO:tasks.workunit.client.1.smithi044.stdout:6/32: symlink d5/d8/lb 0 2024-04-22T23:45:57.016 INFO:tasks.workunit.client.1.smithi044.stdout:5/14: dread f0 [0,4194304] 0 2024-04-22T23:45:57.016 INFO:tasks.workunit.client.1.smithi044.stdout:8/26: dwrite f6 [0,4194304] 0 2024-04-22T23:45:57.017 INFO:tasks.workunit.client.1.smithi044.stdout:4/27: read - d5/f6 zero size 2024-04-22T23:45:57.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/3: sync 2024-04-22T23:45:57.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/4: write - no filename 2024-04-22T23:45:57.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/5: write - no filename 2024-04-22T23:45:57.018 INFO:tasks.workunit.client.1.smithi044.stdout:1/6: fdatasync - no filename 2024-04-22T23:45:57.018 INFO:tasks.workunit.client.1.smithi044.stdout:1/7: truncate - no filename 2024-04-22T23:45:57.018 INFO:tasks.workunit.client.1.smithi044.stdout:1/8: dread - no filename 2024-04-22T23:45:57.018 INFO:tasks.workunit.client.1.smithi044.stdout:1/9: link - no file 2024-04-22T23:45:57.018 INFO:tasks.workunit.client.1.smithi044.stdout:3/33: truncate f7 356322 0 2024-04-22T23:45:57.021 INFO:tasks.workunit.client.1.smithi044.stdout:6/33: dread f0 [0,4194304] 0 2024-04-22T23:45:57.024 INFO:tasks.workunit.client.1.smithi044.stdout:6/34: dread - d5/d8/fa zero size 2024-04-22T23:45:57.024 INFO:tasks.workunit.client.1.smithi044.stdout:6/35: fdatasync f1 0 2024-04-22T23:45:57.026 INFO:tasks.workunit.client.1.smithi044.stdout:7/20: fsync f2 0 2024-04-22T23:45:57.026 INFO:tasks.workunit.client.1.smithi044.stdout:7/21: rmdir - no directory 2024-04-22T23:45:57.026 INFO:tasks.workunit.client.1.smithi044.stdout:7/22: dread - f5 zero size 2024-04-22T23:45:57.036 INFO:tasks.workunit.client.1.smithi044.stdout:2/23: symlink l5 0 2024-04-22T23:45:57.041 INFO:tasks.workunit.client.1.smithi044.stdout:5/15: creat f6 x:0 0 0 2024-04-22T23:45:57.043 INFO:tasks.workunit.client.1.smithi044.stdout:4/28: mknod d5/c9 0 2024-04-22T23:45:57.045 INFO:tasks.workunit.client.1.smithi044.stdout:8/27: mknod c8 0 2024-04-22T23:45:57.049 INFO:tasks.workunit.client.1.smithi044.stdout:3/34: symlink d9/le 0 2024-04-22T23:45:57.049 INFO:tasks.workunit.client.1.smithi044.stdout:3/35: stat l4 0 2024-04-22T23:45:57.053 INFO:tasks.workunit.client.1.smithi044.stdout:8/28: dwrite f5 [0,4194304] 0 2024-04-22T23:45:57.056 INFO:tasks.workunit.client.1.smithi044.stdout:7/23: write f1 [2638637,27579] 0 2024-04-22T23:45:57.061 INFO:tasks.workunit.client.1.smithi044.stdout:8/29: dwrite f0 [0,4194304] 0 2024-04-22T23:45:57.069 INFO:tasks.workunit.client.1.smithi044.stdout:2/24: creat f6 x:0 0 0 2024-04-22T23:45:57.080 INFO:tasks.workunit.client.1.smithi044.stdout:1/10: mknod c0 0 2024-04-22T23:45:57.080 INFO:tasks.workunit.client.1.smithi044.stdout:1/11: dread - no filename 2024-04-22T23:45:57.080 INFO:tasks.workunit.client.1.smithi044.stdout:1/12: dread - no filename 2024-04-22T23:45:57.081 INFO:tasks.workunit.client.1.smithi044.stdout:1/13: write - no filename 2024-04-22T23:45:57.082 INFO:tasks.workunit.client.1.smithi044.stdout:3/36: rmdir d9 39 2024-04-22T23:45:57.087 INFO:tasks.workunit.client.1.smithi044.stdout:7/24: mknod c6 0 2024-04-22T23:45:57.088 INFO:tasks.workunit.client.1.smithi044.stdout:7/25: readlink l4 0 2024-04-22T23:45:57.090 INFO:tasks.workunit.client.1.smithi044.stdout:8/30: mknod c9 0 2024-04-22T23:45:57.090 INFO:tasks.workunit.client.1.smithi044.stdout:8/31: stat c4 0 2024-04-22T23:45:57.093 INFO:tasks.workunit.client.0.smithi031.stdout:9/989: creat d2/dc/d6d/df1/f144 x:0 0 0 2024-04-22T23:45:57.094 INFO:tasks.workunit.client.1.smithi044.stdout:2/25: creat f7 x:0 0 0 2024-04-22T23:45:57.094 INFO:tasks.workunit.client.0.smithi031.stdout:3/997: mknod d4/d54/d6d/d98/d15e/c160 0 2024-04-22T23:45:57.096 INFO:tasks.workunit.client.1.smithi044.stdout:0/21: truncate d0/d2/f5 261140 0 2024-04-22T23:45:57.099 INFO:tasks.workunit.client.0.smithi031.stdout:3/998: mknod d4/d6/d11/d3e/c161 0 2024-04-22T23:45:57.099 INFO:tasks.workunit.client.0.smithi031.stdout:3/999: chown d4/d54/d56/f87 536221 1 2024-04-22T23:45:57.100 INFO:tasks.workunit.client.1.smithi044.stdout:1/14: unlink c0 0 2024-04-22T23:45:57.102 INFO:tasks.workunit.client.1.smithi044.stdout:6/36: truncate f0 1488315 0 2024-04-22T23:45:57.103 INFO:tasks.workunit.client.0.smithi031.stdout:9/990: creat d2/dd/d75/f145 x:0 0 0 2024-04-22T23:45:57.104 INFO:tasks.workunit.client.1.smithi044.stdout:6/37: dread d5/f6 [0,4194304] 0 2024-04-22T23:45:57.104 INFO:tasks.workunit.client.0.smithi031.stdout:9/991: dread - d2/dd/d65/fbf zero size 2024-04-22T23:45:57.105 INFO:tasks.workunit.client.1.smithi044.stdout:7/26: creat f7 x:0 0 0 2024-04-22T23:45:57.105 INFO:tasks.workunit.client.1.smithi044.stdout:7/27: read f1 [2218471,7897] 0 2024-04-22T23:45:57.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/28: stat f1 0 2024-04-22T23:45:57.106 INFO:tasks.workunit.client.1.smithi044.stdout:8/32: unlink c9 0 2024-04-22T23:45:57.107 INFO:tasks.workunit.client.1.smithi044.stdout:8/33: fsync f6 0 2024-04-22T23:45:57.108 INFO:tasks.workunit.client.1.smithi044.stdout:9/65: getdents d2/d6 0 2024-04-22T23:45:57.109 INFO:tasks.workunit.client.0.smithi031.stdout:9/992: dread d2/d15/f2c [0,4194304] 0 2024-04-22T23:45:57.109 INFO:tasks.workunit.client.0.smithi031.stdout:9/993: chown d2/dd/d75/d4a 1638 1 2024-04-22T23:45:57.110 INFO:tasks.workunit.client.0.smithi031.stdout:9/994: chown d2/dd3 37 1 2024-04-22T23:45:57.110 INFO:tasks.workunit.client.1.smithi044.stdout:7/29: dread f2 [0,4194304] 0 2024-04-22T23:45:57.111 INFO:tasks.workunit.client.1.smithi044.stdout:7/30: chown f2 891379704 1 2024-04-22T23:45:57.112 INFO:tasks.workunit.client.1.smithi044.stdout:7/31: write f1 [3016503,121241] 0 2024-04-22T23:45:57.114 INFO:tasks.workunit.client.1.smithi044.stdout:2/26: dread f1 [0,4194304] 0 2024-04-22T23:45:57.114 INFO:tasks.workunit.client.0.smithi031.stdout:9/995: rmdir d2/dd/d75/d80/df8 39 2024-04-22T23:45:57.116 INFO:tasks.workunit.client.1.smithi044.stdout:4/29: link c2 d5/ca 0 2024-04-22T23:45:57.118 INFO:tasks.workunit.client.1.smithi044.stdout:0/22: dread d0/f4 [0,4194304] 0 2024-04-22T23:45:57.118 INFO:tasks.workunit.client.1.smithi044.stdout:1/15: mkdir d1 0 2024-04-22T23:45:57.118 INFO:tasks.workunit.client.1.smithi044.stdout:1/16: dread - no filename 2024-04-22T23:45:57.118 INFO:tasks.workunit.client.1.smithi044.stdout:4/30: truncate d5/f6 823309 0 2024-04-22T23:45:57.119 INFO:tasks.workunit.client.1.smithi044.stdout:3/37: symlink d9/lf 0 2024-04-22T23:45:57.120 INFO:tasks.workunit.client.1.smithi044.stdout:2/27: dread f1 [0,4194304] 0 2024-04-22T23:45:57.121 INFO:tasks.workunit.client.1.smithi044.stdout:2/28: write f4 [709397,104300] 0 2024-04-22T23:45:57.125 INFO:tasks.workunit.client.1.smithi044.stdout:6/38: creat d5/fc x:0 0 0 2024-04-22T23:45:57.128 INFO:tasks.workunit.client.1.smithi044.stdout:5/16: dwrite f1 [0,4194304] 0 2024-04-22T23:45:57.133 INFO:tasks.workunit.client.1.smithi044.stdout:8/34: symlink la 0 2024-04-22T23:45:57.135 INFO:tasks.workunit.client.1.smithi044.stdout:9/66: mknod d2/dc/c16 0 2024-04-22T23:45:57.137 INFO:tasks.workunit.client.0.smithi031.stdout:9/996: rename d2/dc/d8b/d3d/d40/da4/le0 to d2/l146 0 2024-04-22T23:45:57.139 INFO:tasks.workunit.client.1.smithi044.stdout:7/32: creat f8 x:0 0 0 2024-04-22T23:45:57.141 INFO:tasks.workunit.client.0.smithi031.stdout:9/997: fsync d2/dc/d3e/f107 0 2024-04-22T23:45:57.143 INFO:tasks.workunit.client.1.smithi044.stdout:9/67: dwrite d2/d5/f7 [0,4194304] 0 2024-04-22T23:45:57.143 INFO:tasks.workunit.client.1.smithi044.stdout:4/31: mkdir d5/db 0 2024-04-22T23:45:57.144 INFO:tasks.workunit.client.1.smithi044.stdout:9/68: chown d2/d5/f7 1 1 2024-04-22T23:45:57.146 INFO:tasks.workunit.client.1.smithi044.stdout:3/38: fdatasync f5 0 2024-04-22T23:45:57.147 INFO:tasks.workunit.client.0.smithi031.stdout:9/998: dwrite d2/dc/f10 [0,4194304] 0 2024-04-22T23:45:57.149 INFO:tasks.workunit.client.0.smithi031.stdout:9/999: write d2/dd/d75/d4a/da0/d13c/f142 [760113,75203] 0 2024-04-22T23:45:57.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/29: fdatasync f1 0 2024-04-22T23:45:57.152 INFO:tasks.workunit.client.1.smithi044.stdout:2/30: dread - f6 zero size 2024-04-22T23:45:57.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -rf -- ./tmp.7vhw0Vx6mu 2024-04-22T23:45:57.156 INFO:tasks.workunit.client.1.smithi044.stdout:6/39: mkdir d5/dd 0 2024-04-22T23:45:57.156 INFO:tasks.workunit.client.1.smithi044.stdout:6/40: chown d5 0 1 2024-04-22T23:45:57.158 INFO:tasks.workunit.client.1.smithi044.stdout:5/17: truncate f0 1217501 0 2024-04-22T23:45:57.158 INFO:tasks.workunit.client.1.smithi044.stdout:5/18: readlink l3 0 2024-04-22T23:45:57.161 INFO:tasks.workunit.client.1.smithi044.stdout:8/35: mkdir db 0 2024-04-22T23:45:57.164 INFO:tasks.workunit.client.1.smithi044.stdout:7/33: symlink l9 0 2024-04-22T23:45:57.166 INFO:tasks.workunit.client.1.smithi044.stdout:1/17: symlink d1/l2 0 2024-04-22T23:45:57.167 INFO:tasks.workunit.client.1.smithi044.stdout:4/32: creat d5/fc x:0 0 0 2024-04-22T23:45:57.169 INFO:tasks.workunit.client.1.smithi044.stdout:9/69: mknod d2/dc/c17 0 2024-04-22T23:45:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:3/39: rmdir d9 39 2024-04-22T23:45:57.174 INFO:tasks.workunit.client.1.smithi044.stdout:2/31: mkdir d8 0 2024-04-22T23:45:57.176 INFO:tasks.workunit.client.1.smithi044.stdout:6/41: rename d5/fc to d5/dd/fe 0 2024-04-22T23:45:57.179 INFO:tasks.workunit.client.1.smithi044.stdout:7/34: rename f1 to fa 0 2024-04-22T23:45:57.181 INFO:tasks.workunit.client.1.smithi044.stdout:1/18: mknod d1/c3 0 2024-04-22T23:45:57.181 INFO:tasks.workunit.client.1.smithi044.stdout:1/19: dread - no filename 2024-04-22T23:45:57.185 INFO:tasks.workunit.client.1.smithi044.stdout:9/70: unlink d2/dc/c16 0 2024-04-22T23:45:57.187 INFO:tasks.workunit.client.1.smithi044.stdout:3/40: chown d9/lf 19 1 2024-04-22T23:45:57.188 INFO:tasks.workunit.client.1.smithi044.stdout:4/33: dwrite f1 [0,4194304] 0 2024-04-22T23:45:57.189 INFO:tasks.workunit.client.1.smithi044.stdout:4/34: write d5/f6 [1283699,112775] 0 2024-04-22T23:45:57.194 INFO:tasks.workunit.client.1.smithi044.stdout:6/42: mknod d5/dd/cf 0 2024-04-22T23:45:57.194 INFO:tasks.workunit.client.1.smithi044.stdout:5/19: stat f0 0 2024-04-22T23:45:57.196 INFO:tasks.workunit.client.1.smithi044.stdout:8/36: creat db/fc x:0 0 0 2024-04-22T23:45:57.197 INFO:tasks.workunit.client.1.smithi044.stdout:7/35: symlink lb 0 2024-04-22T23:45:57.199 INFO:tasks.workunit.client.1.smithi044.stdout:1/20: mknod d1/c4 0 2024-04-22T23:45:57.199 INFO:tasks.workunit.client.1.smithi044.stdout:1/21: fdatasync - no filename 2024-04-22T23:45:57.199 INFO:tasks.workunit.client.1.smithi044.stdout:1/22: write - no filename 2024-04-22T23:45:57.199 INFO:tasks.workunit.client.1.smithi044.stdout:1/23: read - no filename 2024-04-22T23:45:57.199 INFO:tasks.workunit.client.1.smithi044.stdout:1/24: fsync - no filename 2024-04-22T23:45:57.200 INFO:tasks.workunit.client.1.smithi044.stdout:1/25: dread - no filename 2024-04-22T23:45:57.200 INFO:tasks.workunit.client.1.smithi044.stdout:1/26: dwrite - no filename 2024-04-22T23:45:57.201 INFO:tasks.workunit.client.1.smithi044.stdout:9/71: creat d2/dc/f18 x:0 0 0 2024-04-22T23:45:57.201 INFO:tasks.workunit.client.1.smithi044.stdout:9/72: dread - d2/dc/f18 zero size 2024-04-22T23:45:57.202 INFO:tasks.workunit.client.1.smithi044.stdout:3/41: creat d9/db/f10 x:0 0 0 2024-04-22T23:45:57.204 INFO:tasks.workunit.client.1.smithi044.stdout:2/32: creat d8/f9 x:0 0 0 2024-04-22T23:45:57.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/23: read d0/d2/f5 [81678,94664] 0 2024-04-22T23:45:57.205 INFO:tasks.workunit.client.1.smithi044.stdout:6/43: creat d5/dd/f10 x:0 0 0 2024-04-22T23:45:57.206 INFO:tasks.workunit.client.1.smithi044.stdout:6/44: write d5/dd/f10 [1021479,31849] 0 2024-04-22T23:45:57.206 INFO:tasks.workunit.client.1.smithi044.stdout:8/37: creat db/fd x:0 0 0 2024-04-22T23:45:57.208 INFO:tasks.workunit.client.1.smithi044.stdout:7/36: creat fc x:0 0 0 2024-04-22T23:45:57.209 INFO:tasks.workunit.client.1.smithi044.stdout:1/27: mknod d1/c5 0 2024-04-22T23:45:57.211 INFO:tasks.workunit.client.1.smithi044.stdout:3/42: chown c3 9819524 1 2024-04-22T23:45:57.212 INFO:tasks.workunit.client.1.smithi044.stdout:4/35: mkdir d5/db/dd 0 2024-04-22T23:45:57.214 INFO:tasks.workunit.client.1.smithi044.stdout:0/24: truncate d0/d2/f5 1242741 0 2024-04-22T23:45:57.216 INFO:tasks.workunit.client.1.smithi044.stdout:8/38: creat db/fe x:0 0 0 2024-04-22T23:45:57.217 INFO:tasks.workunit.client.1.smithi044.stdout:0/25: dread d0/f4 [0,4194304] 0 2024-04-22T23:45:57.218 INFO:tasks.workunit.client.1.smithi044.stdout:6/45: rename c4 to d5/dd/c11 0 2024-04-22T23:45:57.219 INFO:tasks.workunit.client.1.smithi044.stdout:6/46: dread - d5/d8/fa zero size 2024-04-22T23:45:57.219 INFO:tasks.workunit.client.1.smithi044.stdout:7/37: write f5 [333631,19039] 0 2024-04-22T23:45:57.220 INFO:tasks.workunit.client.1.smithi044.stdout:1/28: symlink d1/l6 0 2024-04-22T23:45:57.220 INFO:tasks.workunit.client.1.smithi044.stdout:1/29: write - no filename 2024-04-22T23:45:57.220 INFO:tasks.workunit.client.1.smithi044.stdout:1/30: dread - no filename 2024-04-22T23:45:57.221 INFO:tasks.workunit.client.1.smithi044.stdout:5/20: link c4 c7 0 2024-04-22T23:45:57.223 INFO:tasks.workunit.client.1.smithi044.stdout:3/43: rename d9/ca to d9/db/c11 0 2024-04-22T23:45:57.225 INFO:tasks.workunit.client.1.smithi044.stdout:7/38: mkdir dd 0 2024-04-22T23:45:57.226 INFO:tasks.workunit.client.1.smithi044.stdout:9/73: creat d2/f19 x:0 0 0 2024-04-22T23:45:57.226 INFO:tasks.workunit.client.1.smithi044.stdout:6/47: dread f0 [0,4194304] 0 2024-04-22T23:45:57.227 INFO:tasks.workunit.client.1.smithi044.stdout:1/31: creat d1/f7 x:0 0 0 2024-04-22T23:45:57.228 INFO:tasks.workunit.client.1.smithi044.stdout:6/48: stat d5/d8/fa 0 2024-04-22T23:45:57.228 INFO:tasks.workunit.client.1.smithi044.stdout:4/36: symlink d5/db/dd/le 0 2024-04-22T23:45:57.230 INFO:tasks.workunit.client.1.smithi044.stdout:5/21: creat f8 x:0 0 0 2024-04-22T23:45:57.231 INFO:tasks.workunit.client.1.smithi044.stdout:4/37: chown d5/f6 32492 1 2024-04-22T23:45:57.231 INFO:tasks.workunit.client.1.smithi044.stdout:5/22: stat f1 0 2024-04-22T23:45:57.231 INFO:tasks.workunit.client.1.smithi044.stdout:6/49: truncate d5/d8/fa 176151 0 2024-04-22T23:45:57.233 INFO:tasks.workunit.client.1.smithi044.stdout:3/44: mkdir d9/d12 0 2024-04-22T23:45:57.234 INFO:tasks.workunit.client.1.smithi044.stdout:7/39: dwrite fc [0,4194304] 0 2024-04-22T23:45:57.235 INFO:tasks.workunit.client.1.smithi044.stdout:0/26: rename d0/f4 to d0/d2/f6 0 2024-04-22T23:45:57.239 INFO:tasks.workunit.client.1.smithi044.stdout:9/74: mknod d2/d9/c1a 0 2024-04-22T23:45:57.241 INFO:tasks.workunit.client.1.smithi044.stdout:9/75: read d2/d6/f13 [866366,8359] 0 2024-04-22T23:45:57.243 INFO:tasks.workunit.client.1.smithi044.stdout:7/40: dwrite fc [0,4194304] 0 2024-04-22T23:45:57.244 INFO:tasks.workunit.client.1.smithi044.stdout:7/41: dread - f7 zero size 2024-04-22T23:45:57.246 INFO:tasks.workunit.client.1.smithi044.stdout:6/50: creat d5/f12 x:0 0 0 2024-04-22T23:45:57.248 INFO:tasks.workunit.client.1.smithi044.stdout:4/38: symlink d5/db/dd/lf 0 2024-04-22T23:45:57.249 INFO:tasks.workunit.client.1.smithi044.stdout:3/45: creat d9/db/f13 x:0 0 0 2024-04-22T23:45:57.250 INFO:tasks.workunit.client.1.smithi044.stdout:6/51: chown d5/f6 0 1 2024-04-22T23:45:57.250 INFO:tasks.workunit.client.1.smithi044.stdout:3/46: dread - d9/db/f13 zero size 2024-04-22T23:45:57.252 INFO:tasks.workunit.client.1.smithi044.stdout:6/52: chown f1 711 1 2024-04-22T23:45:57.253 INFO:tasks.workunit.client.1.smithi044.stdout:0/27: mknod d0/d2/c7 0 2024-04-22T23:45:57.253 INFO:tasks.workunit.client.1.smithi044.stdout:3/47: read - d9/db/f13 zero size 2024-04-22T23:45:57.255 INFO:tasks.workunit.client.1.smithi044.stdout:9/76: mknod d2/d9/c1b 0 2024-04-22T23:45:57.255 INFO:tasks.workunit.client.1.smithi044.stdout:0/28: readlink d0/d2/l3 0 2024-04-22T23:45:57.256 INFO:tasks.workunit.client.1.smithi044.stdout:7/42: dwrite f2 [0,4194304] 0 2024-04-22T23:45:57.262 INFO:tasks.workunit.client.1.smithi044.stdout:4/39: rename d5/fc to d5/db/dd/f10 0 2024-04-22T23:45:57.262 INFO:tasks.workunit.client.1.smithi044.stdout:4/40: fsync f1 0 2024-04-22T23:45:57.263 INFO:tasks.workunit.client.1.smithi044.stdout:3/48: mkdir d9/d14 0 2024-04-22T23:45:57.263 INFO:tasks.workunit.client.1.smithi044.stdout:4/41: dread - d5/db/dd/f10 zero size 2024-04-22T23:45:57.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/49: read - f8 zero size 2024-04-22T23:45:57.277 INFO:tasks.workunit.client.1.smithi044.stdout:8/39: getdents db 0 2024-04-22T23:45:57.279 INFO:tasks.workunit.client.1.smithi044.stdout:4/42: creat d5/db/dd/f11 x:0 0 0 2024-04-22T23:45:57.280 INFO:tasks.workunit.client.1.smithi044.stdout:3/50: rmdir d9 39 2024-04-22T23:45:57.284 INFO:tasks.workunit.client.1.smithi044.stdout:4/43: dread d5/f6 [0,4194304] 0 2024-04-22T23:45:57.285 INFO:tasks.workunit.client.1.smithi044.stdout:7/43: creat dd/fe x:0 0 0 2024-04-22T23:45:57.285 INFO:tasks.workunit.client.1.smithi044.stdout:1/32: rmdir d1 39 2024-04-22T23:45:57.287 INFO:tasks.workunit.client.1.smithi044.stdout:8/40: creat db/ff x:0 0 0 2024-04-22T23:45:57.288 INFO:tasks.workunit.client.1.smithi044.stdout:5/23: getdents . 0 2024-04-22T23:45:57.289 INFO:tasks.workunit.client.1.smithi044.stdout:5/24: write f6 [714742,59838] 0 2024-04-22T23:45:57.291 INFO:tasks.workunit.client.1.smithi044.stdout:3/51: fdatasync f5 0 2024-04-22T23:45:57.293 INFO:tasks.workunit.client.1.smithi044.stdout:9/77: creat d2/f1c x:0 0 0 2024-04-22T23:45:57.298 INFO:tasks.workunit.client.1.smithi044.stdout:0/29: dread d0/d2/f5 [0,4194304] 0 2024-04-22T23:45:57.299 INFO:tasks.workunit.client.1.smithi044.stdout:7/44: creat dd/ff x:0 0 0 2024-04-22T23:45:57.301 INFO:tasks.workunit.client.1.smithi044.stdout:5/25: symlink l9 0 2024-04-22T23:45:57.302 INFO:tasks.workunit.client.1.smithi044.stdout:5/26: readlink l3 0 2024-04-22T23:45:57.303 INFO:tasks.workunit.client.1.smithi044.stdout:5/27: chown f8 936628 1 2024-04-22T23:45:57.303 INFO:tasks.workunit.client.1.smithi044.stdout:4/44: dwrite d5/db/dd/f10 [0,4194304] 0 2024-04-22T23:45:57.306 INFO:tasks.workunit.client.1.smithi044.stdout:3/52: read - d9/db/f10 zero size 2024-04-22T23:45:57.308 INFO:tasks.workunit.client.1.smithi044.stdout:1/33: mkdir d1/d8 0 2024-04-22T23:45:57.310 INFO:tasks.workunit.client.1.smithi044.stdout:0/30: creat d0/f8 x:0 0 0 2024-04-22T23:45:57.311 INFO:tasks.workunit.client.1.smithi044.stdout:0/31: write d0/f8 [1025072,864] 0 2024-04-22T23:45:57.314 INFO:tasks.workunit.client.1.smithi044.stdout:8/41: link db/fe db/f10 0 2024-04-22T23:45:57.317 INFO:tasks.workunit.client.1.smithi044.stdout:5/28: unlink f1 0 2024-04-22T23:45:57.322 INFO:tasks.workunit.client.1.smithi044.stdout:3/53: mkdir d9/db/d15 0 2024-04-22T23:45:57.323 INFO:tasks.workunit.client.1.smithi044.stdout:0/32: dread d0/f8 [0,4194304] 0 2024-04-22T23:45:57.323 INFO:tasks.workunit.client.1.smithi044.stdout:3/54: fdatasync d9/db/f13 0 2024-04-22T23:45:57.327 INFO:tasks.workunit.client.1.smithi044.stdout:4/45: dwrite d5/f6 [0,4194304] 0 2024-04-22T23:45:57.329 INFO:tasks.workunit.client.1.smithi044.stdout:4/46: write f1 [7753462,2587] 0 2024-04-22T23:45:57.336 INFO:tasks.workunit.client.1.smithi044.stdout:3/55: dwrite d9/db/f13 [0,4194304] 0 2024-04-22T23:45:57.336 INFO:tasks.workunit.client.1.smithi044.stdout:3/56: fsync f8 0 2024-04-22T23:45:57.338 INFO:tasks.workunit.client.1.smithi044.stdout:5/29: creat fa x:0 0 0 2024-04-22T23:45:57.338 INFO:tasks.workunit.client.1.smithi044.stdout:3/57: write d9/db/f13 [1920237,120558] 0 2024-04-22T23:45:57.338 INFO:tasks.workunit.client.1.smithi044.stdout:4/47: dread d5/db/dd/f10 [0,4194304] 0 2024-04-22T23:45:57.340 INFO:tasks.workunit.client.1.smithi044.stdout:5/30: write f8 [473314,110901] 0 2024-04-22T23:45:57.342 INFO:tasks.workunit.client.1.smithi044.stdout:1/34: mknod d1/d8/c9 0 2024-04-22T23:45:57.344 INFO:tasks.workunit.client.1.smithi044.stdout:0/33: symlink d0/l9 0 2024-04-22T23:45:57.349 INFO:tasks.workunit.client.1.smithi044.stdout:3/58: dwrite d9/db/f10 [0,4194304] 0 2024-04-22T23:45:57.354 INFO:tasks.workunit.client.1.smithi044.stdout:8/42: link f5 db/f11 0 2024-04-22T23:45:57.355 INFO:tasks.workunit.client.1.smithi044.stdout:7/45: link lb dd/l10 0 2024-04-22T23:45:57.357 INFO:tasks.workunit.client.1.smithi044.stdout:7/46: fdatasync dd/fe 0 2024-04-22T23:45:57.358 INFO:tasks.workunit.client.1.smithi044.stdout:7/47: dread - f8 zero size 2024-04-22T23:45:57.363 INFO:tasks.workunit.client.1.smithi044.stdout:8/43: dwrite db/fc [0,4194304] 0 2024-04-22T23:45:57.364 INFO:tasks.workunit.client.1.smithi044.stdout:7/48: dread f5 [0,4194304] 0 2024-04-22T23:45:57.370 INFO:tasks.workunit.client.1.smithi044.stdout:1/35: mknod d1/d8/ca 0 2024-04-22T23:45:57.370 INFO:tasks.workunit.client.1.smithi044.stdout:1/36: readlink d1/l6 0 2024-04-22T23:45:57.380 INFO:tasks.workunit.client.1.smithi044.stdout:1/37: dwrite d1/f7 [0,4194304] 0 2024-04-22T23:45:57.382 INFO:tasks.workunit.client.1.smithi044.stdout:1/38: read d1/f7 [3631831,40429] 0 2024-04-22T23:45:57.384 INFO:tasks.workunit.client.1.smithi044.stdout:1/39: write d1/f7 [3694008,14445] 0 2024-04-22T23:45:57.389 INFO:tasks.workunit.client.1.smithi044.stdout:0/34: dwrite d0/f8 [0,4194304] 0 2024-04-22T23:45:57.390 INFO:tasks.workunit.client.1.smithi044.stdout:3/59: write d9/db/f10 [5069758,10045] 0 2024-04-22T23:45:57.391 INFO:tasks.workunit.client.1.smithi044.stdout:0/35: write d0/f8 [1230550,11302] 0 2024-04-22T23:45:57.399 INFO:tasks.workunit.client.1.smithi044.stdout:8/44: creat db/f12 x:0 0 0 2024-04-22T23:45:57.399 INFO:tasks.workunit.client.1.smithi044.stdout:8/45: readlink la 0 2024-04-22T23:45:57.399 INFO:tasks.workunit.client.1.smithi044.stdout:8/46: stat db/fe 0 2024-04-22T23:45:57.400 INFO:tasks.workunit.client.1.smithi044.stdout:7/49: mkdir dd/d11 0 2024-04-22T23:45:57.406 INFO:tasks.workunit.client.1.smithi044.stdout:1/40: mknod d1/d8/cb 0 2024-04-22T23:45:57.407 INFO:tasks.workunit.client.1.smithi044.stdout:1/41: chown d1/d8/ca 6294807 1 2024-04-22T23:45:57.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/48: truncate d5/f6 3075055 0 2024-04-22T23:45:57.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/49: stat d5/c9 0 2024-04-22T23:45:57.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/50: chown d5/db 3849 1 2024-04-22T23:45:57.417 INFO:tasks.workunit.client.1.smithi044.stdout:0/36: creat d0/d2/fa x:0 0 0 2024-04-22T23:45:57.419 INFO:tasks.workunit.client.1.smithi044.stdout:8/47: symlink db/l13 0 2024-04-22T23:45:57.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/31: getdents . 0 2024-04-22T23:45:57.421 INFO:tasks.workunit.client.1.smithi044.stdout:7/50: rmdir dd 39 2024-04-22T23:45:57.425 INFO:tasks.workunit.client.1.smithi044.stdout:1/42: unlink d1/d8/cb 0 2024-04-22T23:45:57.427 INFO:tasks.workunit.client.1.smithi044.stdout:4/51: symlink d5/db/l12 0 2024-04-22T23:45:57.428 INFO:tasks.workunit.client.1.smithi044.stdout:3/60: fsync f7 0 2024-04-22T23:45:57.430 INFO:tasks.workunit.client.1.smithi044.stdout:3/61: dread f7 [0,4194304] 0 2024-04-22T23:45:57.430 INFO:tasks.workunit.client.1.smithi044.stdout:1/43: dread d1/f7 [0,4194304] 0 2024-04-22T23:45:57.430 INFO:tasks.workunit.client.1.smithi044.stdout:8/48: mkdir db/d14 0 2024-04-22T23:45:57.431 INFO:tasks.workunit.client.1.smithi044.stdout:8/49: dread - db/f12 zero size 2024-04-22T23:45:57.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/62: dread d9/db/f10 [0,4194304] 0 2024-04-22T23:45:57.437 INFO:tasks.workunit.client.1.smithi044.stdout:4/52: rename d5/c9 to d5/db/dd/c13 0 2024-04-22T23:45:57.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/53: chown f1 1759982 1 2024-04-22T23:45:57.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/54: chown d5/c7 2016329 1 2024-04-22T23:45:57.439 INFO:tasks.workunit.client.1.smithi044.stdout:4/55: write d5/db/dd/f11 [180612,93172] 0 2024-04-22T23:45:57.440 INFO:tasks.workunit.client.1.smithi044.stdout:5/32: write f0 [2208682,56710] 0 2024-04-22T23:45:57.440 INFO:tasks.workunit.client.1.smithi044.stdout:5/33: stat f0 0 2024-04-22T23:45:57.445 INFO:tasks.workunit.client.1.smithi044.stdout:7/51: symlink dd/l12 0 2024-04-22T23:45:57.447 INFO:tasks.workunit.client.1.smithi044.stdout:4/56: mknod d5/db/dd/c14 0 2024-04-22T23:45:57.450 INFO:tasks.workunit.client.1.smithi044.stdout:5/34: dwrite f8 [0,4194304] 0 2024-04-22T23:45:57.450 INFO:tasks.workunit.client.1.smithi044.stdout:8/50: creat db/d14/f15 x:0 0 0 2024-04-22T23:45:57.450 INFO:tasks.workunit.client.1.smithi044.stdout:3/63: symlink d9/d14/l16 0 2024-04-22T23:45:57.452 INFO:tasks.workunit.client.1.smithi044.stdout:3/64: write d9/db/f13 [1648737,105175] 0 2024-04-22T23:45:57.456 INFO:tasks.workunit.client.1.smithi044.stdout:7/52: getdents dd/d11 0 2024-04-22T23:45:57.457 INFO:tasks.workunit.client.1.smithi044.stdout:3/65: fsync d9/db/f13 0 2024-04-22T23:45:57.458 INFO:tasks.workunit.client.1.smithi044.stdout:4/57: fsync d5/f6 0 2024-04-22T23:45:57.459 INFO:tasks.workunit.client.1.smithi044.stdout:3/66: read - f8 zero size 2024-04-22T23:45:57.461 INFO:tasks.workunit.client.1.smithi044.stdout:1/44: getdents d1 0 2024-04-22T23:45:57.462 INFO:tasks.workunit.client.1.smithi044.stdout:8/51: dwrite db/f12 [0,4194304] 0 2024-04-22T23:45:57.463 INFO:tasks.workunit.client.1.smithi044.stdout:1/45: chown d1/c3 8 1 2024-04-22T23:45:57.464 INFO:tasks.workunit.client.1.smithi044.stdout:5/35: rename c4 to cb 0 2024-04-22T23:45:57.469 INFO:tasks.workunit.client.1.smithi044.stdout:3/67: creat d9/f17 x:0 0 0 2024-04-22T23:45:57.476 INFO:tasks.workunit.client.1.smithi044.stdout:1/46: symlink d1/d8/lc 0 2024-04-22T23:45:57.479 INFO:tasks.workunit.client.1.smithi044.stdout:5/36: unlink f6 0 2024-04-22T23:45:57.479 INFO:tasks.workunit.client.1.smithi044.stdout:4/58: dwrite d5/db/dd/f10 [4194304,4194304] 0 2024-04-22T23:45:57.480 INFO:tasks.workunit.client.1.smithi044.stdout:7/53: creat dd/d11/f13 x:0 0 0 2024-04-22T23:45:57.480 INFO:tasks.workunit.client.1.smithi044.stdout:7/54: readlink l4 0 2024-04-22T23:45:57.482 INFO:tasks.workunit.client.1.smithi044.stdout:3/68: stat d9/cc 0 2024-04-22T23:45:57.485 INFO:tasks.workunit.client.1.smithi044.stdout:1/47: symlink d1/ld 0 2024-04-22T23:45:57.486 INFO:tasks.workunit.client.1.smithi044.stdout:1/48: readlink d1/l2 0 2024-04-22T23:45:57.493 INFO:tasks.workunit.client.1.smithi044.stdout:4/59: mknod d5/db/dd/c15 0 2024-04-22T23:45:57.494 INFO:tasks.workunit.client.1.smithi044.stdout:7/55: mknod dd/d11/c14 0 2024-04-22T23:45:57.494 INFO:tasks.workunit.client.1.smithi044.stdout:5/37: dread f5 [0,4194304] 0 2024-04-22T23:45:57.496 INFO:tasks.workunit.client.1.smithi044.stdout:1/49: symlink d1/le 0 2024-04-22T23:45:57.498 INFO:tasks.workunit.client.1.smithi044.stdout:4/60: chown d5/db/dd/c13 3574494 1 2024-04-22T23:45:57.498 INFO:tasks.workunit.client.1.smithi044.stdout:3/69: creat d9/db/d15/f18 x:0 0 0 2024-04-22T23:45:57.502 INFO:tasks.workunit.client.1.smithi044.stdout:8/52: dwrite f5 [0,4194304] 0 2024-04-22T23:45:57.502 INFO:tasks.workunit.client.1.smithi044.stdout:7/56: rename f2 to dd/d11/f15 0 2024-04-22T23:45:57.503 INFO:tasks.workunit.client.1.smithi044.stdout:7/57: write f7 [240478,112118] 0 2024-04-22T23:45:57.503 INFO:tasks.workunit.client.1.smithi044.stdout:7/58: stat dd/ff 0 2024-04-22T23:45:57.504 INFO:tasks.workunit.client.1.smithi044.stdout:1/50: mknod d1/d8/cf 0 2024-04-22T23:45:57.505 INFO:tasks.workunit.client.1.smithi044.stdout:1/51: truncate d1/f7 4299609 0 2024-04-22T23:45:57.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/70: creat d9/f19 x:0 0 0 2024-04-22T23:45:57.509 INFO:tasks.workunit.client.1.smithi044.stdout:4/61: rmdir d5 39 2024-04-22T23:45:57.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/71: read - d9/f17 zero size 2024-04-22T23:45:57.511 INFO:tasks.workunit.client.1.smithi044.stdout:3/72: write f8 [521586,76774] 0 2024-04-22T23:45:57.514 INFO:tasks.workunit.client.1.smithi044.stdout:8/53: fsync db/f10 0 2024-04-22T23:45:57.515 INFO:tasks.workunit.client.1.smithi044.stdout:7/59: read f5 [286550,50005] 0 2024-04-22T23:45:57.516 INFO:tasks.workunit.client.1.smithi044.stdout:7/60: dread - dd/ff zero size 2024-04-22T23:45:57.518 INFO:tasks.workunit.client.1.smithi044.stdout:1/52: mknod d1/c10 0 2024-04-22T23:45:57.523 INFO:tasks.workunit.client.1.smithi044.stdout:3/73: creat d9/db/f1a x:0 0 0 2024-04-22T23:45:57.525 INFO:tasks.workunit.client.1.smithi044.stdout:8/54: rmdir db/d14 39 2024-04-22T23:45:57.528 INFO:tasks.workunit.client.1.smithi044.stdout:1/53: mknod d1/d8/c11 0 2024-04-22T23:45:57.529 INFO:tasks.workunit.client.1.smithi044.stdout:5/38: getdents . 0 2024-04-22T23:45:57.530 INFO:tasks.workunit.client.1.smithi044.stdout:4/62: mkdir d5/db/d16 0 2024-04-22T23:45:57.530 INFO:tasks.workunit.client.1.smithi044.stdout:4/63: readlink d5/db/l12 0 2024-04-22T23:45:57.532 INFO:tasks.workunit.client.1.smithi044.stdout:8/55: unlink la 0 2024-04-22T23:45:57.533 INFO:tasks.workunit.client.1.smithi044.stdout:1/54: write d1/f7 [890658,43844] 0 2024-04-22T23:45:57.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/55: write d1/f7 [2131966,50991] 0 2024-04-22T23:45:57.535 INFO:tasks.workunit.client.1.smithi044.stdout:4/64: creat d5/db/dd/f17 x:0 0 0 2024-04-22T23:45:57.536 INFO:tasks.workunit.client.1.smithi044.stdout:1/56: chown d1/c10 50450 1 2024-04-22T23:45:57.536 INFO:tasks.workunit.client.1.smithi044.stdout:4/65: read d5/db/dd/f10 [907553,99247] 0 2024-04-22T23:45:57.537 INFO:tasks.workunit.client.1.smithi044.stdout:1/57: read d1/f7 [1508243,101208] 0 2024-04-22T23:45:57.539 INFO:tasks.workunit.client.1.smithi044.stdout:5/39: getdents . 0 2024-04-22T23:45:57.543 INFO:tasks.workunit.client.1.smithi044.stdout:8/56: mknod db/c16 0 2024-04-22T23:45:57.545 INFO:tasks.workunit.client.1.smithi044.stdout:4/66: dwrite d5/db/dd/f10 [8388608,4194304] 0 2024-04-22T23:45:57.548 INFO:tasks.workunit.client.1.smithi044.stdout:1/58: creat d1/d8/f12 x:0 0 0 2024-04-22T23:45:57.549 INFO:tasks.workunit.client.1.smithi044.stdout:7/61: getdents dd/d11 0 2024-04-22T23:45:57.553 INFO:tasks.workunit.client.1.smithi044.stdout:8/57: truncate db/f10 97827 0 2024-04-22T23:45:57.558 INFO:tasks.workunit.client.1.smithi044.stdout:7/62: dwrite dd/ff [0,4194304] 0 2024-04-22T23:45:57.567 INFO:tasks.workunit.client.1.smithi044.stdout:7/63: symlink dd/l16 0 2024-04-22T23:45:57.569 INFO:tasks.workunit.client.1.smithi044.stdout:1/59: fsync d1/d8/f12 0 2024-04-22T23:45:57.570 INFO:tasks.workunit.client.1.smithi044.stdout:1/60: fsync d1/d8/f12 0 2024-04-22T23:45:57.573 INFO:tasks.workunit.client.1.smithi044.stdout:7/64: fdatasync fa 0 2024-04-22T23:45:57.574 INFO:tasks.workunit.client.1.smithi044.stdout:1/61: truncate d1/f7 1352605 0 2024-04-22T23:45:57.575 INFO:tasks.workunit.client.1.smithi044.stdout:1/62: stat d1/d8/c11 0 2024-04-22T23:45:57.575 INFO:tasks.workunit.client.1.smithi044.stdout:4/67: truncate d5/db/dd/f10 12401695 0 2024-04-22T23:45:57.577 INFO:tasks.workunit.client.1.smithi044.stdout:7/65: creat dd/d11/f17 x:0 0 0 2024-04-22T23:45:57.622 INFO:tasks.workunit.client.1.smithi044.stdout:2/33: sync 2024-04-22T23:45:57.623 INFO:tasks.workunit.client.1.smithi044.stdout:1/63: symlink d1/d8/l13 0 2024-04-22T23:45:57.625 INFO:tasks.workunit.client.1.smithi044.stdout:7/66: mknod dd/c18 0 2024-04-22T23:45:57.626 INFO:tasks.workunit.client.1.smithi044.stdout:4/68: mkdir d5/db/d16/d18 0 2024-04-22T23:45:57.628 INFO:tasks.workunit.client.1.smithi044.stdout:1/64: mkdir d1/d14 0 2024-04-22T23:45:57.628 INFO:tasks.workunit.client.1.smithi044.stdout:4/69: write d5/db/dd/f11 [1237394,65942] 0 2024-04-22T23:45:57.629 INFO:tasks.workunit.client.1.smithi044.stdout:7/67: creat dd/d11/f19 x:0 0 0 2024-04-22T23:45:57.632 INFO:tasks.workunit.client.1.smithi044.stdout:8/58: dwrite db/fe [0,4194304] 0 2024-04-22T23:45:57.634 INFO:tasks.workunit.client.1.smithi044.stdout:1/65: symlink d1/d8/l15 0 2024-04-22T23:45:57.635 INFO:tasks.workunit.client.1.smithi044.stdout:1/66: chown d1/d8/c9 316922274 1 2024-04-22T23:45:57.638 INFO:tasks.workunit.client.1.smithi044.stdout:8/59: dread db/f11 [0,4194304] 0 2024-04-22T23:45:57.639 INFO:tasks.workunit.client.1.smithi044.stdout:8/60: chown db/fc 6 1 2024-04-22T23:45:57.641 INFO:tasks.workunit.client.1.smithi044.stdout:7/68: fsync dd/d11/f17 0 2024-04-22T23:45:57.647 INFO:tasks.workunit.client.1.smithi044.stdout:4/70: dread d5/db/dd/f11 [0,4194304] 0 2024-04-22T23:45:57.648 INFO:tasks.workunit.client.1.smithi044.stdout:4/71: chown d5/db/d16 36329 1 2024-04-22T23:45:57.653 INFO:tasks.workunit.client.1.smithi044.stdout:7/69: mkdir dd/d1a 0 2024-04-22T23:45:57.656 INFO:tasks.workunit.client.1.smithi044.stdout:1/67: symlink d1/d14/l16 0 2024-04-22T23:45:57.658 INFO:tasks.workunit.client.1.smithi044.stdout:4/72: creat d5/db/f19 x:0 0 0 2024-04-22T23:45:57.663 INFO:tasks.workunit.client.1.smithi044.stdout:1/68: mknod d1/d14/c17 0 2024-04-22T23:45:57.663 INFO:tasks.workunit.client.1.smithi044.stdout:1/69: dread - d1/d8/f12 zero size 2024-04-22T23:45:57.664 INFO:tasks.workunit.client.1.smithi044.stdout:1/70: write d1/d8/f12 [970403,31067] 0 2024-04-22T23:45:57.665 INFO:tasks.workunit.client.1.smithi044.stdout:1/71: chown d1/d8/l13 3752538 1 2024-04-22T23:45:57.665 INFO:tasks.workunit.client.1.smithi044.stdout:1/72: stat d1/c5 0 2024-04-22T23:45:57.667 INFO:tasks.workunit.client.1.smithi044.stdout:8/61: link db/f12 db/d14/f17 0 2024-04-22T23:45:57.669 INFO:tasks.workunit.client.1.smithi044.stdout:4/73: mknod d5/db/d16/d18/c1a 0 2024-04-22T23:45:57.670 INFO:tasks.workunit.client.1.smithi044.stdout:1/73: creat d1/d8/f18 x:0 0 0 2024-04-22T23:45:57.671 INFO:tasks.workunit.client.1.smithi044.stdout:1/74: truncate d1/d8/f18 992507 0 2024-04-22T23:45:57.673 INFO:tasks.workunit.client.1.smithi044.stdout:4/74: creat d5/db/dd/f1b x:0 0 0 2024-04-22T23:45:57.674 INFO:tasks.workunit.client.1.smithi044.stdout:4/75: write d5/db/dd/f17 [124770,72635] 0 2024-04-22T23:45:57.676 INFO:tasks.workunit.client.1.smithi044.stdout:7/70: write f5 [125415,44314] 0 2024-04-22T23:45:57.677 INFO:tasks.workunit.client.1.smithi044.stdout:7/71: fdatasync dd/d11/f13 0 2024-04-22T23:45:57.678 INFO:tasks.workunit.client.1.smithi044.stdout:4/76: creat d5/db/d16/f1c x:0 0 0 2024-04-22T23:45:57.686 INFO:tasks.workunit.client.1.smithi044.stdout:7/72: symlink dd/d1a/l1b 0 2024-04-22T23:45:57.686 INFO:tasks.workunit.client.1.smithi044.stdout:8/62: write f5 [993075,26656] 0 2024-04-22T23:45:57.688 INFO:tasks.workunit.client.1.smithi044.stdout:4/77: dwrite d5/db/f19 [0,4194304] 0 2024-04-22T23:45:57.689 INFO:tasks.workunit.client.1.smithi044.stdout:8/63: fdatasync db/f11 0 2024-04-22T23:45:57.690 INFO:tasks.workunit.client.1.smithi044.stdout:7/73: creat dd/d11/f1c x:0 0 0 2024-04-22T23:45:57.693 INFO:tasks.workunit.client.1.smithi044.stdout:7/74: dread f7 [0,4194304] 0 2024-04-22T23:45:57.695 INFO:tasks.workunit.client.1.smithi044.stdout:4/78: rename d5/f6 to d5/db/dd/f1d 0 2024-04-22T23:45:57.697 INFO:tasks.workunit.client.1.smithi044.stdout:8/64: symlink db/d14/l18 0 2024-04-22T23:45:57.700 INFO:tasks.workunit.client.1.smithi044.stdout:8/65: read - db/d14/f15 zero size 2024-04-22T23:45:57.702 INFO:tasks.workunit.client.1.smithi044.stdout:1/75: dwrite d1/f7 [0,4194304] 0 2024-04-22T23:45:57.705 INFO:tasks.workunit.client.1.smithi044.stdout:4/79: mknod d5/db/dd/c1e 0 2024-04-22T23:45:57.706 INFO:tasks.workunit.client.1.smithi044.stdout:7/75: dwrite fc [0,4194304] 0 2024-04-22T23:45:57.714 INFO:tasks.workunit.client.1.smithi044.stdout:4/80: creat d5/db/d16/d18/f1f x:0 0 0 2024-04-22T23:45:57.717 INFO:tasks.workunit.client.1.smithi044.stdout:7/76: creat dd/d1a/f1d x:0 0 0 2024-04-22T23:45:57.721 INFO:tasks.workunit.client.1.smithi044.stdout:8/66: dwrite db/ff [0,4194304] 0 2024-04-22T23:45:57.723 INFO:tasks.workunit.client.1.smithi044.stdout:4/81: mkdir d5/db/d16/d18/d20 0 2024-04-22T23:45:57.723 INFO:tasks.workunit.client.1.smithi044.stdout:1/76: dwrite d1/d8/f18 [0,4194304] 0 2024-04-22T23:45:57.725 INFO:tasks.workunit.client.1.smithi044.stdout:1/77: readlink d1/d8/lc 0 2024-04-22T23:45:57.725 INFO:tasks.workunit.client.1.smithi044.stdout:1/78: readlink d1/l2 0 2024-04-22T23:45:57.726 INFO:tasks.workunit.client.1.smithi044.stdout:4/82: truncate d5/db/d16/d18/f1f 859497 0 2024-04-22T23:45:57.726 INFO:tasks.workunit.client.1.smithi044.stdout:7/77: creat dd/d11/f1e x:0 0 0 2024-04-22T23:45:57.728 INFO:tasks.workunit.client.1.smithi044.stdout:1/79: creat d1/d8/f19 x:0 0 0 2024-04-22T23:45:57.729 INFO:tasks.workunit.client.1.smithi044.stdout:1/80: chown d1/d8/f18 215831 1 2024-04-22T23:45:57.733 INFO:tasks.workunit.client.1.smithi044.stdout:8/67: creat db/f19 x:0 0 0 2024-04-22T23:45:57.741 INFO:tasks.workunit.client.1.smithi044.stdout:1/81: dwrite d1/d8/f12 [0,4194304] 0 2024-04-22T23:45:57.742 INFO:tasks.workunit.client.1.smithi044.stdout:8/68: dread f6 [0,4194304] 0 2024-04-22T23:45:57.742 INFO:tasks.workunit.client.1.smithi044.stdout:1/82: write d1/d8/f18 [3996316,38711] 0 2024-04-22T23:45:57.745 INFO:tasks.workunit.client.1.smithi044.stdout:6/53: sync 2024-04-22T23:45:57.745 INFO:tasks.workunit.client.1.smithi044.stdout:0/37: sync 2024-04-22T23:45:57.746 INFO:tasks.workunit.client.1.smithi044.stdout:9/78: sync 2024-04-22T23:45:57.746 INFO:tasks.workunit.client.1.smithi044.stdout:6/54: stat d5/dd/cf 0 2024-04-22T23:45:57.747 INFO:tasks.workunit.client.1.smithi044.stdout:1/83: creat d1/d8/f1a x:0 0 0 2024-04-22T23:45:57.747 INFO:tasks.workunit.client.1.smithi044.stdout:7/78: rmdir dd/d11 39 2024-04-22T23:45:57.749 INFO:tasks.workunit.client.1.smithi044.stdout:1/84: stat d1/d14/c17 0 2024-04-22T23:45:57.754 INFO:tasks.workunit.client.1.smithi044.stdout:4/83: truncate d5/db/dd/f10 7448172 0 2024-04-22T23:45:57.758 INFO:tasks.workunit.client.1.smithi044.stdout:4/84: dread d5/db/f19 [0,4194304] 0 2024-04-22T23:45:57.759 INFO:tasks.workunit.client.1.smithi044.stdout:8/69: rename db/fc to db/f1a 0 2024-04-22T23:45:57.761 INFO:tasks.workunit.client.1.smithi044.stdout:4/85: dread d5/db/dd/f11 [0,4194304] 0 2024-04-22T23:45:57.761 INFO:tasks.workunit.client.1.smithi044.stdout:1/85: creat d1/f1b x:0 0 0 2024-04-22T23:45:57.762 INFO:tasks.workunit.client.1.smithi044.stdout:1/86: chown d1/l2 124 1 2024-04-22T23:45:57.763 INFO:tasks.workunit.client.1.smithi044.stdout:1/87: fdatasync d1/f1b 0 2024-04-22T23:45:57.769 INFO:tasks.workunit.client.1.smithi044.stdout:6/55: dwrite d5/f6 [0,4194304] 0 2024-04-22T23:45:57.775 INFO:tasks.workunit.client.1.smithi044.stdout:0/38: link d0/d2/c7 d0/d2/cb 0 2024-04-22T23:45:57.778 INFO:tasks.workunit.client.1.smithi044.stdout:1/88: mkdir d1/d8/d1c 0 2024-04-22T23:45:57.778 INFO:tasks.workunit.client.1.smithi044.stdout:1/89: stat d1/ld 0 2024-04-22T23:45:57.782 INFO:tasks.workunit.client.1.smithi044.stdout:6/56: truncate f0 28356 0 2024-04-22T23:45:57.783 INFO:tasks.workunit.client.1.smithi044.stdout:9/79: getdents d2/d9 0 2024-04-22T23:45:57.784 INFO:tasks.workunit.client.1.smithi044.stdout:9/80: chown d2/d5 7 1 2024-04-22T23:45:57.786 INFO:tasks.workunit.client.1.smithi044.stdout:0/39: rename d0/d2/cb to d0/d2/cc 0 2024-04-22T23:45:57.790 INFO:tasks.workunit.client.1.smithi044.stdout:7/79: getdents dd/d1a 0 2024-04-22T23:45:57.790 INFO:tasks.workunit.client.1.smithi044.stdout:8/70: link c3 db/d14/c1b 0 2024-04-22T23:45:57.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/57: write d5/dd/fe [844446,27053] 0 2024-04-22T23:45:57.791 INFO:tasks.workunit.client.1.smithi044.stdout:9/81: rmdir d2/dc 39 2024-04-22T23:45:57.792 INFO:tasks.workunit.client.1.smithi044.stdout:0/40: mkdir d0/d2/dd 0 2024-04-22T23:45:57.793 INFO:tasks.workunit.client.1.smithi044.stdout:0/41: write d0/d2/fa [735985,40001] 0 2024-04-22T23:45:57.794 INFO:tasks.workunit.client.1.smithi044.stdout:6/58: dread d5/dd/f10 [0,4194304] 0 2024-04-22T23:45:57.795 INFO:tasks.workunit.client.1.smithi044.stdout:6/59: chown d5/dd/cf 672043139 1 2024-04-22T23:45:57.801 INFO:tasks.workunit.client.1.smithi044.stdout:8/71: chown c3 800812 1 2024-04-22T23:45:57.804 INFO:tasks.workunit.client.1.smithi044.stdout:1/90: truncate d1/f7 2932307 0 2024-04-22T23:45:57.807 INFO:tasks.workunit.client.1.smithi044.stdout:0/42: mknod d0/ce 0 2024-04-22T23:45:57.808 INFO:tasks.workunit.client.1.smithi044.stdout:8/72: dwrite f0 [0,4194304] 0 2024-04-22T23:45:57.810 INFO:tasks.workunit.client.1.smithi044.stdout:0/43: truncate d0/d2/fa 813470 0 2024-04-22T23:45:57.811 INFO:tasks.workunit.client.1.smithi044.stdout:1/91: dread d1/d8/f18 [0,4194304] 0 2024-04-22T23:45:57.812 INFO:tasks.workunit.client.1.smithi044.stdout:7/80: mkdir dd/d1f 0 2024-04-22T23:45:57.819 INFO:tasks.workunit.client.1.smithi044.stdout:9/82: mkdir d2/dc/d1d 0 2024-04-22T23:45:57.823 INFO:tasks.workunit.client.1.smithi044.stdout:7/81: write f7 [995218,10917] 0 2024-04-22T23:45:57.826 INFO:tasks.workunit.client.1.smithi044.stdout:1/92: mknod d1/c1d 0 2024-04-22T23:45:57.832 INFO:tasks.workunit.client.1.smithi044.stdout:0/44: rename d0/d2/c7 to d0/d2/cf 0 2024-04-22T23:45:57.836 INFO:tasks.workunit.client.1.smithi044.stdout:1/93: creat d1/d8/f1e x:0 0 0 2024-04-22T23:45:57.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/73: creat db/f1c x:0 0 0 2024-04-22T23:45:57.838 INFO:tasks.workunit.client.1.smithi044.stdout:7/82: write dd/d11/f17 [194,32986] 0 2024-04-22T23:45:57.839 INFO:tasks.workunit.client.1.smithi044.stdout:7/83: readlink l0 0 2024-04-22T23:45:57.839 INFO:tasks.workunit.client.1.smithi044.stdout:6/60: dread f1 [0,4194304] 0 2024-04-22T23:45:57.841 INFO:tasks.workunit.client.1.smithi044.stdout:9/83: link d2/dc/f18 d2/d5/f1e 0 2024-04-22T23:45:57.841 INFO:tasks.workunit.client.1.smithi044.stdout:0/45: dwrite d0/d2/fa [0,4194304] 0 2024-04-22T23:45:57.851 INFO:tasks.workunit.client.1.smithi044.stdout:0/46: chown d0/d2/cc 24 1 2024-04-22T23:45:57.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/61: link d5/f6 d5/d8/f13 0 2024-04-22T23:45:57.856 INFO:tasks.workunit.client.1.smithi044.stdout:8/74: creat db/f1d x:0 0 0 2024-04-22T23:45:57.857 INFO:tasks.workunit.client.1.smithi044.stdout:6/62: truncate d5/dd/fe 1880630 0 2024-04-22T23:45:57.858 INFO:tasks.workunit.client.1.smithi044.stdout:7/84: creat dd/f20 x:0 0 0 2024-04-22T23:45:57.859 INFO:tasks.workunit.client.1.smithi044.stdout:0/47: creat d0/d2/dd/f10 x:0 0 0 2024-04-22T23:45:57.862 INFO:tasks.workunit.client.1.smithi044.stdout:6/63: mknod d5/d8/c14 0 2024-04-22T23:45:57.867 INFO:tasks.workunit.client.1.smithi044.stdout:7/85: creat dd/d1f/f21 x:0 0 0 2024-04-22T23:45:57.868 INFO:tasks.workunit.client.1.smithi044.stdout:7/86: chown dd/d1f/f21 8162 1 2024-04-22T23:45:57.868 INFO:tasks.workunit.client.1.smithi044.stdout:8/75: rmdir db 39 2024-04-22T23:45:57.868 INFO:tasks.workunit.client.1.smithi044.stdout:7/87: stat l4 0 2024-04-22T23:45:57.871 INFO:tasks.workunit.client.1.smithi044.stdout:0/48: write d0/d2/f5 [2023864,22132] 0 2024-04-22T23:45:57.871 INFO:tasks.workunit.client.1.smithi044.stdout:6/64: write f0 [743137,1608] 0 2024-04-22T23:45:57.873 INFO:tasks.workunit.client.1.smithi044.stdout:8/76: write db/fd [55896,74268] 0 2024-04-22T23:45:57.878 INFO:tasks.workunit.client.1.smithi044.stdout:6/65: creat d5/d8/f15 x:0 0 0 2024-04-22T23:45:57.879 INFO:tasks.workunit.client.1.smithi044.stdout:6/66: truncate d5/d8/fa 863474 0 2024-04-22T23:45:57.879 INFO:tasks.workunit.client.1.smithi044.stdout:8/77: chown c2 6 1 2024-04-22T23:45:57.880 INFO:tasks.workunit.client.1.smithi044.stdout:7/88: link c6 dd/d1a/c22 0 2024-04-22T23:45:57.882 INFO:tasks.workunit.client.1.smithi044.stdout:6/67: creat d5/dd/f16 x:0 0 0 2024-04-22T23:45:57.883 INFO:tasks.workunit.client.1.smithi044.stdout:7/89: dread f7 [0,4194304] 0 2024-04-22T23:45:57.884 INFO:tasks.workunit.client.1.smithi044.stdout:7/90: chown f7 199 1 2024-04-22T23:45:57.886 INFO:tasks.workunit.client.1.smithi044.stdout:7/91: chown dd/d11/f15 508 1 2024-04-22T23:45:57.889 INFO:tasks.workunit.client.1.smithi044.stdout:8/78: mknod db/c1e 0 2024-04-22T23:45:57.891 INFO:tasks.workunit.client.1.smithi044.stdout:6/68: dwrite d5/d8/fa [0,4194304] 0 2024-04-22T23:45:57.896 INFO:tasks.workunit.client.1.smithi044.stdout:0/49: getdents d0/d2 0 2024-04-22T23:45:57.898 INFO:tasks.workunit.client.1.smithi044.stdout:7/92: dwrite dd/f20 [0,4194304] 0 2024-04-22T23:45:57.904 INFO:tasks.workunit.client.1.smithi044.stdout:7/93: symlink dd/d11/l23 0 2024-04-22T23:45:57.905 INFO:tasks.workunit.client.1.smithi044.stdout:6/69: dwrite d5/dd/fe [0,4194304] 0 2024-04-22T23:45:57.906 INFO:tasks.workunit.client.1.smithi044.stdout:6/70: stat d5/dd/fe 0 2024-04-22T23:45:57.907 INFO:tasks.workunit.client.1.smithi044.stdout:8/79: symlink db/l1f 0 2024-04-22T23:45:57.917 INFO:tasks.workunit.client.1.smithi044.stdout:7/94: dwrite f8 [0,4194304] 0 2024-04-22T23:45:57.920 INFO:tasks.workunit.client.1.smithi044.stdout:6/71: mknod d5/dd/c17 0 2024-04-22T23:45:57.924 INFO:tasks.workunit.client.1.smithi044.stdout:0/50: link d0/d2/dd/f10 d0/d2/dd/f11 0 2024-04-22T23:45:57.929 INFO:tasks.workunit.client.1.smithi044.stdout:6/72: mkdir d5/dd/d18 0 2024-04-22T23:45:57.930 INFO:tasks.workunit.client.1.smithi044.stdout:6/73: write f1 [171641,45005] 0 2024-04-22T23:45:57.930 INFO:tasks.workunit.client.1.smithi044.stdout:6/74: readlink d5/d8/lb 0 2024-04-22T23:45:57.933 INFO:tasks.workunit.client.1.smithi044.stdout:6/75: truncate d5/dd/f10 1675043 0 2024-04-22T23:45:57.934 INFO:tasks.workunit.client.1.smithi044.stdout:6/76: chown d5/dd/c17 1990 1 2024-04-22T23:45:57.935 INFO:tasks.workunit.client.1.smithi044.stdout:7/95: rename dd/d11 to dd/d1f/d24 0 2024-04-22T23:45:57.936 INFO:tasks.workunit.client.1.smithi044.stdout:7/96: write fc [2444565,101023] 0 2024-04-22T23:45:57.938 INFO:tasks.workunit.client.1.smithi044.stdout:7/97: write dd/d1f/d24/f17 [192549,10690] 0 2024-04-22T23:45:57.939 INFO:tasks.workunit.client.1.smithi044.stdout:6/77: dread d5/d8/fa [0,4194304] 0 2024-04-22T23:45:57.940 INFO:tasks.workunit.client.1.smithi044.stdout:6/78: chown d5/dd/cf 16530306 1 2024-04-22T23:45:57.941 INFO:tasks.workunit.client.1.smithi044.stdout:6/79: truncate d5/d8/f15 765954 0 2024-04-22T23:45:57.943 INFO:tasks.workunit.client.1.smithi044.stdout:0/51: rename d0/d2/f5 to d0/d2/dd/f12 0 2024-04-22T23:45:57.947 INFO:tasks.workunit.client.1.smithi044.stdout:0/52: dread d0/d2/dd/f12 [0,4194304] 0 2024-04-22T23:45:57.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/80: mknod d5/c19 0 2024-04-22T23:45:57.954 INFO:tasks.workunit.client.1.smithi044.stdout:8/80: dwrite db/d14/f17 [0,4194304] 0 2024-04-22T23:45:57.960 INFO:tasks.workunit.client.1.smithi044.stdout:0/53: creat d0/d2/dd/f13 x:0 0 0 2024-04-22T23:45:57.964 INFO:tasks.workunit.client.1.smithi044.stdout:6/81: symlink d5/dd/l1a 0 2024-04-22T23:45:57.968 INFO:tasks.workunit.client.1.smithi044.stdout:3/74: sync 2024-04-22T23:45:57.969 INFO:tasks.workunit.client.1.smithi044.stdout:8/81: fsync db/f1a 0 2024-04-22T23:45:57.973 INFO:tasks.workunit.client.1.smithi044.stdout:0/54: mknod d0/c14 0 2024-04-22T23:45:57.979 INFO:tasks.workunit.client.1.smithi044.stdout:6/82: creat d5/dd/f1b x:0 0 0 2024-04-22T23:45:57.979 INFO:tasks.workunit.client.1.smithi044.stdout:6/83: chown d5/dd/fe 997 1 2024-04-22T23:45:57.981 INFO:tasks.workunit.client.1.smithi044.stdout:6/84: dread d5/d8/f15 [0,4194304] 0 2024-04-22T23:45:57.984 INFO:tasks.workunit.client.1.smithi044.stdout:6/85: dread d5/d8/f13 [0,4194304] 0 2024-04-22T23:45:58.003 INFO:tasks.workunit.client.1.smithi044.stdout:0/55: write d0/d2/dd/f12 [2678516,2067] 0 2024-04-22T23:45:58.007 INFO:tasks.workunit.client.1.smithi044.stdout:8/82: rename c8 to db/c20 0 2024-04-22T23:45:58.079 INFO:tasks.workunit.client.1.smithi044.stdout:3/75: write d9/db/f10 [4622760,35550] 0 2024-04-22T23:45:58.080 INFO:tasks.workunit.client.1.smithi044.stdout:3/76: write d9/f19 [642734,60410] 0 2024-04-22T23:45:58.082 INFO:tasks.workunit.client.1.smithi044.stdout:0/56: creat d0/d2/f15 x:0 0 0 2024-04-22T23:45:58.087 INFO:tasks.workunit.client.1.smithi044.stdout:3/77: creat d9/db/d15/f1b x:0 0 0 2024-04-22T23:45:58.088 INFO:tasks.workunit.client.1.smithi044.stdout:3/78: rename d9 to d9/db/d1c 22 2024-04-22T23:45:58.088 INFO:tasks.workunit.client.1.smithi044.stdout:3/79: write d9/db/f10 [4536987,90388] 0 2024-04-22T23:45:58.089 INFO:tasks.workunit.client.1.smithi044.stdout:0/57: creat d0/f16 x:0 0 0 2024-04-22T23:45:58.090 INFO:tasks.workunit.client.1.smithi044.stdout:0/58: fsync d0/d2/fa 0 2024-04-22T23:45:58.092 INFO:tasks.workunit.client.1.smithi044.stdout:0/59: write d0/d2/dd/f12 [3695685,92659] 0 2024-04-22T23:45:58.095 INFO:tasks.workunit.client.1.smithi044.stdout:3/80: mknod d9/c1d 0 2024-04-22T23:45:58.102 INFO:tasks.workunit.client.1.smithi044.stdout:2/34: sync 2024-04-22T23:45:58.102 INFO:tasks.workunit.client.1.smithi044.stdout:5/40: sync 2024-04-22T23:45:58.102 INFO:tasks.workunit.client.1.smithi044.stdout:2/35: chown d8 11 1 2024-04-22T23:45:58.102 INFO:tasks.workunit.client.1.smithi044.stdout:2/36: stat f6 0 2024-04-22T23:45:58.103 INFO:tasks.workunit.client.1.smithi044.stdout:2/37: write f4 [989668,116550] 0 2024-04-22T23:45:58.104 INFO:tasks.workunit.client.1.smithi044.stdout:2/38: dread - f6 zero size 2024-04-22T23:45:58.104 INFO:tasks.workunit.client.1.smithi044.stdout:8/83: creat db/f21 x:0 0 0 2024-04-22T23:45:58.105 INFO:tasks.workunit.client.1.smithi044.stdout:8/84: chown db/d14/c1b 2 1 2024-04-22T23:45:58.107 INFO:tasks.workunit.client.1.smithi044.stdout:0/60: creat d0/f17 x:0 0 0 2024-04-22T23:45:58.111 INFO:tasks.workunit.client.1.smithi044.stdout:5/41: truncate f5 2135837 0 2024-04-22T23:45:58.113 INFO:tasks.workunit.client.1.smithi044.stdout:8/85: dwrite db/f1c [0,4194304] 0 2024-04-22T23:45:58.115 INFO:tasks.workunit.client.1.smithi044.stdout:2/39: creat d8/fa x:0 0 0 2024-04-22T23:45:58.126 INFO:tasks.workunit.client.1.smithi044.stdout:2/40: dwrite d8/fa [0,4194304] 0 2024-04-22T23:45:58.126 INFO:tasks.workunit.client.1.smithi044.stdout:3/81: rename d9/db/c11 to d9/d14/c1e 0 2024-04-22T23:45:58.128 INFO:tasks.workunit.client.1.smithi044.stdout:2/41: rename d8 to d8/db 22 2024-04-22T23:45:58.129 INFO:tasks.workunit.client.1.smithi044.stdout:5/42: creat fc x:0 0 0 2024-04-22T23:45:58.130 INFO:tasks.workunit.client.1.smithi044.stdout:3/82: write d9/db/d15/f1b [336298,100106] 0 2024-04-22T23:45:58.130 INFO:tasks.workunit.client.1.smithi044.stdout:2/42: write f6 [703846,53908] 0 2024-04-22T23:45:58.131 INFO:tasks.workunit.client.1.smithi044.stdout:6/86: dwrite d5/dd/f10 [0,4194304] 0 2024-04-22T23:45:58.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/43: dread f2 [0,4194304] 0 2024-04-22T23:45:58.151 INFO:tasks.workunit.client.1.smithi044.stdout:3/83: truncate f7 869870 0 2024-04-22T23:45:58.152 INFO:tasks.workunit.client.1.smithi044.stdout:2/44: write f4 [191577,24907] 0 2024-04-22T23:45:58.152 INFO:tasks.workunit.client.1.smithi044.stdout:3/84: stat d9/db/d15 0 2024-04-22T23:45:58.153 INFO:tasks.workunit.client.1.smithi044.stdout:3/85: dread - d9/f17 zero size 2024-04-22T23:45:58.160 INFO:tasks.workunit.client.1.smithi044.stdout:2/45: creat d8/fc x:0 0 0 2024-04-22T23:45:58.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/61: truncate d0/d2/dd/f12 1315704 0 2024-04-22T23:45:58.168 INFO:tasks.workunit.client.1.smithi044.stdout:6/87: link d5/d8/lb d5/l1c 0 2024-04-22T23:45:58.170 INFO:tasks.workunit.client.1.smithi044.stdout:3/86: creat d9/d12/f1f x:0 0 0 2024-04-22T23:45:58.171 INFO:tasks.workunit.client.1.smithi044.stdout:0/62: symlink d0/l18 0 2024-04-22T23:45:58.173 INFO:tasks.workunit.client.1.smithi044.stdout:8/86: truncate db/d14/f17 836304 0 2024-04-22T23:45:58.174 INFO:tasks.workunit.client.1.smithi044.stdout:6/88: mkdir d5/dd/d1d 0 2024-04-22T23:45:58.179 INFO:tasks.workunit.client.1.smithi044.stdout:3/87: dwrite d9/db/d15/f18 [0,4194304] 0 2024-04-22T23:45:58.179 INFO:tasks.workunit.client.1.smithi044.stdout:0/63: mkdir d0/d2/dd/d19 0 2024-04-22T23:45:58.186 INFO:tasks.workunit.client.1.smithi044.stdout:5/43: dwrite f5 [0,4194304] 0 2024-04-22T23:45:58.186 INFO:tasks.workunit.client.1.smithi044.stdout:5/44: rmdir - no directory 2024-04-22T23:45:58.194 INFO:tasks.workunit.client.1.smithi044.stdout:5/45: dread f0 [0,4194304] 0 2024-04-22T23:45:58.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:58 smithi031 ceph-mon[26721]: pgmap v35: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 67 MiB/s rd, 151 MiB/s wr, 383 op/s 2024-04-22T23:45:58.195 INFO:tasks.workunit.client.1.smithi044.stdout:8/87: link db/f11 db/f22 0 2024-04-22T23:45:58.198 INFO:tasks.workunit.client.1.smithi044.stdout:8/88: dread db/f10 [0,4194304] 0 2024-04-22T23:45:58.203 INFO:tasks.workunit.client.1.smithi044.stdout:5/46: write f0 [2991360,17868] 0 2024-04-22T23:45:58.210 INFO:tasks.workunit.client.1.smithi044.stdout:8/89: unlink c4 0 2024-04-22T23:45:58.214 INFO:tasks.workunit.client.1.smithi044.stdout:8/90: dread f6 [0,4194304] 0 2024-04-22T23:45:58.215 INFO:tasks.workunit.client.1.smithi044.stdout:6/89: write d5/d8/f15 [280342,85337] 0 2024-04-22T23:45:58.216 INFO:tasks.workunit.client.1.smithi044.stdout:6/90: chown d5/dd/f10 3289180 1 2024-04-22T23:45:58.217 INFO:tasks.workunit.client.1.smithi044.stdout:6/91: fsync d5/dd/f1b 0 2024-04-22T23:45:58.218 INFO:tasks.workunit.client.1.smithi044.stdout:0/64: getdents d0/d2/dd 0 2024-04-22T23:45:58.222 INFO:tasks.workunit.client.1.smithi044.stdout:5/47: unlink cb 0 2024-04-22T23:45:58.228 INFO:tasks.workunit.client.1.smithi044.stdout:6/92: fsync d5/f6 0 2024-04-22T23:45:58.228 INFO:tasks.workunit.client.1.smithi044.stdout:4/86: sync 2024-04-22T23:45:58.230 INFO:tasks.workunit.client.1.smithi044.stdout:3/88: truncate d9/db/d15/f1b 64075 0 2024-04-22T23:45:58.231 INFO:tasks.workunit.client.1.smithi044.stdout:6/93: dread f0 [0,4194304] 0 2024-04-22T23:45:58.232 INFO:tasks.workunit.client.1.smithi044.stdout:8/91: symlink db/l23 0 2024-04-22T23:45:58.235 INFO:tasks.workunit.client.1.smithi044.stdout:3/89: mknod d9/c20 0 2024-04-22T23:45:58.236 INFO:tasks.workunit.client.1.smithi044.stdout:0/65: link d0/f16 d0/d2/dd/f1a 0 2024-04-22T23:45:58.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/94: creat d5/dd/f1e x:0 0 0 2024-04-22T23:45:58.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/95: chown c2 14919711 1 2024-04-22T23:45:58.243 INFO:tasks.workunit.client.1.smithi044.stdout:6/96: read d5/d8/f13 [1857339,54948] 0 2024-04-22T23:45:58.243 INFO:tasks.workunit.client.1.smithi044.stdout:6/97: fdatasync d5/dd/f10 0 2024-04-22T23:45:58.261 INFO:tasks.workunit.client.1.smithi044.stdout:8/92: mkdir db/d14/d24 0 2024-04-22T23:45:58.262 INFO:tasks.workunit.client.1.smithi044.stdout:3/90: creat d9/db/d15/f21 x:0 0 0 2024-04-22T23:45:58.262 INFO:tasks.workunit.client.1.smithi044.stdout:5/48: truncate f0 301197 0 2024-04-22T23:45:58.270 INFO:tasks.workunit.client.1.smithi044.stdout:3/91: dwrite d9/db/d15/f18 [0,4194304] 0 2024-04-22T23:45:58.274 INFO:tasks.workunit.client.1.smithi044.stdout:0/66: link d0/d2/f15 d0/d2/dd/f1b 0 2024-04-22T23:45:58.275 INFO:tasks.workunit.client.1.smithi044.stdout:0/67: fsync d0/d2/fa 0 2024-04-22T23:45:58.275 INFO:tasks.workunit.client.1.smithi044.stdout:8/93: creat db/d14/f25 x:0 0 0 2024-04-22T23:45:58.278 INFO:tasks.workunit.client.1.smithi044.stdout:8/94: write db/f21 [599427,47884] 0 2024-04-22T23:45:58.279 INFO:tasks.workunit.client.1.smithi044.stdout:3/92: dwrite d9/db/d15/f18 [4194304,4194304] 0 2024-04-22T23:45:58.280 INFO:tasks.workunit.client.1.smithi044.stdout:8/95: stat db/d14/c1b 0 2024-04-22T23:45:58.282 INFO:tasks.workunit.client.1.smithi044.stdout:5/49: truncate f5 4643375 0 2024-04-22T23:45:58.286 INFO:tasks.workunit.client.1.smithi044.stdout:6/98: symlink d5/dd/d18/l1f 0 2024-04-22T23:45:58.286 INFO:tasks.workunit.client.1.smithi044.stdout:8/96: dread f5 [0,4194304] 0 2024-04-22T23:45:58.289 INFO:tasks.workunit.client.1.smithi044.stdout:3/93: mkdir d9/db/d22 0 2024-04-22T23:45:58.290 INFO:tasks.workunit.client.1.smithi044.stdout:3/94: write d9/d12/f1f [100344,29230] 0 2024-04-22T23:45:58.291 INFO:tasks.workunit.client.1.smithi044.stdout:3/95: chown d9/d14/l16 652443914 1 2024-04-22T23:45:58.294 INFO:tasks.workunit.client.1.smithi044.stdout:6/99: dread f1 [0,4194304] 0 2024-04-22T23:45:58.295 INFO:tasks.workunit.client.1.smithi044.stdout:6/100: chown f1 19 1 2024-04-22T23:45:58.297 INFO:tasks.workunit.client.1.smithi044.stdout:8/97: creat db/d14/d24/f26 x:0 0 0 2024-04-22T23:45:58.299 INFO:tasks.workunit.client.1.smithi044.stdout:0/68: link d0/l9 d0/l1c 0 2024-04-22T23:45:58.301 INFO:tasks.workunit.client.1.smithi044.stdout:5/50: dwrite fc [0,4194304] 0 2024-04-22T23:45:58.305 INFO:tasks.workunit.client.1.smithi044.stdout:8/98: fsync db/f10 0 2024-04-22T23:45:58.311 INFO:tasks.workunit.client.1.smithi044.stdout:0/69: readlink d0/l1c 0 2024-04-22T23:45:58.313 INFO:tasks.workunit.client.1.smithi044.stdout:6/101: write d5/f6 [1815988,24576] 0 2024-04-22T23:45:58.315 INFO:tasks.workunit.client.1.smithi044.stdout:5/51: dwrite f5 [4194304,4194304] 0 2024-04-22T23:45:58.316 INFO:tasks.workunit.client.1.smithi044.stdout:8/99: rmdir db/d14 39 2024-04-22T23:45:58.316 INFO:tasks.workunit.client.1.smithi044.stdout:6/102: chown d5/l7 12 1 2024-04-22T23:45:58.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/70: read d0/d2/dd/f12 [35606,120404] 0 2024-04-22T23:45:58.320 INFO:tasks.workunit.client.1.smithi044.stdout:3/96: dwrite f5 [0,4194304] 0 2024-04-22T23:45:58.323 INFO:tasks.workunit.client.1.smithi044.stdout:1/94: sync 2024-04-22T23:45:58.323 INFO:tasks.workunit.client.1.smithi044.stdout:9/84: sync 2024-04-22T23:45:58.323 INFO:tasks.workunit.client.1.smithi044.stdout:2/46: sync 2024-04-22T23:45:58.323 INFO:tasks.workunit.client.1.smithi044.stdout:7/98: sync 2024-04-22T23:45:58.327 INFO:tasks.workunit.client.1.smithi044.stdout:6/103: creat d5/dd/d18/f20 x:0 0 0 2024-04-22T23:45:58.328 INFO:tasks.workunit.client.1.smithi044.stdout:8/100: dread - db/d14/d24/f26 zero size 2024-04-22T23:45:58.331 INFO:tasks.workunit.client.1.smithi044.stdout:8/101: chown db/d14/d24 454714489 1 2024-04-22T23:45:58.333 INFO:tasks.workunit.client.1.smithi044.stdout:5/52: dread f8 [0,4194304] 0 2024-04-22T23:45:58.335 INFO:tasks.workunit.client.1.smithi044.stdout:0/71: fdatasync d0/d2/dd/f10 0 2024-04-22T23:45:58.337 INFO:tasks.workunit.client.1.smithi044.stdout:8/102: truncate db/d14/f25 173797 0 2024-04-22T23:45:58.338 INFO:tasks.workunit.client.1.smithi044.stdout:9/85: creat d2/d5/f1f x:0 0 0 2024-04-22T23:45:58.339 INFO:tasks.workunit.client.1.smithi044.stdout:7/99: rmdir dd/d1f/d24 39 2024-04-22T23:45:58.339 INFO:tasks.workunit.client.1.smithi044.stdout:1/95: creat d1/d14/f1f x:0 0 0 2024-04-22T23:45:58.341 INFO:tasks.workunit.client.1.smithi044.stdout:2/47: dwrite d8/fc [0,4194304] 0 2024-04-22T23:45:58.344 INFO:tasks.workunit.client.1.smithi044.stdout:0/72: truncate d0/d2/dd/f1a 101919 0 2024-04-22T23:45:58.347 INFO:tasks.workunit.client.1.smithi044.stdout:6/104: link d5/dd/d18/f20 d5/f21 0 2024-04-22T23:45:58.348 INFO:tasks.workunit.client.1.smithi044.stdout:8/103: dread f6 [0,4194304] 0 2024-04-22T23:45:58.352 INFO:tasks.workunit.client.1.smithi044.stdout:1/96: mkdir d1/d8/d20 0 2024-04-22T23:45:58.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/86: rename d2/d5/c10 to d2/d5/c20 0 2024-04-22T23:45:58.362 INFO:tasks.workunit.client.1.smithi044.stdout:3/97: dwrite f7 [0,4194304] 0 2024-04-22T23:45:58.362 INFO:tasks.workunit.client.1.smithi044.stdout:3/98: stat d9/f19 0 2024-04-22T23:45:58.363 INFO:tasks.workunit.client.1.smithi044.stdout:3/99: dread - d9/db/f1a zero size 2024-04-22T23:45:58.364 INFO:tasks.workunit.client.1.smithi044.stdout:0/73: creat d0/d2/dd/f1d x:0 0 0 2024-04-22T23:45:58.367 INFO:tasks.workunit.client.1.smithi044.stdout:2/48: write f3 [943589,36001] 0 2024-04-22T23:45:58.368 INFO:tasks.workunit.client.1.smithi044.stdout:2/49: readlink l5 0 2024-04-22T23:45:58.374 INFO:tasks.workunit.client.1.smithi044.stdout:7/100: dwrite f7 [0,4194304] 0 2024-04-22T23:45:58.376 INFO:tasks.workunit.client.1.smithi044.stdout:3/100: creat d9/f23 x:0 0 0 2024-04-22T23:45:58.384 INFO:tasks.workunit.client.1.smithi044.stdout:1/97: mknod d1/d8/d1c/c21 0 2024-04-22T23:45:58.388 INFO:tasks.workunit.client.1.smithi044.stdout:5/53: rename c7 to cd 0 2024-04-22T23:45:58.395 INFO:tasks.workunit.client.1.smithi044.stdout:6/105: dwrite f1 [0,4194304] 0 2024-04-22T23:45:58.403 INFO:tasks.workunit.client.1.smithi044.stdout:7/101: truncate dd/d1f/d24/f19 41838 0 2024-04-22T23:45:58.406 INFO:tasks.workunit.client.1.smithi044.stdout:7/102: read fc [1185581,25940] 0 2024-04-22T23:45:58.406 INFO:tasks.workunit.client.1.smithi044.stdout:9/87: dwrite d2/dc/f18 [0,4194304] 0 2024-04-22T23:45:58.408 INFO:tasks.workunit.client.1.smithi044.stdout:7/103: write dd/d1a/f1d [877707,91522] 0 2024-04-22T23:45:58.410 INFO:tasks.workunit.client.1.smithi044.stdout:7/104: write f8 [2159939,71323] 0 2024-04-22T23:45:58.411 INFO:tasks.workunit.client.1.smithi044.stdout:2/50: dwrite f1 [0,4194304] 0 2024-04-22T23:45:58.413 INFO:tasks.workunit.client.1.smithi044.stdout:7/105: dread - dd/d1f/d24/f1e zero size 2024-04-22T23:45:58.415 INFO:tasks.workunit.client.1.smithi044.stdout:8/104: rename db/f1a to db/f27 0 2024-04-22T23:45:58.422 INFO:tasks.workunit.client.1.smithi044.stdout:1/98: mknod d1/d8/c22 0 2024-04-22T23:45:58.423 INFO:tasks.workunit.client.1.smithi044.stdout:7/106: dread fc [0,4194304] 0 2024-04-22T23:45:58.424 INFO:tasks.workunit.client.1.smithi044.stdout:2/51: dwrite f6 [0,4194304] 0 2024-04-22T23:45:58.428 INFO:tasks.workunit.client.1.smithi044.stdout:6/106: creat d5/d8/f22 x:0 0 0 2024-04-22T23:45:58.429 INFO:tasks.workunit.client.1.smithi044.stdout:3/101: creat d9/db/d22/f24 x:0 0 0 2024-04-22T23:45:58.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/102: fsync d9/d12/f1f 0 2024-04-22T23:45:58.432 INFO:tasks.workunit.client.1.smithi044.stdout:9/88: symlink d2/d6/l21 0 2024-04-22T23:45:58.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/103: dread d9/d12/f1f [0,4194304] 0 2024-04-22T23:45:58.436 INFO:tasks.workunit.client.1.smithi044.stdout:3/104: write f5 [890985,40832] 0 2024-04-22T23:45:58.437 INFO:tasks.workunit.client.1.smithi044.stdout:8/105: fsync db/fe 0 2024-04-22T23:45:58.443 INFO:tasks.workunit.client.1.smithi044.stdout:2/52: creat d8/fd x:0 0 0 2024-04-22T23:45:58.443 INFO:tasks.workunit.client.1.smithi044.stdout:1/99: dread d1/f7 [0,4194304] 0 2024-04-22T23:45:58.443 INFO:tasks.workunit.client.1.smithi044.stdout:2/53: dread - d8/f9 zero size 2024-04-22T23:45:58.444 INFO:tasks.workunit.client.1.smithi044.stdout:1/100: fdatasync d1/d14/f1f 0 2024-04-22T23:45:58.445 INFO:tasks.workunit.client.1.smithi044.stdout:7/107: symlink dd/d1f/d24/l25 0 2024-04-22T23:45:58.450 INFO:tasks.workunit.client.1.smithi044.stdout:9/89: mknod d2/dc/c22 0 2024-04-22T23:45:58.458 INFO:tasks.workunit.client.1.smithi044.stdout:8/106: mknod db/d14/d24/c28 0 2024-04-22T23:45:58.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:58 smithi044 ceph-mon[34463]: pgmap v35: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 67 MiB/s rd, 151 MiB/s wr, 383 op/s 2024-04-22T23:45:58.460 INFO:tasks.workunit.client.1.smithi044.stdout:2/54: creat d8/fe x:0 0 0 2024-04-22T23:45:58.461 INFO:tasks.workunit.client.1.smithi044.stdout:6/107: mkdir d5/dd/d1d/d23 0 2024-04-22T23:45:58.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/108: mkdir dd/d1f/d24/d26 0 2024-04-22T23:45:58.464 INFO:tasks.workunit.client.1.smithi044.stdout:4/87: sync 2024-04-22T23:45:58.464 INFO:tasks.workunit.client.1.smithi044.stdout:1/101: mkdir d1/d8/d1c/d23 0 2024-04-22T23:45:58.466 INFO:tasks.workunit.client.1.smithi044.stdout:2/55: dread f4 [0,4194304] 0 2024-04-22T23:45:58.474 INFO:tasks.workunit.client.1.smithi044.stdout:4/88: dwrite f1 [0,4194304] 0 2024-04-22T23:45:58.474 INFO:tasks.workunit.client.1.smithi044.stdout:2/56: dread d8/fc [0,4194304] 0 2024-04-22T23:45:58.475 INFO:tasks.workunit.client.1.smithi044.stdout:2/57: read - d8/fe zero size 2024-04-22T23:45:58.476 INFO:tasks.workunit.client.1.smithi044.stdout:6/108: creat d5/dd/d18/f24 x:0 0 0 2024-04-22T23:45:58.477 INFO:tasks.workunit.client.1.smithi044.stdout:2/58: write f1 [4823209,17161] 0 2024-04-22T23:45:58.478 INFO:tasks.workunit.client.1.smithi044.stdout:2/59: dread - f7 zero size 2024-04-22T23:45:58.486 INFO:tasks.workunit.client.1.smithi044.stdout:5/54: link f0 fe 0 2024-04-22T23:45:58.490 INFO:tasks.workunit.client.1.smithi044.stdout:4/89: creat d5/db/f21 x:0 0 0 2024-04-22T23:45:58.491 INFO:tasks.workunit.client.1.smithi044.stdout:4/90: chown d5/db/dd/f17 55911 1 2024-04-22T23:45:58.493 INFO:tasks.workunit.client.1.smithi044.stdout:6/109: dread - d5/dd/d18/f20 zero size 2024-04-22T23:45:58.495 INFO:tasks.workunit.client.1.smithi044.stdout:7/109: mkdir dd/d27 0 2024-04-22T23:45:58.498 INFO:tasks.workunit.client.1.smithi044.stdout:2/60: rename d8/fa to d8/ff 0 2024-04-22T23:45:58.512 INFO:tasks.workunit.client.1.smithi044.stdout:6/110: creat d5/dd/d1d/f25 x:0 0 0 2024-04-22T23:45:58.514 INFO:tasks.workunit.client.1.smithi044.stdout:3/105: dwrite d9/db/d15/f1b [0,4194304] 0 2024-04-22T23:45:58.516 INFO:tasks.workunit.client.1.smithi044.stdout:3/106: readlink d9/d14/l16 0 2024-04-22T23:45:58.519 INFO:tasks.workunit.client.1.smithi044.stdout:8/107: dwrite db/f11 [4194304,4194304] 0 2024-04-22T23:45:58.524 INFO:tasks.workunit.client.1.smithi044.stdout:3/107: dread d9/db/d15/f1b [0,4194304] 0 2024-04-22T23:45:58.525 INFO:tasks.workunit.client.1.smithi044.stdout:3/108: readlink d9/le 0 2024-04-22T23:45:58.533 INFO:tasks.workunit.client.1.smithi044.stdout:2/61: creat d8/f10 x:0 0 0 2024-04-22T23:45:58.537 INFO:tasks.workunit.client.1.smithi044.stdout:8/108: mknod db/d14/d24/c29 0 2024-04-22T23:45:58.541 INFO:tasks.workunit.client.1.smithi044.stdout:5/55: creat ff x:0 0 0 2024-04-22T23:45:58.541 INFO:tasks.workunit.client.1.smithi044.stdout:8/109: dread f6 [0,4194304] 0 2024-04-22T23:45:58.542 INFO:tasks.workunit.client.1.smithi044.stdout:8/110: dread - db/f1d zero size 2024-04-22T23:45:58.547 INFO:tasks.workunit.client.1.smithi044.stdout:5/56: dread - fa zero size 2024-04-22T23:45:58.548 INFO:tasks.workunit.client.1.smithi044.stdout:1/102: truncate d1/d8/f12 3303287 0 2024-04-22T23:45:58.549 INFO:tasks.workunit.client.1.smithi044.stdout:1/103: write d1/d8/f1a [158773,77444] 0 2024-04-22T23:45:58.550 INFO:tasks.workunit.client.1.smithi044.stdout:1/104: write d1/d14/f1f [644567,11137] 0 2024-04-22T23:45:58.551 INFO:tasks.workunit.client.1.smithi044.stdout:2/62: link f2 d8/f11 0 2024-04-22T23:45:58.552 INFO:tasks.workunit.client.1.smithi044.stdout:2/63: fsync d8/fe 0 2024-04-22T23:45:58.552 INFO:tasks.workunit.client.1.smithi044.stdout:2/64: dread - f7 zero size 2024-04-22T23:45:58.562 INFO:tasks.workunit.client.1.smithi044.stdout:2/65: symlink d8/l12 0 2024-04-22T23:45:58.562 INFO:tasks.workunit.client.1.smithi044.stdout:2/66: write f7 [265828,68639] 0 2024-04-22T23:45:58.563 INFO:tasks.workunit.client.1.smithi044.stdout:3/109: getdents d9/db 0 2024-04-22T23:45:58.565 INFO:tasks.workunit.client.1.smithi044.stdout:5/57: mknod c10 0 2024-04-22T23:45:58.569 INFO:tasks.workunit.client.1.smithi044.stdout:3/110: dread d9/f19 [0,4194304] 0 2024-04-22T23:45:58.571 INFO:tasks.workunit.client.1.smithi044.stdout:3/111: chown d9/lf 2 1 2024-04-22T23:45:58.572 INFO:tasks.workunit.client.1.smithi044.stdout:6/111: dwrite d5/dd/d18/f20 [0,4194304] 0 2024-04-22T23:45:58.578 INFO:tasks.workunit.client.1.smithi044.stdout:3/112: dread f8 [0,4194304] 0 2024-04-22T23:45:58.580 INFO:tasks.workunit.client.1.smithi044.stdout:1/105: write d1/d8/f18 [3584608,92132] 0 2024-04-22T23:45:58.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/67: dwrite d8/ff [4194304,4194304] 0 2024-04-22T23:45:58.584 INFO:tasks.workunit.client.1.smithi044.stdout:5/58: write f8 [907021,38429] 0 2024-04-22T23:45:58.586 INFO:tasks.workunit.client.1.smithi044.stdout:1/106: stat d1/d14 0 2024-04-22T23:45:58.588 INFO:tasks.workunit.client.1.smithi044.stdout:6/112: dread d5/d8/f13 [0,4194304] 0 2024-04-22T23:45:58.591 INFO:tasks.workunit.client.1.smithi044.stdout:2/68: truncate d8/f9 149076 0 2024-04-22T23:45:58.592 INFO:tasks.workunit.client.1.smithi044.stdout:6/113: truncate d5/f21 4204636 0 2024-04-22T23:45:58.593 INFO:tasks.workunit.client.1.smithi044.stdout:8/111: dwrite db/f12 [0,4194304] 0 2024-04-22T23:45:58.595 INFO:tasks.workunit.client.1.smithi044.stdout:2/69: chown d8/fe 300 1 2024-04-22T23:45:58.595 INFO:tasks.workunit.client.1.smithi044.stdout:8/112: write db/f1d [1033609,41518] 0 2024-04-22T23:45:58.598 INFO:tasks.workunit.client.1.smithi044.stdout:8/113: chown db/d14/f25 90 1 2024-04-22T23:45:58.600 INFO:tasks.workunit.client.1.smithi044.stdout:5/59: unlink c10 0 2024-04-22T23:45:58.600 INFO:tasks.workunit.client.1.smithi044.stdout:5/60: rmdir - no directory 2024-04-22T23:45:58.604 INFO:tasks.workunit.client.1.smithi044.stdout:1/107: mkdir d1/d8/d24 0 2024-04-22T23:45:58.607 INFO:tasks.workunit.client.1.smithi044.stdout:2/70: dread f2 [0,4194304] 0 2024-04-22T23:45:58.607 INFO:tasks.workunit.client.1.smithi044.stdout:6/114: dwrite d5/f21 [0,4194304] 0 2024-04-22T23:45:58.607 INFO:tasks.workunit.client.1.smithi044.stdout:1/108: fdatasync d1/d14/f1f 0 2024-04-22T23:45:58.614 INFO:tasks.workunit.client.1.smithi044.stdout:1/109: truncate d1/d8/f19 677902 0 2024-04-22T23:45:58.614 INFO:tasks.workunit.client.1.smithi044.stdout:8/114: dwrite db/f19 [0,4194304] 0 2024-04-22T23:45:58.619 INFO:tasks.workunit.client.1.smithi044.stdout:2/71: mknod d8/c13 0 2024-04-22T23:45:58.632 INFO:tasks.workunit.client.1.smithi044.stdout:1/110: mkdir d1/d8/d25 0 2024-04-22T23:45:58.633 INFO:tasks.workunit.client.1.smithi044.stdout:1/111: chown d1/d14/c17 293630219 1 2024-04-22T23:45:58.636 INFO:tasks.workunit.client.1.smithi044.stdout:0/74: sync 2024-04-22T23:45:58.637 INFO:tasks.workunit.client.1.smithi044.stdout:1/112: dread d1/f7 [0,4194304] 0 2024-04-22T23:45:58.642 INFO:tasks.workunit.client.1.smithi044.stdout:6/115: creat d5/dd/d1d/d23/f26 x:0 0 0 2024-04-22T23:45:58.644 INFO:tasks.workunit.client.1.smithi044.stdout:1/113: dread d1/d14/f1f [0,4194304] 0 2024-04-22T23:45:58.644 INFO:tasks.workunit.client.1.smithi044.stdout:6/116: write d5/dd/fe [2965563,115204] 0 2024-04-22T23:45:58.645 INFO:tasks.workunit.client.1.smithi044.stdout:1/114: chown d1/d8/l13 25896987 1 2024-04-22T23:45:58.646 INFO:tasks.workunit.client.1.smithi044.stdout:2/72: write f4 [1727028,117289] 0 2024-04-22T23:45:58.648 INFO:tasks.workunit.client.1.smithi044.stdout:3/113: getdents d9 0 2024-04-22T23:45:58.648 INFO:tasks.workunit.client.1.smithi044.stdout:1/115: fdatasync d1/d8/f18 0 2024-04-22T23:45:58.653 INFO:tasks.workunit.client.1.smithi044.stdout:5/61: link f8 f11 0 2024-04-22T23:45:58.654 INFO:tasks.workunit.client.1.smithi044.stdout:1/116: chown d1/d14/l16 58255530 1 2024-04-22T23:45:58.655 INFO:tasks.workunit.client.1.smithi044.stdout:6/117: dwrite d5/dd/f16 [0,4194304] 0 2024-04-22T23:45:58.656 INFO:tasks.workunit.client.1.smithi044.stdout:2/73: creat d8/f14 x:0 0 0 2024-04-22T23:45:58.657 INFO:tasks.workunit.client.1.smithi044.stdout:2/74: stat d8/f11 0 2024-04-22T23:45:58.661 INFO:tasks.workunit.client.1.smithi044.stdout:0/75: link d0/f16 d0/d2/f1e 0 2024-04-22T23:45:58.665 INFO:tasks.workunit.client.1.smithi044.stdout:3/114: mkdir d9/db/d15/d25 0 2024-04-22T23:45:58.666 INFO:tasks.workunit.client.1.smithi044.stdout:3/115: write d9/db/d15/f18 [1199051,79197] 0 2024-04-22T23:45:58.668 INFO:tasks.workunit.client.1.smithi044.stdout:3/116: write d9/f17 [984810,12722] 0 2024-04-22T23:45:58.669 INFO:tasks.workunit.client.1.smithi044.stdout:1/117: creat d1/d8/f26 x:0 0 0 2024-04-22T23:45:58.670 INFO:tasks.workunit.client.1.smithi044.stdout:8/115: link db/fe db/f2a 0 2024-04-22T23:45:58.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/116: chown c3 53199 1 2024-04-22T23:45:58.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/117: fdatasync db/f11 0 2024-04-22T23:45:58.673 INFO:tasks.workunit.client.1.smithi044.stdout:8/118: write db/d14/d24/f26 [702988,129294] 0 2024-04-22T23:45:58.676 INFO:tasks.workunit.client.1.smithi044.stdout:6/118: rename d5/d8/f13 to d5/dd/f27 0 2024-04-22T23:45:58.677 INFO:tasks.workunit.client.1.smithi044.stdout:5/62: write f11 [2594457,91080] 0 2024-04-22T23:45:58.678 INFO:tasks.workunit.client.1.smithi044.stdout:2/75: truncate f1 3384003 0 2024-04-22T23:45:58.682 INFO:tasks.workunit.client.1.smithi044.stdout:5/63: dread f8 [0,4194304] 0 2024-04-22T23:45:58.688 INFO:tasks.workunit.client.1.smithi044.stdout:1/118: fsync d1/d8/f26 0 2024-04-22T23:45:58.688 INFO:tasks.workunit.client.1.smithi044.stdout:9/90: sync 2024-04-22T23:45:58.688 INFO:tasks.workunit.client.1.smithi044.stdout:4/91: sync 2024-04-22T23:45:58.689 INFO:tasks.workunit.client.1.smithi044.stdout:4/92: readlink d5/db/dd/lf 0 2024-04-22T23:45:58.690 INFO:tasks.workunit.client.1.smithi044.stdout:9/91: write d2/f3 [4650114,76631] 0 2024-04-22T23:45:58.699 INFO:tasks.workunit.client.1.smithi044.stdout:8/119: rmdir db 39 2024-04-22T23:45:58.706 INFO:tasks.workunit.client.1.smithi044.stdout:3/117: dwrite d9/f19 [0,4194304] 0 2024-04-22T23:45:58.708 INFO:tasks.workunit.client.1.smithi044.stdout:6/119: mkdir d5/dd/d28 0 2024-04-22T23:45:58.710 INFO:tasks.workunit.client.1.smithi044.stdout:0/76: rename d0/c14 to d0/d2/dd/d19/c1f 0 2024-04-22T23:45:58.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/76: rmdir d8 39 2024-04-22T23:45:58.785 INFO:tasks.workunit.client.1.smithi044.stdout:4/93: write d5/db/dd/f11 [1957415,110854] 0 2024-04-22T23:45:58.786 INFO:tasks.workunit.client.1.smithi044.stdout:9/92: symlink d2/d9/l23 0 2024-04-22T23:45:58.787 INFO:tasks.workunit.client.1.smithi044.stdout:4/94: truncate d5/db/d16/f1c 457274 0 2024-04-22T23:45:58.788 INFO:tasks.workunit.client.1.smithi044.stdout:4/95: fdatasync d5/db/dd/f11 0 2024-04-22T23:45:58.788 INFO:tasks.workunit.client.1.smithi044.stdout:8/120: stat db/d14/d24/f26 0 2024-04-22T23:45:58.790 INFO:tasks.workunit.client.1.smithi044.stdout:0/77: creat d0/d2/dd/d19/f20 x:0 0 0 2024-04-22T23:45:58.792 INFO:tasks.workunit.client.1.smithi044.stdout:0/78: write d0/d2/dd/f13 [1030619,102054] 0 2024-04-22T23:45:58.794 INFO:tasks.workunit.client.1.smithi044.stdout:0/79: truncate d0/f16 865613 0 2024-04-22T23:45:58.800 INFO:tasks.workunit.client.1.smithi044.stdout:2/77: dwrite d8/fd [0,4194304] 0 2024-04-22T23:45:58.802 INFO:tasks.workunit.client.1.smithi044.stdout:8/121: creat db/d14/f2b x:0 0 0 2024-04-22T23:45:58.804 INFO:tasks.workunit.client.1.smithi044.stdout:5/64: creat f12 x:0 0 0 2024-04-22T23:45:58.808 INFO:tasks.workunit.client.1.smithi044.stdout:9/93: symlink d2/l24 0 2024-04-22T23:45:58.814 INFO:tasks.workunit.client.1.smithi044.stdout:2/78: creat d8/f15 x:0 0 0 2024-04-22T23:45:58.821 INFO:tasks.workunit.client.1.smithi044.stdout:1/119: link d1/d8/f12 d1/d8/d1c/d23/f27 0 2024-04-22T23:45:58.826 INFO:tasks.workunit.client.1.smithi044.stdout:0/80: dwrite d0/d2/dd/f1a [0,4194304] 0 2024-04-22T23:45:58.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/120: getdents d5/dd/d18 0 2024-04-22T23:45:58.830 INFO:tasks.workunit.client.1.smithi044.stdout:4/96: write d5/db/dd/f10 [7538970,59736] 0 2024-04-22T23:45:58.839 INFO:tasks.workunit.client.1.smithi044.stdout:9/94: write d2/f4 [1861517,29294] 0 2024-04-22T23:45:58.840 INFO:tasks.workunit.client.1.smithi044.stdout:9/95: readlink d2/d9/l23 0 2024-04-22T23:45:58.847 INFO:tasks.workunit.client.1.smithi044.stdout:1/120: dread d1/d8/d1c/d23/f27 [0,4194304] 0 2024-04-22T23:45:58.848 INFO:tasks.workunit.client.1.smithi044.stdout:0/81: creat d0/d2/dd/f21 x:0 0 0 2024-04-22T23:45:58.850 INFO:tasks.workunit.client.1.smithi044.stdout:0/82: readlink d0/l18 0 2024-04-22T23:45:58.852 INFO:tasks.workunit.client.1.smithi044.stdout:0/83: dread - d0/d2/f15 zero size 2024-04-22T23:45:58.853 INFO:tasks.workunit.client.1.smithi044.stdout:4/97: unlink d5/db/d16/d18/f1f 0 2024-04-22T23:45:58.854 INFO:tasks.workunit.client.1.smithi044.stdout:8/122: creat db/f2c x:0 0 0 2024-04-22T23:45:58.856 INFO:tasks.workunit.client.1.smithi044.stdout:5/65: rename fc to f13 0 2024-04-22T23:45:58.858 INFO:tasks.workunit.client.1.smithi044.stdout:5/66: write fa [283096,91871] 0 2024-04-22T23:45:58.858 INFO:tasks.workunit.client.1.smithi044.stdout:4/98: write d5/db/dd/f1b [69635,30904] 0 2024-04-22T23:45:58.859 INFO:tasks.workunit.client.1.smithi044.stdout:2/79: write d8/f11 [3294350,105113] 0 2024-04-22T23:45:58.860 INFO:tasks.workunit.client.1.smithi044.stdout:1/121: dwrite d1/d8/f19 [0,4194304] 0 2024-04-22T23:45:58.862 INFO:tasks.workunit.client.1.smithi044.stdout:5/67: chown f13 15851 1 2024-04-22T23:45:58.862 INFO:tasks.workunit.client.1.smithi044.stdout:5/68: chown f13 433089668 1 2024-04-22T23:45:58.864 INFO:tasks.workunit.client.1.smithi044.stdout:2/80: write f2 [2841252,103518] 0 2024-04-22T23:45:58.866 INFO:tasks.workunit.client.1.smithi044.stdout:2/81: read f4 [1350813,13981] 0 2024-04-22T23:45:58.869 INFO:tasks.workunit.client.1.smithi044.stdout:7/110: sync 2024-04-22T23:45:58.871 INFO:tasks.workunit.client.1.smithi044.stdout:2/82: dread f2 [0,4194304] 0 2024-04-22T23:45:58.871 INFO:tasks.workunit.client.1.smithi044.stdout:8/123: unlink f0 0 2024-04-22T23:45:58.872 INFO:tasks.workunit.client.1.smithi044.stdout:2/83: write d8/fe [573162,95869] 0 2024-04-22T23:45:58.875 INFO:tasks.workunit.client.1.smithi044.stdout:9/96: mknod d2/c25 0 2024-04-22T23:45:58.876 INFO:tasks.workunit.client.1.smithi044.stdout:8/124: dread db/fe [0,4194304] 0 2024-04-22T23:45:58.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/69: readlink l3 0 2024-04-22T23:45:58.886 INFO:tasks.workunit.client.1.smithi044.stdout:1/122: unlink d1/d8/lc 0 2024-04-22T23:45:58.889 INFO:tasks.workunit.client.1.smithi044.stdout:7/111: creat dd/d1f/d24/f28 x:0 0 0 2024-04-22T23:45:58.889 INFO:tasks.workunit.client.1.smithi044.stdout:7/112: write dd/d1a/f1d [1530714,9068] 0 2024-04-22T23:45:58.892 INFO:tasks.workunit.client.1.smithi044.stdout:2/84: mknod d8/c16 0 2024-04-22T23:45:58.892 INFO:tasks.workunit.client.1.smithi044.stdout:2/85: chown d8/fd 27715 1 2024-04-22T23:45:58.894 INFO:tasks.workunit.client.1.smithi044.stdout:2/86: write f3 [399972,11739] 0 2024-04-22T23:45:58.895 INFO:tasks.workunit.client.1.smithi044.stdout:2/87: chown f6 223073571 1 2024-04-22T23:45:58.897 INFO:tasks.workunit.client.1.smithi044.stdout:7/113: dwrite dd/ff [0,4194304] 0 2024-04-22T23:45:58.900 INFO:tasks.workunit.client.1.smithi044.stdout:8/125: symlink db/d14/l2d 0 2024-04-22T23:45:58.906 INFO:tasks.workunit.client.1.smithi044.stdout:4/99: mkdir d5/db/d16/d18/d20/d22 0 2024-04-22T23:45:58.908 INFO:tasks.workunit.client.1.smithi044.stdout:1/123: truncate d1/f7 3700582 0 2024-04-22T23:45:58.910 INFO:tasks.workunit.client.1.smithi044.stdout:1/124: read d1/d14/f1f [36954,74304] 0 2024-04-22T23:45:58.912 INFO:tasks.workunit.client.1.smithi044.stdout:0/84: truncate d0/f8 2819230 0 2024-04-22T23:45:58.919 INFO:tasks.workunit.client.1.smithi044.stdout:8/126: chown db/c20 6 1 2024-04-22T23:45:58.920 INFO:tasks.workunit.client.1.smithi044.stdout:9/97: dwrite d2/dc/f15 [0,4194304] 0 2024-04-22T23:45:58.921 INFO:tasks.workunit.client.1.smithi044.stdout:9/98: fdatasync d2/d5/f1f 0 2024-04-22T23:45:58.922 INFO:tasks.workunit.client.1.smithi044.stdout:4/100: creat d5/db/d16/d18/d20/f23 x:0 0 0 2024-04-22T23:45:58.923 INFO:tasks.workunit.client.1.smithi044.stdout:4/101: write d5/db/dd/f17 [856802,9190] 0 2024-04-22T23:45:58.924 INFO:tasks.workunit.client.1.smithi044.stdout:4/102: fsync d5/db/f21 0 2024-04-22T23:45:58.932 INFO:tasks.workunit.client.1.smithi044.stdout:2/88: link d8/f15 d8/f17 0 2024-04-22T23:45:58.933 INFO:tasks.workunit.client.1.smithi044.stdout:7/114: write fc [2909329,2582] 0 2024-04-22T23:45:58.936 INFO:tasks.workunit.client.1.smithi044.stdout:2/89: dread d8/ff [4194304,4194304] 0 2024-04-22T23:45:58.937 INFO:tasks.workunit.client.1.smithi044.stdout:5/70: creat f14 x:0 0 0 2024-04-22T23:45:58.937 INFO:tasks.workunit.client.1.smithi044.stdout:2/90: readlink d8/l12 0 2024-04-22T23:45:58.938 INFO:tasks.workunit.client.1.smithi044.stdout:8/127: symlink db/d14/l2e 0 2024-04-22T23:45:58.939 INFO:tasks.workunit.client.1.smithi044.stdout:8/128: write f5 [5878478,14659] 0 2024-04-22T23:45:58.941 INFO:tasks.workunit.client.1.smithi044.stdout:9/99: symlink d2/dc/l26 0 2024-04-22T23:45:58.946 INFO:tasks.workunit.client.1.smithi044.stdout:0/85: write d0/d2/f15 [417021,54360] 0 2024-04-22T23:45:58.949 INFO:tasks.workunit.client.1.smithi044.stdout:0/86: truncate d0/d2/dd/f1d 597751 0 2024-04-22T23:45:58.953 INFO:tasks.workunit.client.1.smithi044.stdout:7/115: unlink dd/d1f/d24/f17 0 2024-04-22T23:45:58.958 INFO:tasks.workunit.client.1.smithi044.stdout:1/125: write d1/f7 [3383519,107838] 0 2024-04-22T23:45:58.960 INFO:tasks.workunit.client.1.smithi044.stdout:7/116: dwrite dd/d1f/d24/f19 [0,4194304] 0 2024-04-22T23:45:58.966 INFO:tasks.workunit.client.1.smithi044.stdout:0/87: unlink d0/d2/dd/f21 0 2024-04-22T23:45:58.967 INFO:tasks.workunit.client.1.smithi044.stdout:0/88: read d0/d2/f15 [154773,93362] 0 2024-04-22T23:45:58.970 INFO:tasks.workunit.client.1.smithi044.stdout:5/71: mkdir d15 0 2024-04-22T23:45:58.975 INFO:tasks.workunit.client.1.smithi044.stdout:9/100: link d2/dc/f15 d2/dc/f27 0 2024-04-22T23:45:58.980 INFO:tasks.workunit.client.1.smithi044.stdout:1/126: symlink d1/d14/l28 0 2024-04-22T23:45:58.983 INFO:tasks.workunit.client.1.smithi044.stdout:7/117: symlink dd/d1f/d24/l29 0 2024-04-22T23:45:58.987 INFO:tasks.workunit.client.1.smithi044.stdout:7/118: dread f8 [0,4194304] 0 2024-04-22T23:45:58.988 INFO:tasks.workunit.client.1.smithi044.stdout:7/119: write dd/d1f/d24/f1e [256896,52144] 0 2024-04-22T23:45:58.990 INFO:tasks.workunit.client.1.smithi044.stdout:2/91: truncate d8/fe 350571 0 2024-04-22T23:45:58.991 INFO:tasks.workunit.client.1.smithi044.stdout:2/92: fdatasync d8/f10 0 2024-04-22T23:45:58.992 INFO:tasks.workunit.client.1.smithi044.stdout:2/93: dread - d8/f15 zero size 2024-04-22T23:45:58.998 INFO:tasks.workunit.client.1.smithi044.stdout:4/103: getdents d5/db/d16 0 2024-04-22T23:45:59.002 INFO:tasks.workunit.client.1.smithi044.stdout:4/104: dread d5/db/dd/f17 [0,4194304] 0 2024-04-22T23:45:59.004 INFO:tasks.workunit.client.1.smithi044.stdout:4/105: truncate d5/db/dd/f1b 393207 0 2024-04-22T23:45:59.004 INFO:tasks.workunit.client.1.smithi044.stdout:4/106: chown d5 0 1 2024-04-22T23:45:59.009 INFO:tasks.workunit.client.1.smithi044.stdout:7/120: mknod dd/d1f/d24/c2a 0 2024-04-22T23:45:59.017 INFO:tasks.workunit.client.1.smithi044.stdout:0/89: dwrite d0/d2/f6 [0,4194304] 0 2024-04-22T23:45:59.022 INFO:tasks.workunit.client.1.smithi044.stdout:2/94: creat d8/f18 x:0 0 0 2024-04-22T23:45:59.023 INFO:tasks.workunit.client.1.smithi044.stdout:5/72: mkdir d15/d16 0 2024-04-22T23:45:59.024 INFO:tasks.workunit.client.1.smithi044.stdout:5/73: write f12 [63262,8132] 0 2024-04-22T23:45:59.026 INFO:tasks.workunit.client.1.smithi044.stdout:9/101: mknod d2/c28 0 2024-04-22T23:45:59.032 INFO:tasks.workunit.client.1.smithi044.stdout:1/127: creat d1/d8/d25/f29 x:0 0 0 2024-04-22T23:45:59.036 INFO:tasks.workunit.client.1.smithi044.stdout:4/107: mkdir d5/db/dd/d24 0 2024-04-22T23:45:59.042 INFO:tasks.workunit.client.1.smithi044.stdout:2/95: mkdir d8/d19 0 2024-04-22T23:45:59.042 INFO:tasks.workunit.client.1.smithi044.stdout:0/90: dread d0/d2/f15 [0,4194304] 0 2024-04-22T23:45:59.044 INFO:tasks.workunit.client.1.smithi044.stdout:9/102: rmdir d2/d6 39 2024-04-22T23:45:59.047 INFO:tasks.workunit.client.1.smithi044.stdout:1/128: fsync d1/d8/d1c/d23/f27 0 2024-04-22T23:45:59.050 INFO:tasks.workunit.client.1.smithi044.stdout:4/108: rename d5/c8 to d5/db/d16/c25 0 2024-04-22T23:45:59.058 INFO:tasks.workunit.client.1.smithi044.stdout:2/96: dread f1 [0,4194304] 0 2024-04-22T23:45:59.058 INFO:tasks.workunit.client.1.smithi044.stdout:5/74: creat d15/d16/f17 x:0 0 0 2024-04-22T23:45:59.059 INFO:tasks.workunit.client.1.smithi044.stdout:5/75: dread - d15/d16/f17 zero size 2024-04-22T23:45:59.062 INFO:tasks.workunit.client.1.smithi044.stdout:2/97: dread f1 [0,4194304] 0 2024-04-22T23:45:59.068 INFO:tasks.workunit.client.1.smithi044.stdout:1/129: unlink d1/d14/f1f 0 2024-04-22T23:45:59.069 INFO:tasks.workunit.client.1.smithi044.stdout:1/130: write d1/d8/f1e [470481,2917] 0 2024-04-22T23:45:59.073 INFO:tasks.workunit.client.1.smithi044.stdout:7/121: rmdir dd/d1f/d24/d26 0 2024-04-22T23:45:59.076 INFO:tasks.workunit.client.1.smithi044.stdout:5/76: creat d15/f18 x:0 0 0 2024-04-22T23:45:59.082 INFO:tasks.workunit.client.1.smithi044.stdout:2/98: creat d8/f1a x:0 0 0 2024-04-22T23:45:59.085 INFO:tasks.workunit.client.1.smithi044.stdout:9/103: rename d2/dc/l14 to d2/d6/l29 0 2024-04-22T23:45:59.085 INFO:tasks.workunit.client.1.smithi044.stdout:9/104: stat d2/dc/c22 0 2024-04-22T23:45:59.090 INFO:tasks.workunit.client.1.smithi044.stdout:7/122: dread dd/d1f/d24/f15 [0,4194304] 0 2024-04-22T23:45:59.090 INFO:tasks.workunit.client.1.smithi044.stdout:7/123: dread - dd/fe zero size 2024-04-22T23:45:59.091 INFO:tasks.workunit.client.1.smithi044.stdout:1/131: creat d1/d8/d25/f2a x:0 0 0 2024-04-22T23:45:59.097 INFO:tasks.workunit.client.1.smithi044.stdout:5/77: symlink d15/d16/l19 0 2024-04-22T23:45:59.098 INFO:tasks.workunit.client.1.smithi044.stdout:5/78: dread - f14 zero size 2024-04-22T23:45:59.099 INFO:tasks.workunit.client.1.smithi044.stdout:1/132: dwrite d1/d8/d25/f29 [0,4194304] 0 2024-04-22T23:45:59.103 INFO:tasks.workunit.client.1.smithi044.stdout:5/79: dread f11 [0,4194304] 0 2024-04-22T23:45:59.104 INFO:tasks.workunit.client.1.smithi044.stdout:7/124: creat dd/d1f/d24/f2b x:0 0 0 2024-04-22T23:45:59.105 INFO:tasks.workunit.client.1.smithi044.stdout:7/125: chown dd/d1f/d24/f15 89510 1 2024-04-22T23:45:59.107 INFO:tasks.workunit.client.1.smithi044.stdout:5/80: dread f5 [4194304,4194304] 0 2024-04-22T23:45:59.108 INFO:tasks.workunit.client.1.smithi044.stdout:3/118: sync 2024-04-22T23:45:59.108 INFO:tasks.workunit.client.1.smithi044.stdout:6/121: sync 2024-04-22T23:45:59.110 INFO:tasks.workunit.client.1.smithi044.stdout:5/81: write d15/d16/f17 [560326,113450] 0 2024-04-22T23:45:59.113 INFO:tasks.workunit.client.1.smithi044.stdout:0/91: truncate d0/d2/f1e 558258 0 2024-04-22T23:45:59.115 INFO:tasks.workunit.client.1.smithi044.stdout:3/119: dread d9/f19 [0,4194304] 0 2024-04-22T23:45:59.115 INFO:tasks.workunit.client.1.smithi044.stdout:1/133: symlink d1/l2b 0 2024-04-22T23:45:59.116 INFO:tasks.workunit.client.1.smithi044.stdout:9/105: getdents d2/dc/d1d 0 2024-04-22T23:45:59.117 INFO:tasks.workunit.client.1.smithi044.stdout:9/106: readlink d2/d6/l21 0 2024-04-22T23:45:59.118 INFO:tasks.workunit.client.1.smithi044.stdout:9/107: chown d2/f1c 4148007 1 2024-04-22T23:45:59.127 INFO:tasks.workunit.client.1.smithi044.stdout:6/122: rmdir d5/dd 39 2024-04-22T23:45:59.131 INFO:tasks.workunit.client.1.smithi044.stdout:4/109: write d5/db/dd/f1d [3509721,100498] 0 2024-04-22T23:45:59.135 INFO:tasks.workunit.client.1.smithi044.stdout:0/92: creat d0/f22 x:0 0 0 2024-04-22T23:45:59.136 INFO:tasks.workunit.client.1.smithi044.stdout:5/82: dwrite f13 [0,4194304] 0 2024-04-22T23:45:59.138 INFO:tasks.workunit.client.1.smithi044.stdout:5/83: truncate ff 807184 0 2024-04-22T23:45:59.138 INFO:tasks.workunit.client.1.smithi044.stdout:1/134: creat d1/d8/d25/f2c x:0 0 0 2024-04-22T23:45:59.148 INFO:tasks.workunit.client.1.smithi044.stdout:5/84: dwrite d15/d16/f17 [0,4194304] 0 2024-04-22T23:45:59.156 INFO:tasks.workunit.client.1.smithi044.stdout:6/123: write d5/dd/f1b [693657,82247] 0 2024-04-22T23:45:59.159 INFO:tasks.workunit.client.1.smithi044.stdout:5/85: dwrite f13 [4194304,4194304] 0 2024-04-22T23:45:59.159 INFO:tasks.workunit.client.1.smithi044.stdout:6/124: chown d5/dd/f1e 16360 1 2024-04-22T23:45:59.159 INFO:tasks.workunit.client.1.smithi044.stdout:2/99: getdents d8 0 2024-04-22T23:45:59.160 INFO:tasks.workunit.client.1.smithi044.stdout:5/86: stat d15/d16/f17 0 2024-04-22T23:45:59.162 INFO:tasks.workunit.client.1.smithi044.stdout:4/110: rename d5/db/dd/c1e to d5/db/c26 0 2024-04-22T23:45:59.169 INFO:tasks.workunit.client.1.smithi044.stdout:3/120: symlink d9/db/d15/d25/l26 0 2024-04-22T23:45:59.171 INFO:tasks.workunit.client.1.smithi044.stdout:9/108: truncate d2/dc/f15 3930237 0 2024-04-22T23:45:59.171 INFO:tasks.workunit.client.1.smithi044.stdout:9/109: chown d2 17 1 2024-04-22T23:45:59.173 INFO:tasks.workunit.client.1.smithi044.stdout:7/126: creat dd/d1f/f2c x:0 0 0 2024-04-22T23:45:59.176 INFO:tasks.workunit.client.1.smithi044.stdout:2/100: rmdir d8 39 2024-04-22T23:45:59.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/125: rmdir d5/dd/d1d 39 2024-04-22T23:45:59.180 INFO:tasks.workunit.client.1.smithi044.stdout:5/87: creat d15/d16/f1a x:0 0 0 2024-04-22T23:45:59.181 INFO:tasks.workunit.client.1.smithi044.stdout:8/129: sync 2024-04-22T23:45:59.183 INFO:tasks.workunit.client.1.smithi044.stdout:1/135: rename d1/d8/c11 to d1/c2d 0 2024-04-22T23:45:59.188 INFO:tasks.workunit.client.1.smithi044.stdout:8/130: dwrite db/d14/f15 [0,4194304] 0 2024-04-22T23:45:59.190 INFO:tasks.workunit.client.1.smithi044.stdout:3/121: creat d9/db/f27 x:0 0 0 2024-04-22T23:45:59.198 INFO:tasks.workunit.client.1.smithi044.stdout:6/126: rmdir d5/dd/d18 39 2024-04-22T23:45:59.199 INFO:tasks.workunit.client.1.smithi044.stdout:8/131: dread db/f2a [0,4194304] 0 2024-04-22T23:45:59.199 INFO:tasks.workunit.client.1.smithi044.stdout:3/122: dread d9/db/d15/f18 [0,4194304] 0 2024-04-22T23:45:59.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/88: mknod d15/d16/c1b 0 2024-04-22T23:45:59.200 INFO:tasks.workunit.client.1.smithi044.stdout:8/132: stat db/d14/f2b 0 2024-04-22T23:45:59.204 INFO:tasks.workunit.client.1.smithi044.stdout:4/111: creat d5/db/dd/d24/f27 x:0 0 0 2024-04-22T23:45:59.220 INFO:tasks.workunit.client.1.smithi044.stdout:9/110: write d2/d6/fa [475451,786] 0 2024-04-22T23:45:59.221 INFO:tasks.workunit.client.1.smithi044.stdout:9/111: fsync d2/dc/f18 0 2024-04-22T23:45:59.226 INFO:tasks.workunit.client.1.smithi044.stdout:5/89: mkdir d15/d1c 0 2024-04-22T23:45:59.226 INFO:tasks.workunit.client.1.smithi044.stdout:5/90: stat f12 0 2024-04-22T23:45:59.226 INFO:tasks.workunit.client.1.smithi044.stdout:8/133: symlink db/d14/d24/l2f 0 2024-04-22T23:45:59.228 INFO:tasks.workunit.client.1.smithi044.stdout:1/136: getdents d1/d8/d24 0 2024-04-22T23:45:59.231 INFO:tasks.workunit.client.1.smithi044.stdout:1/137: read d1/d8/d25/f29 [3150853,83384] 0 2024-04-22T23:45:59.233 INFO:tasks.workunit.client.1.smithi044.stdout:2/101: dwrite f1 [0,4194304] 0 2024-04-22T23:45:59.235 INFO:tasks.workunit.client.1.smithi044.stdout:5/91: dwrite fa [0,4194304] 0 2024-04-22T23:45:59.242 INFO:tasks.workunit.client.1.smithi044.stdout:4/112: dwrite d5/db/f19 [0,4194304] 0 2024-04-22T23:45:59.245 INFO:tasks.workunit.client.1.smithi044.stdout:4/113: write d5/db/dd/f1b [1338384,37825] 0 2024-04-22T23:45:59.247 INFO:tasks.workunit.client.1.smithi044.stdout:4/114: write d5/db/d16/f1c [373804,99366] 0 2024-04-22T23:45:59.250 INFO:tasks.workunit.client.1.smithi044.stdout:9/112: mknod d2/dc/c2a 0 2024-04-22T23:45:59.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/115: dread - d5/db/dd/d24/f27 zero size 2024-04-22T23:45:59.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/116: chown d5/db/dd/lf 1909474 1 2024-04-22T23:45:59.253 INFO:tasks.workunit.client.1.smithi044.stdout:7/127: truncate dd/d1f/d24/f19 1347435 0 2024-04-22T23:45:59.257 INFO:tasks.workunit.client.1.smithi044.stdout:8/134: unlink db/d14/f17 0 2024-04-22T23:45:59.261 INFO:tasks.workunit.client.1.smithi044.stdout:1/138: mknod d1/d8/c2e 0 2024-04-22T23:45:59.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/102: stat d8/f18 0 2024-04-22T23:45:59.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/103: stat f2 0 2024-04-22T23:45:59.277 INFO:tasks.workunit.client.1.smithi044.stdout:4/117: mknod d5/db/d16/d18/d20/c28 0 2024-04-22T23:45:59.280 INFO:tasks.workunit.client.1.smithi044.stdout:4/118: write d5/db/d16/d18/d20/f23 [586919,73240] 0 2024-04-22T23:45:59.285 INFO:tasks.workunit.client.1.smithi044.stdout:6/127: dwrite d5/dd/f27 [4194304,4194304] 0 2024-04-22T23:45:59.289 INFO:tasks.workunit.client.1.smithi044.stdout:1/139: dread d1/d8/f18 [0,4194304] 0 2024-04-22T23:45:59.292 INFO:tasks.workunit.client.1.smithi044.stdout:0/93: dwrite d0/d2/f1e [0,4194304] 0 2024-04-22T23:45:59.293 INFO:tasks.workunit.client.1.smithi044.stdout:1/140: truncate d1/d8/d25/f2c 993405 0 2024-04-22T23:45:59.295 INFO:tasks.workunit.client.1.smithi044.stdout:4/119: creat d5/db/d16/d18/f29 x:0 0 0 2024-04-22T23:45:59.297 INFO:tasks.workunit.client.1.smithi044.stdout:2/104: getdents d8/d19 0 2024-04-22T23:45:59.298 INFO:tasks.workunit.client.1.smithi044.stdout:2/105: chown d8/l12 5 1 2024-04-22T23:45:59.300 INFO:tasks.workunit.client.1.smithi044.stdout:2/106: write f3 [1324199,79559] 0 2024-04-22T23:45:59.308 INFO:tasks.workunit.client.1.smithi044.stdout:0/94: creat d0/d2/dd/f23 x:0 0 0 2024-04-22T23:45:59.311 INFO:tasks.workunit.client.1.smithi044.stdout:4/120: symlink d5/db/d16/l2a 0 2024-04-22T23:45:59.311 INFO:tasks.workunit.client.1.smithi044.stdout:5/92: write f0 [945522,61780] 0 2024-04-22T23:45:59.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/128: creat d5/dd/d1d/d23/f29 x:0 0 0 2024-04-22T23:45:59.322 INFO:tasks.workunit.client.1.smithi044.stdout:7/128: dwrite dd/d1f/d24/f19 [0,4194304] 0 2024-04-22T23:45:59.323 INFO:tasks.workunit.client.1.smithi044.stdout:5/93: dwrite f8 [0,4194304] 0 2024-04-22T23:45:59.324 INFO:tasks.workunit.client.1.smithi044.stdout:0/95: mknod d0/d2/dd/d19/c24 0 2024-04-22T23:45:59.326 INFO:tasks.workunit.client.1.smithi044.stdout:4/121: mknod d5/db/d16/c2b 0 2024-04-22T23:45:59.328 INFO:tasks.workunit.client.1.smithi044.stdout:7/129: readlink l4 0 2024-04-22T23:45:59.329 INFO:tasks.workunit.client.1.smithi044.stdout:7/130: fdatasync fc 0 2024-04-22T23:45:59.336 INFO:tasks.workunit.client.1.smithi044.stdout:1/141: getdents d1/d8 0 2024-04-22T23:45:59.338 INFO:tasks.workunit.client.1.smithi044.stdout:7/131: truncate fa 4656974 0 2024-04-22T23:45:59.339 INFO:tasks.workunit.client.1.smithi044.stdout:4/122: mknod d5/db/d16/d18/d20/d22/c2c 0 2024-04-22T23:45:59.342 INFO:tasks.workunit.client.1.smithi044.stdout:7/132: rmdir dd/d1a 39 2024-04-22T23:45:59.344 INFO:tasks.workunit.client.1.smithi044.stdout:5/94: dwrite f8 [0,4194304] 0 2024-04-22T23:45:59.344 INFO:tasks.workunit.client.1.smithi044.stdout:2/107: dwrite d8/fc [0,4194304] 0 2024-04-22T23:45:59.347 INFO:tasks.workunit.client.1.smithi044.stdout:2/108: chown d8/ff 26527410 1 2024-04-22T23:45:59.348 INFO:tasks.workunit.client.1.smithi044.stdout:2/109: chown d8/ff 23 1 2024-04-22T23:45:59.349 INFO:tasks.workunit.client.1.smithi044.stdout:7/133: symlink dd/l2d 0 2024-04-22T23:45:59.349 INFO:tasks.workunit.client.1.smithi044.stdout:1/142: creat d1/d8/d20/f2f x:0 0 0 2024-04-22T23:45:59.352 INFO:tasks.workunit.client.1.smithi044.stdout:6/129: getdents d5/dd/d1d/d23 0 2024-04-22T23:45:59.353 INFO:tasks.workunit.client.1.smithi044.stdout:5/95: unlink f0 0 2024-04-22T23:45:59.357 INFO:tasks.workunit.client.1.smithi044.stdout:2/110: symlink d8/l1b 0 2024-04-22T23:45:59.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/143: dwrite d1/d8/d25/f2c [0,4194304] 0 2024-04-22T23:45:59.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/144: fdatasync d1/d8/f19 0 2024-04-22T23:45:59.369 INFO:tasks.workunit.client.1.smithi044.stdout:4/123: dwrite d5/db/dd/f17 [0,4194304] 0 2024-04-22T23:45:59.373 INFO:tasks.workunit.client.1.smithi044.stdout:2/111: mkdir d8/d19/d1c 0 2024-04-22T23:45:59.376 INFO:tasks.workunit.client.1.smithi044.stdout:4/124: write d5/db/d16/d18/d20/f23 [1439963,46355] 0 2024-04-22T23:45:59.381 INFO:tasks.workunit.client.1.smithi044.stdout:1/145: rename d1/d8/l15 to d1/d14/l30 0 2024-04-22T23:45:59.385 INFO:tasks.workunit.client.1.smithi044.stdout:7/134: link dd/d1f/d24/l29 dd/d27/l2e 0 2024-04-22T23:45:59.390 INFO:tasks.workunit.client.1.smithi044.stdout:6/130: link d5/dd/f27 d5/dd/d18/f2a 0 2024-04-22T23:45:59.391 INFO:tasks.workunit.client.1.smithi044.stdout:4/125: write d5/db/dd/f17 [4879065,130411] 0 2024-04-22T23:45:59.393 INFO:tasks.workunit.client.1.smithi044.stdout:4/126: fsync d5/db/dd/d24/f27 0 2024-04-22T23:45:59.393 INFO:tasks.workunit.client.1.smithi044.stdout:6/131: write d5/dd/d1d/d23/f26 [1025160,1205] 0 2024-04-22T23:45:59.394 INFO:tasks.workunit.client.1.smithi044.stdout:6/132: fsync d5/f12 0 2024-04-22T23:45:59.397 INFO:tasks.workunit.client.1.smithi044.stdout:6/133: truncate d5/d8/f22 783135 0 2024-04-22T23:45:59.398 INFO:tasks.workunit.client.1.smithi044.stdout:1/146: mknod d1/d14/c31 0 2024-04-22T23:45:59.405 INFO:tasks.workunit.client.1.smithi044.stdout:2/112: creat d8/d19/d1c/f1d x:0 0 0 2024-04-22T23:45:59.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/127: rename d5/db/dd/c13 to d5/db/d16/d18/c2d 0 2024-04-22T23:45:59.417 INFO:tasks.workunit.client.1.smithi044.stdout:4/128: dwrite d5/db/d16/d18/d20/f23 [0,4194304] 0 2024-04-22T23:45:59.424 INFO:tasks.workunit.client.1.smithi044.stdout:1/147: unlink d1/c2d 0 2024-04-22T23:45:59.425 INFO:tasks.workunit.client.1.smithi044.stdout:7/135: link dd/d1f/f21 dd/d1a/f2f 0 2024-04-22T23:45:59.428 INFO:tasks.workunit.client.1.smithi044.stdout:2/113: link f6 d8/d19/f1e 0 2024-04-22T23:45:59.429 INFO:tasks.workunit.client.1.smithi044.stdout:4/129: creat d5/db/f2e x:0 0 0 2024-04-22T23:45:59.430 INFO:tasks.workunit.client.1.smithi044.stdout:4/130: chown d5/db/f19 3186482 1 2024-04-22T23:45:59.432 INFO:tasks.workunit.client.1.smithi044.stdout:1/148: mknod d1/d8/c32 0 2024-04-22T23:45:59.435 INFO:tasks.workunit.client.1.smithi044.stdout:2/114: creat d8/d19/f1f x:0 0 0 2024-04-22T23:45:59.436 INFO:tasks.workunit.client.1.smithi044.stdout:1/149: dread d1/d8/f1e [0,4194304] 0 2024-04-22T23:45:59.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/131: rename d5/db/d16/f1c to d5/db/dd/d24/f2f 0 2024-04-22T23:45:59.442 INFO:tasks.workunit.client.1.smithi044.stdout:1/150: dread d1/d8/f19 [0,4194304] 0 2024-04-22T23:45:59.453 INFO:tasks.workunit.client.1.smithi044.stdout:4/132: dread d5/db/dd/f1d [0,4194304] 0 2024-04-22T23:45:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr fail", "who": "smithi044.iglcti"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr fail", "who": "smithi044.iglcti"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd='[{"prefix": "mgr fail", "who": "smithi044.iglcti"}]': finished 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: mgrmap e26: smithi031.fazdcv(active, starting, since 0.0114032s) 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:45:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:45:59.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:45:59.463 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:45:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:45:59.539 INFO:tasks.workunit.client.1.smithi044.stdout:8/135: sync 2024-04-22T23:45:59.539 INFO:tasks.workunit.client.1.smithi044.stdout:9/113: sync 2024-04-22T23:45:59.542 INFO:tasks.workunit.client.1.smithi044.stdout:8/136: dread db/fd [0,4194304] 0 2024-04-22T23:45:59.569 INFO:tasks.workunit.client.1.smithi044.stdout:1/151: rename d1/l2 to d1/d8/d1c/l33 0 2024-04-22T23:45:59.571 INFO:tasks.workunit.client.1.smithi044.stdout:7/136: write fa [796191,13815] 0 2024-04-22T23:45:59.574 INFO:tasks.workunit.client.1.smithi044.stdout:4/133: unlink d5/db/dd/le 0 2024-04-22T23:45:59.575 INFO:tasks.workunit.client.1.smithi044.stdout:2/115: write f6 [4308582,105181] 0 2024-04-22T23:45:59.575 INFO:tasks.workunit.client.1.smithi044.stdout:4/134: fdatasync d5/db/f19 0 2024-04-22T23:45:59.577 INFO:tasks.workunit.client.1.smithi044.stdout:4/135: dread - d5/db/dd/d24/f27 zero size 2024-04-22T23:45:59.579 INFO:tasks.workunit.client.1.smithi044.stdout:9/114: mknod d2/d9/c2b 0 2024-04-22T23:45:59.582 INFO:tasks.workunit.client.1.smithi044.stdout:1/152: mkdir d1/d8/d34 0 2024-04-22T23:45:59.583 INFO:tasks.workunit.client.1.smithi044.stdout:1/153: fsync d1/f1b 0 2024-04-22T23:45:59.584 INFO:tasks.workunit.client.1.smithi044.stdout:7/137: creat dd/d1f/d24/f30 x:0 0 0 2024-04-22T23:45:59.585 INFO:tasks.workunit.client.1.smithi044.stdout:7/138: write dd/ff [2589768,81647] 0 2024-04-22T23:45:59.590 INFO:tasks.workunit.client.1.smithi044.stdout:2/116: rename f1 to d8/d19/f20 0 2024-04-22T23:45:59.591 INFO:tasks.workunit.client.1.smithi044.stdout:4/136: creat d5/db/d16/d18/f30 x:0 0 0 2024-04-22T23:45:59.592 INFO:tasks.workunit.client.1.smithi044.stdout:4/137: dread - d5/db/dd/d24/f27 zero size 2024-04-22T23:45:59.592 INFO:tasks.workunit.client.1.smithi044.stdout:8/137: fsync db/f12 0 2024-04-22T23:45:59.594 INFO:tasks.workunit.client.1.smithi044.stdout:8/138: chown db/d14/l2d 767 1 2024-04-22T23:45:59.594 INFO:tasks.workunit.client.1.smithi044.stdout:8/139: fsync db/f22 0 2024-04-22T23:45:59.608 INFO:tasks.workunit.client.1.smithi044.stdout:8/140: creat db/d14/d24/f30 x:0 0 0 2024-04-22T23:45:59.608 INFO:tasks.workunit.client.1.smithi044.stdout:4/138: dread d5/db/dd/f10 [0,4194304] 0 2024-04-22T23:45:59.638 INFO:tasks.workunit.client.1.smithi044.stdout:4/139: read - d5/db/f21 zero size 2024-04-22T23:45:59.639 INFO:tasks.workunit.client.1.smithi044.stdout:9/115: rename d2/d6/f13 to d2/f2c 0 2024-04-22T23:45:59.639 INFO:tasks.workunit.client.1.smithi044.stdout:7/139: rmdir dd/d1f 39 2024-04-22T23:45:59.640 INFO:tasks.workunit.client.1.smithi044.stdout:8/141: mknod db/d14/d24/c31 0 2024-04-22T23:45:59.643 INFO:tasks.workunit.client.1.smithi044.stdout:1/154: rename d1/d14/l16 to d1/d8/d34/l35 0 2024-04-22T23:45:59.643 INFO:tasks.workunit.client.1.smithi044.stdout:9/116: fsync d2/f2c 0 2024-04-22T23:45:59.644 INFO:tasks.workunit.client.1.smithi044.stdout:1/155: chown d1/d8/f1e 6 1 2024-04-22T23:45:59.648 INFO:tasks.workunit.client.1.smithi044.stdout:8/142: symlink db/d14/l32 0 2024-04-22T23:45:59.648 INFO:tasks.workunit.client.1.smithi044.stdout:9/117: dread d2/dc/f18 [0,4194304] 0 2024-04-22T23:45:59.649 INFO:tasks.workunit.client.1.smithi044.stdout:9/118: fdatasync d2/f1c 0 2024-04-22T23:45:59.649 INFO:tasks.workunit.client.1.smithi044.stdout:8/143: chown c7 83679 1 2024-04-22T23:45:59.650 INFO:tasks.workunit.client.1.smithi044.stdout:8/144: chown db/d14/c1b 1713 1 2024-04-22T23:45:59.656 INFO:tasks.workunit.client.1.smithi044.stdout:8/145: dwrite f6 [4194304,4194304] 0 2024-04-22T23:45:59.658 INFO:tasks.workunit.client.1.smithi044.stdout:1/156: read d1/d8/d25/f29 [626538,124112] 0 2024-04-22T23:45:59.664 INFO:tasks.workunit.client.1.smithi044.stdout:4/140: truncate d5/db/dd/f17 3101328 0 2024-04-22T23:45:59.669 INFO:tasks.workunit.client.1.smithi044.stdout:8/146: unlink db/d14/c1b 0 2024-04-22T23:45:59.671 INFO:tasks.workunit.client.1.smithi044.stdout:1/157: creat d1/d14/f36 x:0 0 0 2024-04-22T23:45:59.674 INFO:tasks.workunit.client.1.smithi044.stdout:9/119: symlink d2/l2d 0 2024-04-22T23:45:59.675 INFO:tasks.workunit.client.1.smithi044.stdout:9/120: stat d2/d6/fa 0 2024-04-22T23:45:59.677 INFO:tasks.workunit.client.1.smithi044.stdout:7/140: creat dd/d1f/f31 x:0 0 0 2024-04-22T23:45:59.677 INFO:tasks.workunit.client.1.smithi044.stdout:4/141: rmdir d5/db/d16 39 2024-04-22T23:45:59.679 INFO:tasks.workunit.client.1.smithi044.stdout:8/147: symlink db/d14/d24/l33 0 2024-04-22T23:45:59.682 INFO:tasks.workunit.client.1.smithi044.stdout:7/141: creat dd/d27/f32 x:0 0 0 2024-04-22T23:45:59.684 INFO:tasks.workunit.client.1.smithi044.stdout:8/148: truncate db/fd 834442 0 2024-04-22T23:45:59.686 INFO:tasks.workunit.client.1.smithi044.stdout:7/142: dread fc [0,4194304] 0 2024-04-22T23:45:59.692 INFO:tasks.workunit.client.1.smithi044.stdout:8/149: creat db/f34 x:0 0 0 2024-04-22T23:45:59.693 INFO:tasks.workunit.client.1.smithi044.stdout:7/143: dwrite dd/ff [4194304,4194304] 0 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 172.21.15.44:0/191815504' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr fail", "who": "smithi044.iglcti"}]: dispatch 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "mgr fail", "who": "smithi044.iglcti"}]: dispatch 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.24391 ' entity='mgr.smithi044.iglcti' cmd='[{"prefix": "mgr fail", "who": "smithi044.iglcti"}]': finished 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: mgrmap e26: smithi031.fazdcv(active, starting, since 0.0114032s) 2024-04-22T23:45:59.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:45:59.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:45:59.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:45:59.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:59.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:59.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:45:59.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:45:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:45:59.697 INFO:tasks.workunit.client.1.smithi044.stdout:7/144: mknod dd/d1a/c33 0 2024-04-22T23:45:59.699 INFO:tasks.workunit.client.1.smithi044.stdout:8/150: dread f6 [0,4194304] 0 2024-04-22T23:45:59.700 INFO:tasks.workunit.client.1.smithi044.stdout:7/145: unlink dd/d1f/d24/f15 0 2024-04-22T23:45:59.700 INFO:tasks.workunit.client.1.smithi044.stdout:7/146: dread - dd/d1f/f21 zero size 2024-04-22T23:45:59.701 INFO:tasks.workunit.client.1.smithi044.stdout:3/123: sync 2024-04-22T23:45:59.701 INFO:tasks.workunit.client.1.smithi044.stdout:0/96: sync 2024-04-22T23:45:59.701 INFO:tasks.workunit.client.1.smithi044.stdout:8/151: creat db/d14/d24/f35 x:0 0 0 2024-04-22T23:45:59.702 INFO:tasks.workunit.client.1.smithi044.stdout:3/124: readlink l4 0 2024-04-22T23:45:59.702 INFO:tasks.workunit.client.1.smithi044.stdout:3/125: chown f5 15614 1 2024-04-22T23:45:59.703 INFO:tasks.workunit.client.1.smithi044.stdout:8/152: write db/ff [1767575,53599] 0 2024-04-22T23:45:59.709 INFO:tasks.workunit.client.1.smithi044.stdout:8/153: dread db/d14/f15 [0,4194304] 0 2024-04-22T23:45:59.709 INFO:tasks.workunit.client.1.smithi044.stdout:8/154: read db/f2a [3397074,13254] 0 2024-04-22T23:45:59.716 INFO:tasks.workunit.client.1.smithi044.stdout:8/155: creat db/d14/f36 x:0 0 0 2024-04-22T23:45:59.722 INFO:tasks.workunit.client.1.smithi044.stdout:8/156: creat db/d14/f37 x:0 0 0 2024-04-22T23:45:59.723 INFO:tasks.workunit.client.1.smithi044.stdout:7/147: creat dd/d1f/f34 x:0 0 0 2024-04-22T23:45:59.723 INFO:tasks.workunit.client.1.smithi044.stdout:7/148: stat l0 0 2024-04-22T23:45:59.724 INFO:tasks.workunit.client.1.smithi044.stdout:7/149: write dd/fe [455217,7110] 0 2024-04-22T23:45:59.724 INFO:tasks.workunit.client.1.smithi044.stdout:3/126: link d9/c20 d9/c28 0 2024-04-22T23:45:59.726 INFO:tasks.workunit.client.1.smithi044.stdout:0/97: link d0/f8 d0/d2/f25 0 2024-04-22T23:45:59.732 INFO:tasks.workunit.client.1.smithi044.stdout:3/127: mkdir d9/db/d15/d29 0 2024-04-22T23:45:59.738 INFO:tasks.workunit.client.1.smithi044.stdout:0/98: rename d0/d2/f15 to d0/d2/dd/d19/f26 0 2024-04-22T23:45:59.740 INFO:tasks.workunit.client.1.smithi044.stdout:8/157: write db/fe [3244946,20661] 0 2024-04-22T23:45:59.741 INFO:tasks.workunit.client.1.smithi044.stdout:3/128: dwrite d9/db/f27 [0,4194304] 0 2024-04-22T23:45:59.742 INFO:tasks.workunit.client.1.smithi044.stdout:3/129: fdatasync f7 0 2024-04-22T23:45:59.743 INFO:tasks.workunit.client.1.smithi044.stdout:7/150: symlink dd/d1f/l35 0 2024-04-22T23:45:59.750 INFO:tasks.workunit.client.1.smithi044.stdout:3/130: creat d9/f2a x:0 0 0 2024-04-22T23:45:59.754 INFO:tasks.workunit.client.1.smithi044.stdout:3/131: creat d9/d14/f2b x:0 0 0 2024-04-22T23:45:59.758 INFO:tasks.workunit.client.1.smithi044.stdout:8/158: creat db/f38 x:0 0 0 2024-04-22T23:45:59.759 INFO:tasks.workunit.client.1.smithi044.stdout:7/151: creat dd/d1f/f36 x:0 0 0 2024-04-22T23:45:59.760 INFO:tasks.workunit.client.1.smithi044.stdout:0/99: getdents d0/d2/dd/d19 0 2024-04-22T23:45:59.762 INFO:tasks.workunit.client.1.smithi044.stdout:8/159: write f5 [3074286,31333] 0 2024-04-22T23:45:59.764 INFO:tasks.workunit.client.1.smithi044.stdout:3/132: dwrite d9/d12/f1f [0,4194304] 0 2024-04-22T23:45:59.765 INFO:tasks.workunit.client.1.smithi044.stdout:8/160: write db/d14/f37 [582872,42947] 0 2024-04-22T23:45:59.772 INFO:tasks.workunit.client.1.smithi044.stdout:7/152: creat dd/d27/f37 x:0 0 0 2024-04-22T23:45:59.772 INFO:tasks.workunit.client.1.smithi044.stdout:0/100: creat d0/d2/dd/d19/f27 x:0 0 0 2024-04-22T23:45:59.775 INFO:tasks.workunit.client.1.smithi044.stdout:8/161: creat db/d14/d24/f39 x:0 0 0 2024-04-22T23:45:59.775 INFO:tasks.workunit.client.1.smithi044.stdout:8/162: stat db 0 2024-04-22T23:45:59.775 INFO:tasks.workunit.client.1.smithi044.stdout:3/133: dread d9/db/f27 [0,4194304] 0 2024-04-22T23:45:59.777 INFO:tasks.workunit.client.1.smithi044.stdout:3/134: write d9/d12/f1f [2647730,36928] 0 2024-04-22T23:45:59.779 INFO:tasks.workunit.client.1.smithi044.stdout:0/101: mknod d0/d2/c28 0 2024-04-22T23:45:59.782 INFO:tasks.workunit.client.1.smithi044.stdout:8/163: creat db/d14/f3a x:0 0 0 2024-04-22T23:45:59.783 INFO:tasks.workunit.client.1.smithi044.stdout:8/164: write db/d14/f2b [307985,122642] 0 2024-04-22T23:45:59.795 INFO:tasks.workunit.client.1.smithi044.stdout:7/153: creat dd/d1f/f38 x:0 0 0 2024-04-22T23:45:59.796 INFO:tasks.workunit.client.1.smithi044.stdout:3/135: symlink d9/db/d15/d29/l2c 0 2024-04-22T23:45:59.801 INFO:tasks.workunit.client.1.smithi044.stdout:7/154: creat dd/d1f/d24/f39 x:0 0 0 2024-04-22T23:45:59.804 INFO:tasks.workunit.client.1.smithi044.stdout:0/102: write d0/d2/dd/d19/f26 [631803,94412] 0 2024-04-22T23:45:59.804 INFO:tasks.workunit.client.1.smithi044.stdout:7/155: read dd/d1a/f1d [400805,128666] 0 2024-04-22T23:45:59.805 INFO:tasks.workunit.client.1.smithi044.stdout:0/103: stat d0/d2/cc 0 2024-04-22T23:45:59.809 INFO:tasks.workunit.client.1.smithi044.stdout:8/165: getdents db/d14/d24 0 2024-04-22T23:45:59.813 INFO:tasks.workunit.client.1.smithi044.stdout:0/104: unlink d0/d2/dd/d19/f27 0 2024-04-22T23:45:59.815 INFO:tasks.workunit.client.1.smithi044.stdout:3/136: dread d9/db/f13 [0,4194304] 0 2024-04-22T23:45:59.815 INFO:tasks.workunit.client.1.smithi044.stdout:8/166: creat db/d14/d24/f3b x:0 0 0 2024-04-22T23:45:59.821 INFO:tasks.workunit.client.1.smithi044.stdout:0/105: symlink d0/d2/l29 0 2024-04-22T23:45:59.823 INFO:tasks.workunit.client.1.smithi044.stdout:3/137: creat d9/d12/f2d x:0 0 0 2024-04-22T23:45:59.827 INFO:tasks.workunit.client.1.smithi044.stdout:8/167: creat db/f3c x:0 0 0 2024-04-22T23:45:59.828 INFO:tasks.workunit.client.1.smithi044.stdout:7/156: dread f7 [0,4194304] 0 2024-04-22T23:45:59.829 INFO:tasks.workunit.client.1.smithi044.stdout:0/106: dread d0/d2/dd/f12 [0,4194304] 0 2024-04-22T23:45:59.830 INFO:tasks.workunit.client.1.smithi044.stdout:3/138: dread d9/f17 [0,4194304] 0 2024-04-22T23:45:59.830 INFO:tasks.workunit.client.1.smithi044.stdout:0/107: readlink d0/d2/l29 0 2024-04-22T23:45:59.832 INFO:tasks.workunit.client.1.smithi044.stdout:3/139: chown d9/db/d15/d29/l2c 688 1 2024-04-22T23:45:59.833 INFO:tasks.workunit.client.1.smithi044.stdout:3/140: write d9/db/d15/f21 [508259,125734] 0 2024-04-22T23:45:59.833 INFO:tasks.workunit.client.1.smithi044.stdout:8/168: creat db/f3d x:0 0 0 2024-04-22T23:45:59.834 INFO:tasks.workunit.client.1.smithi044.stdout:0/108: creat d0/d2/dd/f2a x:0 0 0 2024-04-22T23:45:59.837 INFO:tasks.workunit.client.1.smithi044.stdout:8/169: chown db/d14 858 1 2024-04-22T23:45:59.838 INFO:tasks.workunit.client.1.smithi044.stdout:3/141: write d9/db/d15/f21 [19520,63020] 0 2024-04-22T23:45:59.838 INFO:tasks.workunit.client.1.smithi044.stdout:7/157: link dd/d1f/d24/f28 dd/f3a 0 2024-04-22T23:45:59.841 INFO:tasks.workunit.client.1.smithi044.stdout:7/158: dread - dd/d1f/d24/f2b zero size 2024-04-22T23:45:59.843 INFO:tasks.workunit.client.1.smithi044.stdout:8/170: creat db/d14/f3e x:0 0 0 2024-04-22T23:45:59.845 INFO:tasks.workunit.client.1.smithi044.stdout:7/159: write dd/d1f/f2c [698840,95202] 0 2024-04-22T23:45:59.847 INFO:tasks.workunit.client.1.smithi044.stdout:0/109: dwrite d0/d2/f1e [0,4194304] 0 2024-04-22T23:45:59.850 INFO:tasks.workunit.client.1.smithi044.stdout:3/142: dwrite d9/f2a [0,4194304] 0 2024-04-22T23:45:59.861 INFO:tasks.workunit.client.1.smithi044.stdout:0/110: rename d0/d2/dd/f1b to d0/d2/dd/f2b 0 2024-04-22T23:45:59.863 INFO:tasks.workunit.client.1.smithi044.stdout:7/160: dwrite dd/d1f/f34 [0,4194304] 0 2024-04-22T23:45:59.866 INFO:tasks.workunit.client.1.smithi044.stdout:3/143: read d9/db/d15/f18 [4107585,34799] 0 2024-04-22T23:45:59.866 INFO:tasks.workunit.client.1.smithi044.stdout:0/111: write d0/f16 [4063055,120868] 0 2024-04-22T23:45:59.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/171: rename db/d14/f3a to db/f3f 0 2024-04-22T23:45:59.880 INFO:tasks.workunit.client.1.smithi044.stdout:7/161: dread f5 [0,4194304] 0 2024-04-22T23:45:59.899 INFO:tasks.workunit.client.1.smithi044.stdout:5/96: sync 2024-04-22T23:45:59.901 INFO:tasks.workunit.client.1.smithi044.stdout:3/144: mkdir d9/db/d15/d2e 0 2024-04-22T23:45:59.905 INFO:tasks.workunit.client.1.smithi044.stdout:0/112: mknod d0/d2/dd/c2c 0 2024-04-22T23:45:59.906 INFO:tasks.workunit.client.1.smithi044.stdout:0/113: write d0/d2/f1e [331709,32204] 0 2024-04-22T23:45:59.915 INFO:tasks.workunit.client.1.smithi044.stdout:0/114: dwrite d0/d2/dd/f13 [0,4194304] 0 2024-04-22T23:45:59.918 INFO:tasks.workunit.client.1.smithi044.stdout:7/162: rename dd/d1f/d24/f2b to dd/d1a/f3b 0 2024-04-22T23:45:59.925 INFO:tasks.workunit.client.1.smithi044.stdout:3/145: rmdir d9/d14 39 2024-04-22T23:45:59.925 INFO:tasks.workunit.client.1.smithi044.stdout:3/146: chown d9/db 10077038 1 2024-04-22T23:45:59.937 INFO:tasks.workunit.client.1.smithi044.stdout:7/163: mkdir dd/d1a/d3c 0 2024-04-22T23:45:59.938 INFO:tasks.workunit.client.1.smithi044.stdout:5/97: mkdir d15/d16/d1d 0 2024-04-22T23:45:59.941 INFO:tasks.workunit.client.1.smithi044.stdout:8/172: truncate db/f2a 615031 0 2024-04-22T23:45:59.942 INFO:tasks.workunit.client.1.smithi044.stdout:3/147: mknod d9/d12/c2f 0 2024-04-22T23:45:59.942 INFO:tasks.workunit.client.1.smithi044.stdout:0/115: write d0/d2/dd/f11 [402026,10413] 0 2024-04-22T23:45:59.943 INFO:tasks.workunit.client.1.smithi044.stdout:0/116: stat d0/d2/l3 0 2024-04-22T23:45:59.945 INFO:tasks.workunit.client.1.smithi044.stdout:7/164: creat dd/d1f/d24/f3d x:0 0 0 2024-04-22T23:45:59.946 INFO:tasks.workunit.client.1.smithi044.stdout:5/98: creat d15/f1e x:0 0 0 2024-04-22T23:45:59.947 INFO:tasks.workunit.client.1.smithi044.stdout:3/148: mkdir d9/db/d30 0 2024-04-22T23:45:59.948 INFO:tasks.workunit.client.1.smithi044.stdout:3/149: stat f5 0 2024-04-22T23:45:59.949 INFO:tasks.workunit.client.1.smithi044.stdout:5/99: mknod d15/d16/c1f 0 2024-04-22T23:45:59.949 INFO:tasks.workunit.client.1.smithi044.stdout:7/165: getdents dd/d1a/d3c 0 2024-04-22T23:45:59.952 INFO:tasks.workunit.client.1.smithi044.stdout:7/166: creat dd/d27/f3e x:0 0 0 2024-04-22T23:45:59.958 INFO:tasks.workunit.client.1.smithi044.stdout:5/100: dwrite f14 [0,4194304] 0 2024-04-22T23:45:59.960 INFO:tasks.workunit.client.1.smithi044.stdout:7/167: dwrite dd/d27/f32 [0,4194304] 0 2024-04-22T23:45:59.961 INFO:tasks.workunit.client.1.smithi044.stdout:7/168: chown fa 6338821 1 2024-04-22T23:45:59.963 INFO:tasks.workunit.client.1.smithi044.stdout:5/101: link f14 d15/f20 0 2024-04-22T23:45:59.963 INFO:tasks.workunit.client.1.smithi044.stdout:7/169: chown dd/d1f/d24/f13 493 1 2024-04-22T23:45:59.972 INFO:tasks.workunit.client.1.smithi044.stdout:5/102: creat d15/d1c/f21 x:0 0 0 2024-04-22T23:45:59.973 INFO:tasks.workunit.client.1.smithi044.stdout:5/103: chown d15/d1c/f21 362 1 2024-04-22T23:45:59.974 INFO:tasks.workunit.client.1.smithi044.stdout:7/170: dwrite dd/d1f/d24/f39 [0,4194304] 0 2024-04-22T23:45:59.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/104: dread f5 [4194304,4194304] 0 2024-04-22T23:45:59.985 INFO:tasks.workunit.client.1.smithi044.stdout:5/105: chown d15 95619 1 2024-04-22T23:45:59.988 INFO:tasks.workunit.client.1.smithi044.stdout:7/171: dwrite dd/fe [0,4194304] 0 2024-04-22T23:45:59.990 INFO:tasks.workunit.client.1.smithi044.stdout:5/106: fdatasync f11 0 2024-04-22T23:45:59.992 INFO:tasks.workunit.client.1.smithi044.stdout:7/172: creat dd/d1f/d24/f3f x:0 0 0 2024-04-22T23:45:59.997 INFO:tasks.workunit.client.1.smithi044.stdout:7/173: dread dd/d27/f32 [0,4194304] 0 2024-04-22T23:46:00.000 INFO:tasks.workunit.client.1.smithi044.stdout:5/107: write fa [2180175,93992] 0 2024-04-22T23:46:00.003 INFO:tasks.workunit.client.1.smithi044.stdout:7/174: dread dd/ff [4194304,4194304] 0 2024-04-22T23:46:00.010 INFO:tasks.workunit.client.1.smithi044.stdout:7/175: symlink dd/d1f/d24/l40 0 2024-04-22T23:46:00.013 INFO:tasks.workunit.client.1.smithi044.stdout:7/176: mknod dd/d1a/c41 0 2024-04-22T23:46:00.017 INFO:tasks.workunit.client.1.smithi044.stdout:7/177: symlink dd/d1f/l42 0 2024-04-22T23:46:00.018 INFO:tasks.workunit.client.1.smithi044.stdout:7/178: write dd/d1f/d24/f1e [907063,10667] 0 2024-04-22T23:46:00.019 INFO:tasks.workunit.client.1.smithi044.stdout:7/179: dread - dd/d1f/f21 zero size 2024-04-22T23:46:00.034 INFO:tasks.workunit.client.1.smithi044.stdout:7/180: dread fa [0,4194304] 0 2024-04-22T23:46:00.035 INFO:tasks.workunit.client.1.smithi044.stdout:7/181: stat dd/d1a/f1d 0 2024-04-22T23:46:00.041 INFO:tasks.workunit.client.1.smithi044.stdout:7/182: dwrite dd/d1f/d24/f30 [0,4194304] 0 2024-04-22T23:46:00.042 INFO:tasks.workunit.client.1.smithi044.stdout:7/183: read - dd/d27/f3e zero size 2024-04-22T23:46:00.050 INFO:tasks.workunit.client.1.smithi044.stdout:7/184: dwrite dd/d1a/f1d [0,4194304] 0 2024-04-22T23:46:00.058 INFO:tasks.workunit.client.1.smithi044.stdout:7/185: dwrite dd/d1f/d24/f19 [0,4194304] 0 2024-04-22T23:46:00.067 INFO:tasks.workunit.client.1.smithi044.stdout:7/186: dread dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:00.082 INFO:tasks.workunit.client.1.smithi044.stdout:2/117: sync 2024-04-22T23:46:00.082 INFO:tasks.workunit.client.1.smithi044.stdout:6/134: sync 2024-04-22T23:46:00.083 INFO:tasks.workunit.client.1.smithi044.stdout:2/118: write f7 [667921,93924] 0 2024-04-22T23:46:00.084 INFO:tasks.workunit.client.1.smithi044.stdout:2/119: write d8/f10 [160073,48671] 0 2024-04-22T23:46:00.200 INFO:tasks.workunit.client.1.smithi044.stdout:2/120: rmdir d8 39 2024-04-22T23:46:00.243 INFO:tasks.workunit.client.1.smithi044.stdout:1/158: sync 2024-04-22T23:46:00.269 INFO:tasks.workunit.client.1.smithi044.stdout:6/135: mknod d5/dd/c2b 0 2024-04-22T23:46:00.272 INFO:tasks.workunit.client.1.smithi044.stdout:2/121: write d8/f10 [782702,108311] 0 2024-04-22T23:46:00.278 INFO:tasks.workunit.client.1.smithi044.stdout:6/136: readlink d5/l1c 0 2024-04-22T23:46:00.281 INFO:tasks.workunit.client.1.smithi044.stdout:2/122: fsync d8/f15 0 2024-04-22T23:46:00.290 INFO:tasks.workunit.client.1.smithi044.stdout:6/137: creat d5/dd/d28/f2c x:0 0 0 2024-04-22T23:46:00.294 INFO:tasks.workunit.client.1.smithi044.stdout:6/138: dread d5/dd/d18/f2a [4194304,4194304] 0 2024-04-22T23:46:00.295 INFO:tasks.workunit.client.1.smithi044.stdout:1/159: getdents d1/d8/d25 0 2024-04-22T23:46:00.297 INFO:tasks.workunit.client.1.smithi044.stdout:6/139: write d5/dd/d18/f24 [231981,91213] 0 2024-04-22T23:46:00.303 INFO:tasks.workunit.client.1.smithi044.stdout:1/160: fdatasync d1/d8/d1c/d23/f27 0 2024-04-22T23:46:00.307 INFO:tasks.workunit.client.1.smithi044.stdout:6/140: symlink d5/dd/d1d/l2d 0 2024-04-22T23:46:00.308 INFO:tasks.workunit.client.1.smithi044.stdout:6/141: dread d5/d8/f22 [0,4194304] 0 2024-04-22T23:46:00.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/161: mknod d1/d8/d25/c37 0 2024-04-22T23:46:00.311 INFO:tasks.workunit.client.1.smithi044.stdout:1/162: write d1/d8/f1a [815569,62422] 0 2024-04-22T23:46:00.312 INFO:tasks.workunit.client.1.smithi044.stdout:1/163: dread - d1/d8/f26 zero size 2024-04-22T23:46:00.315 INFO:tasks.workunit.client.1.smithi044.stdout:6/142: creat d5/dd/d28/f2e x:0 0 0 2024-04-22T23:46:00.325 INFO:tasks.workunit.client.1.smithi044.stdout:1/164: creat d1/d8/d34/f38 x:0 0 0 2024-04-22T23:46:00.326 INFO:tasks.workunit.client.1.smithi044.stdout:1/165: chown d1/d8/f1a 1427 1 2024-04-22T23:46:00.328 INFO:tasks.workunit.client.1.smithi044.stdout:6/143: creat d5/dd/d1d/d23/f2f x:0 0 0 2024-04-22T23:46:00.334 INFO:tasks.workunit.client.1.smithi044.stdout:1/166: dwrite d1/d8/d25/f2a [0,4194304] 0 2024-04-22T23:46:00.336 INFO:tasks.workunit.client.1.smithi044.stdout:1/167: read d1/d8/d25/f2c [2655830,37698] 0 2024-04-22T23:46:00.350 INFO:tasks.workunit.client.1.smithi044.stdout:1/168: creat d1/d8/d25/f39 x:0 0 0 2024-04-22T23:46:00.355 INFO:tasks.workunit.client.1.smithi044.stdout:1/169: fdatasync d1/d8/d1c/d23/f27 0 2024-04-22T23:46:00.365 INFO:tasks.workunit.client.1.smithi044.stdout:1/170: symlink d1/d8/d34/l3a 0 2024-04-22T23:46:00.368 INFO:tasks.workunit.client.1.smithi044.stdout:1/171: creat d1/d14/f3b x:0 0 0 2024-04-22T23:46:00.420 INFO:tasks.workunit.client.1.smithi044.stdout:4/142: sync 2024-04-22T23:46:00.420 INFO:tasks.workunit.client.1.smithi044.stdout:8/173: sync 2024-04-22T23:46:00.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/121: sync 2024-04-22T23:46:00.421 INFO:tasks.workunit.client.1.smithi044.stdout:8/174: chown db/d14/d24/c29 218744 1 2024-04-22T23:46:00.425 INFO:tasks.workunit.client.1.smithi044.stdout:4/143: rename d5/db/dd to d5/d31 0 2024-04-22T23:46:00.427 INFO:tasks.workunit.client.1.smithi044.stdout:8/175: symlink db/d14/d24/l40 0 2024-04-22T23:46:00.431 INFO:tasks.workunit.client.1.smithi044.stdout:9/122: rename d2/d6/c11 to d2/dc/d1d/c2e 0 2024-04-22T23:46:00.432 INFO:tasks.workunit.client.1.smithi044.stdout:9/123: write d2/d5/f1f [680493,3150] 0 2024-04-22T23:46:00.432 INFO:tasks.workunit.client.1.smithi044.stdout:8/176: creat db/d14/f41 x:0 0 0 2024-04-22T23:46:00.433 INFO:tasks.workunit.client.1.smithi044.stdout:8/177: readlink db/d14/l32 0 2024-04-22T23:46:00.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/144: rename f1 to d5/d31/f32 0 2024-04-22T23:46:00.440 INFO:tasks.workunit.client.1.smithi044.stdout:4/145: dread - d5/d31/d24/f27 zero size 2024-04-22T23:46:00.440 INFO:tasks.workunit.client.1.smithi044.stdout:8/178: dwrite db/d14/d24/f26 [0,4194304] 0 2024-04-22T23:46:00.465 INFO:tasks.workunit.client.1.smithi044.stdout:7/187: write f5 [1184400,93962] 0 2024-04-22T23:46:00.468 INFO:tasks.workunit.client.1.smithi044.stdout:7/188: unlink dd/d1f/d24/l23 0 2024-04-22T23:46:00.469 INFO:tasks.workunit.client.1.smithi044.stdout:7/189: stat dd/l10 0 2024-04-22T23:46:00.473 INFO:tasks.workunit.client.1.smithi044.stdout:7/190: symlink dd/l43 0 2024-04-22T23:46:00.473 INFO:tasks.workunit.client.1.smithi044.stdout:7/191: dread - dd/d27/f3e zero size 2024-04-22T23:46:00.477 INFO:tasks.workunit.client.1.smithi044.stdout:0/117: sync 2024-04-22T23:46:00.477 INFO:tasks.workunit.client.1.smithi044.stdout:3/150: sync 2024-04-22T23:46:00.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/151: truncate d9/d14/f2b 495296 0 2024-04-22T23:46:00.480 INFO:tasks.workunit.client.1.smithi044.stdout:7/192: creat dd/f44 x:0 0 0 2024-04-22T23:46:00.481 INFO:tasks.workunit.client.1.smithi044.stdout:0/118: mknod d0/d2/dd/c2d 0 2024-04-22T23:46:00.488 INFO:tasks.workunit.client.1.smithi044.stdout:0/119: dwrite d0/d2/dd/f1d [0,4194304] 0 2024-04-22T23:46:00.488 INFO:tasks.workunit.client.1.smithi044.stdout:0/120: read - d0/f22 zero size 2024-04-22T23:46:00.489 INFO:tasks.workunit.client.1.smithi044.stdout:0/121: readlink d0/d2/l3 0 2024-04-22T23:46:00.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/152: rmdir d9/db/d30 0 2024-04-22T23:46:00.494 INFO:tasks.workunit.client.1.smithi044.stdout:0/122: mkdir d0/d2e 0 2024-04-22T23:46:00.496 INFO:tasks.workunit.client.1.smithi044.stdout:0/123: chown d0/f16 20368441 1 2024-04-22T23:46:00.497 INFO:tasks.workunit.client.1.smithi044.stdout:0/124: truncate d0/d2/dd/f23 232676 0 2024-04-22T23:46:00.503 INFO:tasks.workunit.client.1.smithi044.stdout:0/125: mkdir d0/d2e/d2f 0 2024-04-22T23:46:00.510 INFO:tasks.workunit.client.1.smithi044.stdout:0/126: dwrite d0/d2/fa [0,4194304] 0 2024-04-22T23:46:00.512 INFO:tasks.workunit.client.1.smithi044.stdout:0/127: truncate d0/d2/f6 5185647 0 2024-04-22T23:46:00.517 INFO:tasks.workunit.client.1.smithi044.stdout:5/108: sync 2024-04-22T23:46:00.517 INFO:tasks.workunit.client.1.smithi044.stdout:2/123: sync 2024-04-22T23:46:00.525 INFO:tasks.workunit.client.1.smithi044.stdout:0/128: dread d0/d2/dd/d19/f26 [0,4194304] 0 2024-04-22T23:46:00.529 INFO:tasks.workunit.client.1.smithi044.stdout:2/124: unlink d8/f14 0 2024-04-22T23:46:00.530 INFO:tasks.workunit.client.1.smithi044.stdout:2/125: chown d8/fc 31922729 1 2024-04-22T23:46:00.534 INFO:tasks.workunit.client.1.smithi044.stdout:2/126: symlink d8/d19/d1c/l21 0 2024-04-22T23:46:00.534 INFO:tasks.workunit.client.1.smithi044.stdout:0/129: dwrite d0/d2/dd/f1a [0,4194304] 0 2024-04-22T23:46:00.536 INFO:tasks.workunit.client.1.smithi044.stdout:0/130: chown d0/d2/dd/f11 2354 1 2024-04-22T23:46:00.537 INFO:tasks.workunit.client.1.smithi044.stdout:2/127: fsync d8/ff 0 2024-04-22T23:46:00.545 INFO:tasks.workunit.client.1.smithi044.stdout:0/131: mknod d0/c30 0 2024-04-22T23:46:00.547 INFO:tasks.workunit.client.1.smithi044.stdout:2/128: symlink d8/d19/l22 0 2024-04-22T23:46:00.559 INFO:tasks.workunit.client.1.smithi044.stdout:1/172: write d1/d8/d25/f2a [4373570,81887] 0 2024-04-22T23:46:00.560 INFO:tasks.workunit.client.1.smithi044.stdout:2/129: link d8/f17 d8/d19/f23 0 2024-04-22T23:46:00.565 INFO:tasks.workunit.client.1.smithi044.stdout:1/173: unlink d1/d8/d34/f38 0 2024-04-22T23:46:00.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/130: fsync d8/fe 0 2024-04-22T23:46:00.571 INFO:tasks.workunit.client.1.smithi044.stdout:1/174: creat d1/d8/d34/f3c x:0 0 0 2024-04-22T23:46:00.578 INFO:tasks.workunit.client.1.smithi044.stdout:1/175: creat d1/d8/d25/f3d x:0 0 0 2024-04-22T23:46:00.595 INFO:tasks.workunit.client.1.smithi044.stdout:1/176: mkdir d1/d8/d1c/d23/d3e 0 2024-04-22T23:46:00.595 INFO:tasks.workunit.client.1.smithi044.stdout:1/177: creat d1/d8/d25/f3f x:0 0 0 2024-04-22T23:46:00.634 INFO:tasks.workunit.client.1.smithi044.stdout:9/124: rename d2/l2d to d2/d6/l2f 0 2024-04-22T23:46:00.634 INFO:tasks.workunit.client.1.smithi044.stdout:9/125: chown d2/l24 130 1 2024-04-22T23:46:00.635 INFO:tasks.workunit.client.1.smithi044.stdout:6/144: mkdir d5/d30 0 2024-04-22T23:46:00.636 INFO:tasks.workunit.client.1.smithi044.stdout:6/145: write d5/dd/fe [996894,114714] 0 2024-04-22T23:46:00.638 INFO:tasks.workunit.client.1.smithi044.stdout:8/179: rename db/f19 to db/d14/f42 0 2024-04-22T23:46:00.639 INFO:tasks.workunit.client.1.smithi044.stdout:9/126: creat d2/d5/f30 x:0 0 0 2024-04-22T23:46:00.711 INFO:tasks.workunit.client.1.smithi044.stdout:4/146: rename d5/db/d16/c25 to d5/db/d16/d18/d20/d22/c33 0 2024-04-22T23:46:00.711 INFO:tasks.workunit.client.1.smithi044.stdout:8/180: link db/f1c db/d14/d24/f43 0 2024-04-22T23:46:00.713 INFO:tasks.workunit.client.1.smithi044.stdout:7/193: write f8 [3240719,50496] 0 2024-04-22T23:46:00.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/153: write d9/f19 [3871822,6708] 0 2024-04-22T23:46:00.719 INFO:tasks.workunit.client.1.smithi044.stdout:5/109: rename d15/d1c to d15/d16/d1d/d22 0 2024-04-22T23:46:00.720 INFO:tasks.workunit.client.1.smithi044.stdout:9/127: rename d2/c28 to d2/c31 0 2024-04-22T23:46:00.725 INFO:tasks.workunit.client.1.smithi044.stdout:6/146: getdents d5/dd 0 2024-04-22T23:46:00.726 INFO:tasks.workunit.client.1.smithi044.stdout:5/110: truncate d15/f1e 830756 0 2024-04-22T23:46:00.727 INFO:tasks.workunit.client.1.smithi044.stdout:6/147: chown d5/d30 171443957 1 2024-04-22T23:46:00.729 INFO:tasks.workunit.client.1.smithi044.stdout:0/132: dwrite d0/d2/dd/f12 [0,4194304] 0 2024-04-22T23:46:00.730 INFO:tasks.workunit.client.1.smithi044.stdout:5/111: dread - d15/f18 zero size 2024-04-22T23:46:00.731 INFO:tasks.workunit.client.1.smithi044.stdout:2/131: dwrite d8/d19/f23 [0,4194304] 0 2024-04-22T23:46:00.732 INFO:tasks.workunit.client.1.smithi044.stdout:6/148: read - d5/dd/d1d/d23/f2f zero size 2024-04-22T23:46:00.733 INFO:tasks.workunit.client.1.smithi044.stdout:7/194: dread dd/d1f/d24/f30 [0,4194304] 0 2024-04-22T23:46:00.733 INFO:tasks.workunit.client.1.smithi044.stdout:4/147: dread d5/db/f19 [0,4194304] 0 2024-04-22T23:46:00.735 INFO:tasks.workunit.client.1.smithi044.stdout:4/148: stat d5/d31 0 2024-04-22T23:46:00.737 INFO:tasks.workunit.client.1.smithi044.stdout:8/181: creat db/f44 x:0 0 0 2024-04-22T23:46:00.741 INFO:tasks.workunit.client.1.smithi044.stdout:3/154: link d9/db/f13 d9/db/f31 0 2024-04-22T23:46:00.742 INFO:tasks.workunit.client.1.smithi044.stdout:3/155: write d9/f23 [489041,35250] 0 2024-04-22T23:46:00.743 INFO:tasks.workunit.client.1.smithi044.stdout:4/149: dwrite d5/db/f2e [0,4194304] 0 2024-04-22T23:46:00.745 INFO:tasks.workunit.client.1.smithi044.stdout:3/156: truncate d9/db/d15/f21 1491256 0 2024-04-22T23:46:00.746 INFO:tasks.workunit.client.1.smithi044.stdout:4/150: write d5/db/d16/d18/f29 [251242,25930] 0 2024-04-22T23:46:00.748 INFO:tasks.workunit.client.1.smithi044.stdout:0/133: mknod d0/c31 0 2024-04-22T23:46:00.750 INFO:tasks.workunit.client.1.smithi044.stdout:2/132: creat d8/d19/d1c/f24 x:0 0 0 2024-04-22T23:46:00.753 INFO:tasks.workunit.client.1.smithi044.stdout:2/133: write d8/f9 [626818,123125] 0 2024-04-22T23:46:00.758 INFO:tasks.workunit.client.1.smithi044.stdout:7/195: rmdir dd/d1f 39 2024-04-22T23:46:00.764 INFO:tasks.workunit.client.1.smithi044.stdout:5/112: creat d15/d16/d1d/d22/f23 x:0 0 0 2024-04-22T23:46:00.765 INFO:tasks.workunit.client.1.smithi044.stdout:8/182: symlink db/d14/d24/l45 0 2024-04-22T23:46:00.769 INFO:tasks.workunit.client.1.smithi044.stdout:3/157: rmdir d9/db/d15 39 2024-04-22T23:46:00.776 INFO:tasks.workunit.client.1.smithi044.stdout:2/134: rename d8/f18 to d8/d19/f25 0 2024-04-22T23:46:00.777 INFO:tasks.workunit.client.1.smithi044.stdout:4/151: dwrite d5/d31/f1d [0,4194304] 0 2024-04-22T23:46:00.778 INFO:tasks.workunit.client.1.smithi044.stdout:2/135: chown d8/d19/d1c/l21 643407731 1 2024-04-22T23:46:00.781 INFO:tasks.workunit.client.1.smithi044.stdout:2/136: write d8/d19/f1f [432541,83188] 0 2024-04-22T23:46:00.782 INFO:tasks.workunit.client.1.smithi044.stdout:4/152: write d5/db/d16/d18/f29 [1153842,21779] 0 2024-04-22T23:46:00.785 INFO:tasks.workunit.client.1.smithi044.stdout:8/183: creat db/d14/f46 x:0 0 0 2024-04-22T23:46:00.788 INFO:tasks.workunit.client.1.smithi044.stdout:3/158: unlink d9/db/f27 0 2024-04-22T23:46:00.789 INFO:tasks.workunit.client.1.smithi044.stdout:8/184: chown db/f2c 5068781 1 2024-04-22T23:46:00.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/149: dwrite d5/d8/f22 [0,4194304] 0 2024-04-22T23:46:00.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:00 smithi031 ceph-mon[26721]: mgrmap e27: smithi031.fazdcv(active, since 1.01122s) 2024-04-22T23:46:00.796 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:00 smithi031 ceph-mon[26721]: [22/Apr/2024:23:45:59] ENGINE Bus STARTING 2024-04-22T23:46:00.796 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:00 smithi031 ceph-mon[26721]: [22/Apr/2024:23:45:59] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:46:00.796 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:00 smithi031 ceph-mon[26721]: [22/Apr/2024:23:46:00] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:46:00.796 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:00 smithi031 ceph-mon[26721]: [22/Apr/2024:23:46:00] ENGINE Bus STARTED 2024-04-22T23:46:00.798 INFO:tasks.workunit.client.1.smithi044.stdout:5/113: dread f8 [0,4194304] 0 2024-04-22T23:46:00.798 INFO:tasks.workunit.client.1.smithi044.stdout:4/153: dread d5/d31/f1d [0,4194304] 0 2024-04-22T23:46:00.800 INFO:tasks.workunit.client.1.smithi044.stdout:2/137: dwrite d8/d19/d1c/f24 [0,4194304] 0 2024-04-22T23:46:00.802 INFO:tasks.workunit.client.1.smithi044.stdout:5/114: write d15/d16/d1d/d22/f21 [653939,69186] 0 2024-04-22T23:46:00.802 INFO:tasks.workunit.client.1.smithi044.stdout:2/138: fdatasync f6 0 2024-04-22T23:46:00.808 INFO:tasks.workunit.client.1.smithi044.stdout:6/150: rename d5/dd/fe to d5/d30/f31 0 2024-04-22T23:46:00.812 INFO:tasks.workunit.client.1.smithi044.stdout:0/134: link d0/d2/f25 d0/d2e/f32 0 2024-04-22T23:46:00.813 INFO:tasks.workunit.client.1.smithi044.stdout:0/135: chown d0/d2/cc 498 1 2024-04-22T23:46:00.816 INFO:tasks.workunit.client.1.smithi044.stdout:5/115: mknod d15/c24 0 2024-04-22T23:46:00.819 INFO:tasks.workunit.client.1.smithi044.stdout:8/185: mknod db/c47 0 2024-04-22T23:46:00.822 INFO:tasks.workunit.client.1.smithi044.stdout:7/196: truncate dd/d1f/f2c 399807 0 2024-04-22T23:46:00.824 INFO:tasks.workunit.client.1.smithi044.stdout:3/159: creat d9/db/d15/d2e/f32 x:0 0 0 2024-04-22T23:46:00.825 INFO:tasks.workunit.client.1.smithi044.stdout:3/160: chown f8 24885150 1 2024-04-22T23:46:00.826 INFO:tasks.workunit.client.1.smithi044.stdout:0/136: mkdir d0/d2e/d33 0 2024-04-22T23:46:00.829 INFO:tasks.workunit.client.1.smithi044.stdout:2/139: link d8/f10 d8/f26 0 2024-04-22T23:46:00.832 INFO:tasks.workunit.client.1.smithi044.stdout:6/151: mknod d5/c32 0 2024-04-22T23:46:00.837 INFO:tasks.workunit.client.1.smithi044.stdout:2/140: mkdir d8/d27 0 2024-04-22T23:46:00.840 INFO:tasks.workunit.client.1.smithi044.stdout:4/154: getdents d5/d31/d24 0 2024-04-22T23:46:00.843 INFO:tasks.workunit.client.1.smithi044.stdout:5/116: link f5 d15/d16/d1d/d22/f25 0 2024-04-22T23:46:00.844 INFO:tasks.workunit.client.1.smithi044.stdout:8/186: creat db/f48 x:0 0 0 2024-04-22T23:46:00.848 INFO:tasks.workunit.client.1.smithi044.stdout:4/155: write d5/d31/f32 [5352683,70663] 0 2024-04-22T23:46:00.853 INFO:tasks.workunit.client.1.smithi044.stdout:8/187: creat db/d14/d24/f49 x:0 0 0 2024-04-22T23:46:00.854 INFO:tasks.workunit.client.1.smithi044.stdout:2/141: creat d8/d27/f28 x:0 0 0 2024-04-22T23:46:00.857 INFO:tasks.workunit.client.1.smithi044.stdout:3/161: getdents d9/d14 0 2024-04-22T23:46:00.858 INFO:tasks.workunit.client.1.smithi044.stdout:4/156: rmdir d5/db/d16/d18/d20/d22 39 2024-04-22T23:46:00.859 INFO:tasks.workunit.client.1.smithi044.stdout:4/157: read d5/d31/f10 [2102696,61533] 0 2024-04-22T23:46:00.860 INFO:tasks.workunit.client.1.smithi044.stdout:0/137: write d0/f8 [3624256,57018] 0 2024-04-22T23:46:00.862 INFO:tasks.workunit.client.1.smithi044.stdout:5/117: rename d15/d16/d1d/d22/f25 to d15/d16/d1d/f26 0 2024-04-22T23:46:00.866 INFO:tasks.workunit.client.1.smithi044.stdout:2/142: unlink d8/d19/f23 0 2024-04-22T23:46:00.868 INFO:tasks.workunit.client.1.smithi044.stdout:2/143: fdatasync f3 0 2024-04-22T23:46:00.870 INFO:tasks.workunit.client.1.smithi044.stdout:7/197: dwrite dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:00.873 INFO:tasks.workunit.client.1.smithi044.stdout:8/188: dwrite f6 [0,4194304] 0 2024-04-22T23:46:00.875 INFO:tasks.workunit.client.1.smithi044.stdout:0/138: mknod d0/d2/dd/d19/c34 0 2024-04-22T23:46:00.876 INFO:tasks.workunit.client.1.smithi044.stdout:6/152: truncate d5/dd/f1b 543451 0 2024-04-22T23:46:00.883 INFO:tasks.workunit.client.1.smithi044.stdout:3/162: mkdir d9/db/d33 0 2024-04-22T23:46:00.885 INFO:tasks.workunit.client.1.smithi044.stdout:4/158: mkdir d5/db/d34 0 2024-04-22T23:46:00.886 INFO:tasks.workunit.client.1.smithi044.stdout:7/198: symlink dd/d1a/l45 0 2024-04-22T23:46:00.888 INFO:tasks.workunit.client.1.smithi044.stdout:5/118: dread fe [0,4194304] 0 2024-04-22T23:46:00.890 INFO:tasks.workunit.client.1.smithi044.stdout:0/139: mknod d0/d2/dd/c35 0 2024-04-22T23:46:00.894 INFO:tasks.workunit.client.1.smithi044.stdout:0/140: write d0/d2/dd/f1d [2656518,87454] 0 2024-04-22T23:46:00.895 INFO:tasks.workunit.client.1.smithi044.stdout:8/189: symlink db/l4a 0 2024-04-22T23:46:00.897 INFO:tasks.workunit.client.1.smithi044.stdout:4/159: dwrite d5/db/f19 [0,4194304] 0 2024-04-22T23:46:00.900 INFO:tasks.workunit.client.1.smithi044.stdout:6/153: symlink d5/dd/d1d/l33 0 2024-04-22T23:46:00.900 INFO:tasks.workunit.client.1.smithi044.stdout:4/160: chown d5/db/d16/d18 16940 1 2024-04-22T23:46:00.900 INFO:tasks.workunit.client.1.smithi044.stdout:7/199: dwrite dd/d1f/f31 [0,4194304] 0 2024-04-22T23:46:00.904 INFO:tasks.workunit.client.1.smithi044.stdout:4/161: write d5/db/d16/d18/f30 [825322,34759] 0 2024-04-22T23:46:00.905 INFO:tasks.workunit.client.1.smithi044.stdout:3/163: getdents d9/db/d33 0 2024-04-22T23:46:00.906 INFO:tasks.workunit.client.1.smithi044.stdout:9/128: sync 2024-04-22T23:46:00.906 INFO:tasks.workunit.client.1.smithi044.stdout:1/178: sync 2024-04-22T23:46:00.908 INFO:tasks.workunit.client.1.smithi044.stdout:9/129: chown d2/f3 1543 1 2024-04-22T23:46:00.909 INFO:tasks.workunit.client.1.smithi044.stdout:0/141: dwrite d0/d2/dd/f13 [0,4194304] 0 2024-04-22T23:46:00.910 INFO:tasks.workunit.client.1.smithi044.stdout:3/164: dread - d9/db/f1a zero size 2024-04-22T23:46:00.912 INFO:tasks.workunit.client.1.smithi044.stdout:3/165: readlink d9/db/d15/d25/l26 0 2024-04-22T23:46:00.921 INFO:tasks.workunit.client.1.smithi044.stdout:7/200: creat dd/d27/f46 x:0 0 0 2024-04-22T23:46:00.924 INFO:tasks.workunit.client.1.smithi044.stdout:2/144: getdents d8 0 2024-04-22T23:46:00.931 INFO:tasks.workunit.client.1.smithi044.stdout:0/142: dwrite d0/d2/dd/f10 [0,4194304] 0 2024-04-22T23:46:00.931 INFO:tasks.workunit.client.1.smithi044.stdout:9/130: dwrite d2/d5/f1f [0,4194304] 0 2024-04-22T23:46:00.934 INFO:tasks.workunit.client.1.smithi044.stdout:0/143: chown d0/d2/fa 2 1 2024-04-22T23:46:00.937 INFO:tasks.workunit.client.1.smithi044.stdout:2/145: dread d8/d19/d1c/f24 [0,4194304] 0 2024-04-22T23:46:00.945 INFO:tasks.workunit.client.1.smithi044.stdout:4/162: creat d5/db/d16/d18/d20/d22/f35 x:0 0 0 2024-04-22T23:46:00.946 INFO:tasks.workunit.client.1.smithi044.stdout:5/119: dwrite d15/d16/d1d/f26 [0,4194304] 0 2024-04-22T23:46:00.949 INFO:tasks.workunit.client.1.smithi044.stdout:4/163: fdatasync d5/d31/f11 0 2024-04-22T23:46:00.950 INFO:tasks.workunit.client.1.smithi044.stdout:9/131: stat d2/d6/l29 0 2024-04-22T23:46:00.951 INFO:tasks.workunit.client.1.smithi044.stdout:2/146: dwrite f7 [0,4194304] 0 2024-04-22T23:46:00.956 INFO:tasks.workunit.client.1.smithi044.stdout:1/179: dwrite d1/d8/f12 [0,4194304] 0 2024-04-22T23:46:00.957 INFO:tasks.workunit.client.1.smithi044.stdout:8/190: creat db/f4b x:0 0 0 2024-04-22T23:46:00.959 INFO:tasks.workunit.client.1.smithi044.stdout:9/132: truncate d2/f2c 4956537 0 2024-04-22T23:46:00.959 INFO:tasks.workunit.client.1.smithi044.stdout:7/201: mknod dd/d1f/c47 0 2024-04-22T23:46:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:00 smithi044 ceph-mon[34463]: mgrmap e27: smithi031.fazdcv(active, since 1.01122s) 2024-04-22T23:46:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:00 smithi044 ceph-mon[34463]: [22/Apr/2024:23:45:59] ENGINE Bus STARTING 2024-04-22T23:46:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:00 smithi044 ceph-mon[34463]: [22/Apr/2024:23:45:59] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:46:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:00 smithi044 ceph-mon[34463]: [22/Apr/2024:23:46:00] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:46:00.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:00 smithi044 ceph-mon[34463]: [22/Apr/2024:23:46:00] ENGINE Bus STARTED 2024-04-22T23:46:00.961 INFO:tasks.workunit.client.1.smithi044.stdout:7/202: fsync dd/d1f/f36 0 2024-04-22T23:46:00.961 INFO:tasks.workunit.client.1.smithi044.stdout:2/147: unlink f2 0 2024-04-22T23:46:00.964 INFO:tasks.workunit.client.1.smithi044.stdout:3/166: rename d9/db/f31 to d9/f34 0 2024-04-22T23:46:00.966 INFO:tasks.workunit.client.1.smithi044.stdout:8/191: dread db/d14/f37 [0,4194304] 0 2024-04-22T23:46:00.968 INFO:tasks.workunit.client.1.smithi044.stdout:8/192: chown c7 3 1 2024-04-22T23:46:00.968 INFO:tasks.workunit.client.1.smithi044.stdout:4/164: mknod d5/db/c36 0 2024-04-22T23:46:00.969 INFO:tasks.workunit.client.1.smithi044.stdout:8/193: chown db/ff 435951450 1 2024-04-22T23:46:00.970 INFO:tasks.workunit.client.1.smithi044.stdout:5/120: dwrite d15/f20 [0,4194304] 0 2024-04-22T23:46:00.971 INFO:tasks.workunit.client.1.smithi044.stdout:2/148: creat d8/d19/f29 x:0 0 0 2024-04-22T23:46:00.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/194: chown db/d14/l32 1527335 1 2024-04-22T23:46:00.973 INFO:tasks.workunit.client.1.smithi044.stdout:7/203: unlink dd/d1f/f36 0 2024-04-22T23:46:00.973 INFO:tasks.workunit.client.1.smithi044.stdout:4/165: read d5/d31/f32 [2569019,27819] 0 2024-04-22T23:46:00.975 INFO:tasks.workunit.client.1.smithi044.stdout:1/180: getdents d1/d8/d1c/d23/d3e 0 2024-04-22T23:46:00.976 INFO:tasks.workunit.client.1.smithi044.stdout:6/154: rename d5/dd/d1d/d23/f2f to d5/dd/d28/f34 0 2024-04-22T23:46:00.978 INFO:tasks.workunit.client.1.smithi044.stdout:1/181: dread - d1/d8/f26 zero size 2024-04-22T23:46:00.984 INFO:tasks.workunit.client.1.smithi044.stdout:5/121: dwrite f14 [0,4194304] 0 2024-04-22T23:46:00.986 INFO:tasks.workunit.client.1.smithi044.stdout:2/149: write d8/d19/d1c/f24 [3709682,61958] 0 2024-04-22T23:46:00.990 INFO:tasks.workunit.client.1.smithi044.stdout:4/166: dwrite d5/d31/f32 [4194304,4194304] 0 2024-04-22T23:46:00.991 INFO:tasks.workunit.client.1.smithi044.stdout:2/150: dread - d8/d19/d1c/f1d zero size 2024-04-22T23:46:00.997 INFO:tasks.workunit.client.1.smithi044.stdout:7/204: dwrite dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:00.999 INFO:tasks.workunit.client.1.smithi044.stdout:2/151: write d8/d19/f29 [270530,407] 0 2024-04-22T23:46:00.999 INFO:tasks.workunit.client.1.smithi044.stdout:7/205: read - dd/d1f/d24/f3f zero size 2024-04-22T23:46:01.004 INFO:tasks.workunit.client.1.smithi044.stdout:7/206: dread - dd/d27/f46 zero size 2024-04-22T23:46:01.007 INFO:tasks.workunit.client.1.smithi044.stdout:1/182: creat d1/d8/d34/f40 x:0 0 0 2024-04-22T23:46:01.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/183: fsync d1/f7 0 2024-04-22T23:46:01.009 INFO:tasks.workunit.client.1.smithi044.stdout:8/195: symlink db/l4c 0 2024-04-22T23:46:01.010 INFO:tasks.workunit.client.1.smithi044.stdout:5/122: mkdir d15/d16/d1d/d22/d27 0 2024-04-22T23:46:01.013 INFO:tasks.workunit.client.1.smithi044.stdout:6/155: rmdir d5/dd/d28 39 2024-04-22T23:46:01.015 INFO:tasks.workunit.client.1.smithi044.stdout:4/167: creat d5/db/d16/d18/d20/d22/f37 x:0 0 0 2024-04-22T23:46:01.017 INFO:tasks.workunit.client.1.smithi044.stdout:4/168: write d5/db/f21 [908008,89064] 0 2024-04-22T23:46:01.019 INFO:tasks.workunit.client.1.smithi044.stdout:7/207: creat dd/d27/f48 x:0 0 0 2024-04-22T23:46:01.020 INFO:tasks.workunit.client.1.smithi044.stdout:7/208: fdatasync fc 0 2024-04-22T23:46:01.021 INFO:tasks.workunit.client.1.smithi044.stdout:1/184: symlink d1/d8/d34/l41 0 2024-04-22T23:46:01.024 INFO:tasks.workunit.client.1.smithi044.stdout:1/185: write d1/d8/f26 [761235,24263] 0 2024-04-22T23:46:01.025 INFO:tasks.workunit.client.1.smithi044.stdout:2/152: dwrite d8/d19/f25 [0,4194304] 0 2024-04-22T23:46:01.026 INFO:tasks.workunit.client.1.smithi044.stdout:1/186: read d1/d8/f19 [1079456,53488] 0 2024-04-22T23:46:01.026 INFO:tasks.workunit.client.1.smithi044.stdout:5/123: unlink d15/f20 0 2024-04-22T23:46:01.028 INFO:tasks.workunit.client.1.smithi044.stdout:2/153: chown d8/d19/f25 1139 1 2024-04-22T23:46:01.028 INFO:tasks.workunit.client.1.smithi044.stdout:5/124: dread - d15/d16/d1d/d22/f23 zero size 2024-04-22T23:46:01.032 INFO:tasks.workunit.client.1.smithi044.stdout:5/125: write d15/d16/d1d/d22/f21 [231568,76425] 0 2024-04-22T23:46:01.038 INFO:tasks.workunit.client.1.smithi044.stdout:2/154: dwrite f7 [0,4194304] 0 2024-04-22T23:46:01.038 INFO:tasks.workunit.client.1.smithi044.stdout:6/156: dread d5/dd/f27 [4194304,4194304] 0 2024-04-22T23:46:01.041 INFO:tasks.workunit.client.1.smithi044.stdout:6/157: truncate d5/dd/d1d/f25 474337 0 2024-04-22T23:46:01.042 INFO:tasks.workunit.client.1.smithi044.stdout:7/209: rename dd/fe to dd/d1a/d3c/f49 0 2024-04-22T23:46:01.042 INFO:tasks.workunit.client.1.smithi044.stdout:7/210: dread - dd/f44 zero size 2024-04-22T23:46:01.044 INFO:tasks.workunit.client.1.smithi044.stdout:8/196: symlink db/l4d 0 2024-04-22T23:46:01.051 INFO:tasks.workunit.client.1.smithi044.stdout:1/187: rmdir d1/d8 39 2024-04-22T23:46:01.052 INFO:tasks.workunit.client.1.smithi044.stdout:1/188: fsync d1/d14/f3b 0 2024-04-22T23:46:01.064 INFO:tasks.workunit.client.1.smithi044.stdout:2/155: creat d8/d27/f2a x:0 0 0 2024-04-22T23:46:01.070 INFO:tasks.workunit.client.1.smithi044.stdout:5/126: rename d15/d16/d1d/d22/f21 to d15/d16/f28 0 2024-04-22T23:46:01.073 INFO:tasks.workunit.client.1.smithi044.stdout:1/189: fsync d1/d8/f12 0 2024-04-22T23:46:01.082 INFO:tasks.workunit.client.1.smithi044.stdout:4/169: dwrite d5/d31/f17 [0,4194304] 0 2024-04-22T23:46:01.083 INFO:tasks.workunit.client.1.smithi044.stdout:1/190: creat d1/d8/d1c/d23/f42 x:0 0 0 2024-04-22T23:46:01.092 INFO:tasks.workunit.client.1.smithi044.stdout:4/170: mkdir d5/d31/d24/d38 0 2024-04-22T23:46:01.093 INFO:tasks.workunit.client.1.smithi044.stdout:4/171: write d5/d31/f11 [1385232,91582] 0 2024-04-22T23:46:01.094 INFO:tasks.workunit.client.1.smithi044.stdout:1/191: creat d1/d8/d34/f43 x:0 0 0 2024-04-22T23:46:01.095 INFO:tasks.workunit.client.1.smithi044.stdout:4/172: dread - d5/db/d16/d18/d20/d22/f35 zero size 2024-04-22T23:46:01.096 INFO:tasks.workunit.client.1.smithi044.stdout:4/173: stat d5/d31/d24/f27 0 2024-04-22T23:46:01.096 INFO:tasks.workunit.client.1.smithi044.stdout:6/158: getdents d5/d30 0 2024-04-22T23:46:01.101 INFO:tasks.workunit.client.1.smithi044.stdout:7/211: rename dd/d1f/f21 to dd/f4a 0 2024-04-22T23:46:01.102 INFO:tasks.workunit.client.1.smithi044.stdout:7/212: write dd/d1a/f1d [2369349,113044] 0 2024-04-22T23:46:01.104 INFO:tasks.workunit.client.1.smithi044.stdout:1/192: creat d1/d8/d20/f44 x:0 0 0 2024-04-22T23:46:01.107 INFO:tasks.workunit.client.1.smithi044.stdout:6/159: chown d5/dd/d18/f2a 1005926185 1 2024-04-22T23:46:01.111 INFO:tasks.workunit.client.1.smithi044.stdout:5/127: getdents d15/d16/d1d 0 2024-04-22T23:46:01.112 INFO:tasks.workunit.client.1.smithi044.stdout:4/174: dwrite d5/d31/d24/f2f [0,4194304] 0 2024-04-22T23:46:01.113 INFO:tasks.workunit.client.1.smithi044.stdout:4/175: chown d5/db/d34 0 1 2024-04-22T23:46:01.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/197: rename db/fe to db/d14/d24/f4e 0 2024-04-22T23:46:01.118 INFO:tasks.workunit.client.1.smithi044.stdout:6/160: creat d5/d8/f35 x:0 0 0 2024-04-22T23:46:01.123 INFO:tasks.workunit.client.1.smithi044.stdout:7/213: mknod dd/d1f/c4b 0 2024-04-22T23:46:01.127 INFO:tasks.workunit.client.1.smithi044.stdout:4/176: chown d5/db/d16/d18/c2d 153 1 2024-04-22T23:46:01.132 INFO:tasks.workunit.client.1.smithi044.stdout:2/156: rename d8/f9 to d8/d19/f2b 0 2024-04-22T23:46:01.133 INFO:tasks.workunit.client.1.smithi044.stdout:2/157: readlink d8/d19/l22 0 2024-04-22T23:46:01.145 INFO:tasks.workunit.client.1.smithi044.stdout:1/193: rename d1/d8/d25 to d1/d8/d20/d45 0 2024-04-22T23:46:01.147 INFO:tasks.workunit.client.1.smithi044.stdout:2/158: creat d8/d27/f2c x:0 0 0 2024-04-22T23:46:01.150 INFO:tasks.workunit.client.1.smithi044.stdout:5/128: link d15/d16/c1f d15/c29 0 2024-04-22T23:46:01.150 INFO:tasks.workunit.client.1.smithi044.stdout:5/129: read - d15/d16/d1d/d22/f23 zero size 2024-04-22T23:46:01.154 INFO:tasks.workunit.client.1.smithi044.stdout:8/198: creat db/f4f x:0 0 0 2024-04-22T23:46:01.158 INFO:tasks.workunit.client.1.smithi044.stdout:7/214: rename l9 to dd/d1a/l4c 0 2024-04-22T23:46:01.160 INFO:tasks.workunit.client.1.smithi044.stdout:8/199: dread db/f1d [0,4194304] 0 2024-04-22T23:46:01.161 INFO:tasks.workunit.client.1.smithi044.stdout:1/194: rmdir d1/d8/d1c/d23 39 2024-04-22T23:46:01.163 INFO:tasks.workunit.client.1.smithi044.stdout:6/161: dwrite d5/dd/d18/f2a [0,4194304] 0 2024-04-22T23:46:01.164 INFO:tasks.workunit.client.1.smithi044.stdout:1/195: write d1/d8/f26 [469155,93300] 0 2024-04-22T23:46:01.164 INFO:tasks.workunit.client.1.smithi044.stdout:5/130: chown l3 54 1 2024-04-22T23:46:01.167 INFO:tasks.workunit.client.1.smithi044.stdout:4/177: link d5/db/d16/d18/c1a d5/c39 0 2024-04-22T23:46:01.172 INFO:tasks.workunit.client.1.smithi044.stdout:4/178: dread d5/db/f21 [0,4194304] 0 2024-04-22T23:46:01.174 INFO:tasks.workunit.client.1.smithi044.stdout:2/159: rename l5 to d8/d27/l2d 0 2024-04-22T23:46:01.176 INFO:tasks.workunit.client.1.smithi044.stdout:7/215: rmdir dd/d1f 39 2024-04-22T23:46:01.181 INFO:tasks.workunit.client.1.smithi044.stdout:6/162: dread - d5/dd/d28/f2c zero size 2024-04-22T23:46:01.191 INFO:tasks.workunit.client.1.smithi044.stdout:1/196: mkdir d1/d8/d24/d46 0 2024-04-22T23:46:01.192 INFO:tasks.workunit.client.1.smithi044.stdout:4/179: dread d5/db/d16/d18/d20/f23 [0,4194304] 0 2024-04-22T23:46:01.193 INFO:tasks.workunit.client.1.smithi044.stdout:6/163: unlink d5/d8/c14 0 2024-04-22T23:46:01.198 INFO:tasks.workunit.client.1.smithi044.stdout:1/197: dwrite d1/d14/f3b [0,4194304] 0 2024-04-22T23:46:01.199 INFO:tasks.workunit.client.1.smithi044.stdout:2/160: rename d8/d27/l2d to d8/d19/d1c/l2e 0 2024-04-22T23:46:01.201 INFO:tasks.workunit.client.1.smithi044.stdout:8/200: creat db/f50 x:0 0 0 2024-04-22T23:46:01.201 INFO:tasks.workunit.client.1.smithi044.stdout:6/164: chown d5/dd/c17 840395376 1 2024-04-22T23:46:01.206 INFO:tasks.workunit.client.1.smithi044.stdout:7/216: rename dd/d1f/d24/c14 to dd/d1a/c4d 0 2024-04-22T23:46:01.208 INFO:tasks.workunit.client.1.smithi044.stdout:1/198: mkdir d1/d8/d20/d47 0 2024-04-22T23:46:01.211 INFO:tasks.workunit.client.1.smithi044.stdout:8/201: creat db/d14/d24/f51 x:0 0 0 2024-04-22T23:46:01.211 INFO:tasks.workunit.client.1.smithi044.stdout:7/217: dread dd/d1f/f31 [0,4194304] 0 2024-04-22T23:46:01.212 INFO:tasks.workunit.client.1.smithi044.stdout:7/218: fdatasync dd/d27/f37 0 2024-04-22T23:46:01.217 INFO:tasks.workunit.client.1.smithi044.stdout:7/219: dread dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:01.221 INFO:tasks.workunit.client.1.smithi044.stdout:2/161: rename d8/d19/f1e to d8/d19/d1c/f2f 0 2024-04-22T23:46:01.224 INFO:tasks.workunit.client.1.smithi044.stdout:4/180: truncate d5/d31/f11 1228161 0 2024-04-22T23:46:01.230 INFO:tasks.workunit.client.1.smithi044.stdout:4/181: dwrite d5/d31/f32 [0,4194304] 0 2024-04-22T23:46:01.235 INFO:tasks.workunit.client.1.smithi044.stdout:2/162: creat d8/d19/f30 x:0 0 0 2024-04-22T23:46:01.238 INFO:tasks.workunit.client.1.smithi044.stdout:1/199: mkdir d1/d8/d1c/d23/d48 0 2024-04-22T23:46:01.240 INFO:tasks.workunit.client.1.smithi044.stdout:4/182: rmdir d5/d31 39 2024-04-22T23:46:01.241 INFO:tasks.workunit.client.1.smithi044.stdout:1/200: truncate d1/d8/d34/f40 256645 0 2024-04-22T23:46:01.243 INFO:tasks.workunit.client.1.smithi044.stdout:7/220: symlink dd/l4e 0 2024-04-22T23:46:01.245 INFO:tasks.workunit.client.1.smithi044.stdout:2/163: dwrite d8/d19/f30 [0,4194304] 0 2024-04-22T23:46:01.248 INFO:tasks.workunit.client.1.smithi044.stdout:6/165: getdents d5 0 2024-04-22T23:46:01.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/183: mknod d5/db/d16/d18/c3a 0 2024-04-22T23:46:01.256 INFO:tasks.workunit.client.1.smithi044.stdout:1/201: unlink d1/d8/d34/f3c 0 2024-04-22T23:46:01.257 INFO:tasks.workunit.client.1.smithi044.stdout:1/202: fdatasync d1/f1b 0 2024-04-22T23:46:01.258 INFO:tasks.workunit.client.1.smithi044.stdout:8/202: getdents db/d14 0 2024-04-22T23:46:01.259 INFO:tasks.workunit.client.1.smithi044.stdout:6/166: dwrite d5/dd/d1d/d23/f29 [0,4194304] 0 2024-04-22T23:46:01.261 INFO:tasks.workunit.client.1.smithi044.stdout:2/164: mknod d8/d19/c31 0 2024-04-22T23:46:01.263 INFO:tasks.workunit.client.1.smithi044.stdout:2/165: stat d8/d19/l22 0 2024-04-22T23:46:01.269 INFO:tasks.workunit.client.1.smithi044.stdout:4/184: mknod d5/db/d34/c3b 0 2024-04-22T23:46:01.270 INFO:tasks.workunit.client.1.smithi044.stdout:6/167: mknod d5/d8/c36 0 2024-04-22T23:46:01.273 INFO:tasks.workunit.client.1.smithi044.stdout:2/166: creat d8/d19/f32 x:0 0 0 2024-04-22T23:46:01.274 INFO:tasks.workunit.client.1.smithi044.stdout:8/203: mkdir db/d52 0 2024-04-22T23:46:01.280 INFO:tasks.workunit.client.1.smithi044.stdout:6/168: dwrite f1 [0,4194304] 0 2024-04-22T23:46:01.285 INFO:tasks.workunit.client.1.smithi044.stdout:8/204: truncate db/f1d 1917778 0 2024-04-22T23:46:01.286 INFO:tasks.workunit.client.1.smithi044.stdout:8/205: dread - db/d14/d24/f39 zero size 2024-04-22T23:46:01.289 INFO:tasks.workunit.client.1.smithi044.stdout:1/203: link d1/d8/d20/d45/c37 d1/d8/d1c/d23/d48/c49 0 2024-04-22T23:46:01.296 INFO:tasks.workunit.client.1.smithi044.stdout:8/206: mknod db/d14/c53 0 2024-04-22T23:46:01.296 INFO:tasks.workunit.client.1.smithi044.stdout:8/207: dread - db/f48 zero size 2024-04-22T23:46:01.299 INFO:tasks.workunit.client.1.smithi044.stdout:4/185: truncate d5/db/f19 3736573 0 2024-04-22T23:46:01.306 INFO:tasks.workunit.client.1.smithi044.stdout:6/169: truncate d5/f6 4078994 0 2024-04-22T23:46:01.308 INFO:tasks.workunit.client.1.smithi044.stdout:6/170: truncate d5/dd/d1d/d23/f26 1957410 0 2024-04-22T23:46:01.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/204: mkdir d1/d8/d20/d47/d4a 0 2024-04-22T23:46:01.315 INFO:tasks.workunit.client.1.smithi044.stdout:8/208: fsync db/d14/d24/f4e 0 2024-04-22T23:46:01.316 INFO:tasks.workunit.client.1.smithi044.stdout:8/209: chown db/d14/c53 10 1 2024-04-22T23:46:01.318 INFO:tasks.workunit.client.1.smithi044.stdout:1/205: fdatasync d1/d8/d20/d45/f2c 0 2024-04-22T23:46:01.323 INFO:tasks.workunit.client.1.smithi044.stdout:4/186: mkdir d5/db/d3c 0 2024-04-22T23:46:01.326 INFO:tasks.workunit.client.1.smithi044.stdout:6/171: mknod d5/dd/c37 0 2024-04-22T23:46:01.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/206: mkdir d1/d8/d24/d4b 0 2024-04-22T23:46:01.332 INFO:tasks.workunit.client.1.smithi044.stdout:1/207: dread - d1/d8/d20/d45/f3d zero size 2024-04-22T23:46:01.333 INFO:tasks.workunit.client.1.smithi044.stdout:1/208: fdatasync d1/d8/d20/d45/f39 0 2024-04-22T23:46:01.337 INFO:tasks.workunit.client.1.smithi044.stdout:6/172: creat d5/f38 x:0 0 0 2024-04-22T23:46:01.339 INFO:tasks.workunit.client.1.smithi044.stdout:4/187: dread d5/d31/f32 [0,4194304] 0 2024-04-22T23:46:01.346 INFO:tasks.workunit.client.1.smithi044.stdout:8/210: rename db/d14/d24/c29 to db/c54 0 2024-04-22T23:46:01.347 INFO:tasks.workunit.client.1.smithi044.stdout:8/211: write db/d14/f3e [302703,69691] 0 2024-04-22T23:46:01.359 INFO:tasks.workunit.client.1.smithi044.stdout:6/173: chown d5/dd/f10 5675 1 2024-04-22T23:46:01.360 INFO:tasks.workunit.client.1.smithi044.stdout:8/212: write db/d14/f37 [648142,23200] 0 2024-04-22T23:46:01.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/209: dwrite d1/d8/d20/d45/f29 [0,4194304] 0 2024-04-22T23:46:01.363 INFO:tasks.workunit.client.1.smithi044.stdout:4/188: truncate d5/d31/d24/f2f 3444753 0 2024-04-22T23:46:01.364 INFO:tasks.workunit.client.1.smithi044.stdout:6/174: fsync d5/f21 0 2024-04-22T23:46:01.365 INFO:tasks.workunit.client.1.smithi044.stdout:8/213: creat db/d14/d24/f55 x:0 0 0 2024-04-22T23:46:01.365 INFO:tasks.workunit.client.1.smithi044.stdout:8/214: stat db/f3c 0 2024-04-22T23:46:01.365 INFO:tasks.workunit.client.1.smithi044.stdout:8/215: chown db/d14/d24/f4e 0 1 2024-04-22T23:46:01.367 INFO:tasks.workunit.client.1.smithi044.stdout:4/189: creat d5/d31/d24/f3d x:0 0 0 2024-04-22T23:46:01.370 INFO:tasks.workunit.client.1.smithi044.stdout:8/216: mknod db/d14/d24/c56 0 2024-04-22T23:46:01.373 INFO:tasks.workunit.client.1.smithi044.stdout:1/210: link d1/d8/cf d1/d8/d20/d47/c4c 0 2024-04-22T23:46:01.375 INFO:tasks.workunit.client.1.smithi044.stdout:1/211: write d1/d8/d20/d45/f29 [2296270,104491] 0 2024-04-22T23:46:01.376 INFO:tasks.workunit.client.1.smithi044.stdout:8/217: symlink db/d52/l57 0 2024-04-22T23:46:01.377 INFO:tasks.workunit.client.1.smithi044.stdout:8/218: chown db/d14/d24/f30 131802887 1 2024-04-22T23:46:01.378 INFO:tasks.workunit.client.1.smithi044.stdout:1/212: mkdir d1/d8/d1c/d23/d3e/d4d 0 2024-04-22T23:46:01.378 INFO:tasks.workunit.client.1.smithi044.stdout:8/219: creat db/d14/d24/f58 x:0 0 0 2024-04-22T23:46:01.379 INFO:tasks.workunit.client.1.smithi044.stdout:1/213: fsync d1/d8/d20/f44 0 2024-04-22T23:46:01.381 INFO:tasks.workunit.client.1.smithi044.stdout:1/214: unlink d1/d14/l28 0 2024-04-22T23:46:01.382 INFO:tasks.workunit.client.1.smithi044.stdout:1/215: write d1/d8/d20/d45/f3f [218678,121445] 0 2024-04-22T23:46:01.393 INFO:tasks.workunit.client.1.smithi044.stdout:8/220: getdents db/d52 0 2024-04-22T23:46:01.396 INFO:tasks.workunit.client.1.smithi044.stdout:4/190: dwrite d5/d31/f11 [0,4194304] 0 2024-04-22T23:46:01.400 INFO:tasks.workunit.client.1.smithi044.stdout:4/191: rmdir d5/d31/d24 39 2024-04-22T23:46:01.403 INFO:tasks.workunit.client.1.smithi044.stdout:4/192: fdatasync d5/db/d16/d18/d20/f23 0 2024-04-22T23:46:01.403 INFO:tasks.workunit.client.1.smithi044.stdout:4/193: stat d5/db/d16 0 2024-04-22T23:46:01.404 INFO:tasks.workunit.client.1.smithi044.stdout:4/194: chown d5/d31/f32 9 1 2024-04-22T23:46:01.405 INFO:tasks.workunit.client.1.smithi044.stdout:4/195: fsync d5/db/d16/d18/d20/d22/f37 0 2024-04-22T23:46:01.413 INFO:tasks.workunit.client.1.smithi044.stdout:4/196: read d5/d31/d24/f2f [663470,63365] 0 2024-04-22T23:46:01.415 INFO:tasks.workunit.client.1.smithi044.stdout:4/197: rmdir d5/db/d16/d18 39 2024-04-22T23:46:01.523 INFO:tasks.workunit.client.1.smithi044.stdout:3/167: sync 2024-04-22T23:46:01.523 INFO:tasks.workunit.client.1.smithi044.stdout:0/144: sync 2024-04-22T23:46:01.523 INFO:tasks.workunit.client.1.smithi044.stdout:9/133: sync 2024-04-22T23:46:01.523 INFO:tasks.workunit.client.1.smithi044.stdout:5/131: sync 2024-04-22T23:46:01.524 INFO:tasks.workunit.client.1.smithi044.stdout:0/145: read d0/d2/dd/f11 [2716588,56123] 0 2024-04-22T23:46:01.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/198: dread - d5/db/d16/d18/d20/d22/f37 zero size 2024-04-22T23:46:01.548 INFO:tasks.workunit.client.1.smithi044.stdout:9/134: creat d2/dc/f32 x:0 0 0 2024-04-22T23:46:01.549 INFO:tasks.workunit.client.1.smithi044.stdout:4/199: write d5/d31/d24/f3d [987179,35107] 0 2024-04-22T23:46:01.553 INFO:tasks.workunit.client.1.smithi044.stdout:5/132: mkdir d15/d16/d1d/d22/d2a 0 2024-04-22T23:46:01.554 INFO:tasks.workunit.client.1.smithi044.stdout:0/146: creat d0/d2/dd/f36 x:0 0 0 2024-04-22T23:46:01.555 INFO:tasks.workunit.client.1.smithi044.stdout:5/133: read d15/d16/f28 [262243,117941] 0 2024-04-22T23:46:01.556 INFO:tasks.workunit.client.1.smithi044.stdout:3/168: symlink d9/l35 0 2024-04-22T23:46:01.559 INFO:tasks.workunit.client.1.smithi044.stdout:9/135: creat d2/d6/f33 x:0 0 0 2024-04-22T23:46:01.567 INFO:tasks.workunit.client.1.smithi044.stdout:4/200: fsync d5/db/f19 0 2024-04-22T23:46:01.569 INFO:tasks.workunit.client.1.smithi044.stdout:4/201: read - d5/db/d16/d18/d20/d22/f37 zero size 2024-04-22T23:46:01.569 INFO:tasks.workunit.client.1.smithi044.stdout:0/147: mknod d0/d2e/d33/c37 0 2024-04-22T23:46:01.570 INFO:tasks.workunit.client.1.smithi044.stdout:0/148: write d0/d2/dd/f36 [953742,76070] 0 2024-04-22T23:46:01.571 INFO:tasks.workunit.client.1.smithi044.stdout:0/149: stat d0/d2/dd 0 2024-04-22T23:46:01.577 INFO:tasks.workunit.client.1.smithi044.stdout:0/150: dwrite d0/d2/f6 [0,4194304] 0 2024-04-22T23:46:01.579 INFO:tasks.workunit.client.1.smithi044.stdout:0/151: truncate d0/f16 4263789 0 2024-04-22T23:46:01.589 INFO:tasks.workunit.client.1.smithi044.stdout:4/202: creat d5/db/d16/f3e x:0 0 0 2024-04-22T23:46:01.591 INFO:tasks.workunit.client.1.smithi044.stdout:3/169: write f8 [853454,30889] 0 2024-04-22T23:46:01.594 INFO:tasks.workunit.client.1.smithi044.stdout:0/152: dwrite d0/d2/dd/f10 [0,4194304] 0 2024-04-22T23:46:01.594 INFO:tasks.workunit.client.1.smithi044.stdout:3/170: fdatasync d9/f34 0 2024-04-22T23:46:01.597 INFO:tasks.workunit.client.1.smithi044.stdout:4/203: dread d5/db/d16/d18/d20/f23 [0,4194304] 0 2024-04-22T23:46:01.597 INFO:tasks.workunit.client.1.smithi044.stdout:3/171: write d9/db/f1a [549192,9733] 0 2024-04-22T23:46:01.598 INFO:tasks.workunit.client.1.smithi044.stdout:3/172: chown f5 93283 1 2024-04-22T23:46:01.599 INFO:tasks.workunit.client.1.smithi044.stdout:3/173: chown d9/lf 6 1 2024-04-22T23:46:01.603 INFO:tasks.workunit.client.1.smithi044.stdout:0/153: write d0/d2/dd/d19/f26 [1237504,51613] 0 2024-04-22T23:46:01.604 INFO:tasks.workunit.client.1.smithi044.stdout:0/154: readlink d0/d2/l29 0 2024-04-22T23:46:01.606 INFO:tasks.workunit.client.1.smithi044.stdout:4/204: getdents d5/db/d3c 0 2024-04-22T23:46:01.607 INFO:tasks.workunit.client.1.smithi044.stdout:4/205: dread - d5/d31/d24/f27 zero size 2024-04-22T23:46:01.610 INFO:tasks.workunit.client.1.smithi044.stdout:0/155: rmdir d0/d2e/d33 39 2024-04-22T23:46:01.611 INFO:tasks.workunit.client.1.smithi044.stdout:3/174: mkdir d9/db/d36 0 2024-04-22T23:46:01.614 INFO:tasks.workunit.client.1.smithi044.stdout:9/136: write d2/dc/f27 [3309038,116774] 0 2024-04-22T23:46:01.615 INFO:tasks.workunit.client.1.smithi044.stdout:9/137: stat d2/d9/c1b 0 2024-04-22T23:46:01.616 INFO:tasks.workunit.client.1.smithi044.stdout:9/138: fsync d2/f1c 0 2024-04-22T23:46:01.618 INFO:tasks.workunit.client.1.smithi044.stdout:4/206: symlink d5/l3f 0 2024-04-22T23:46:01.619 INFO:tasks.workunit.client.1.smithi044.stdout:0/156: mknod d0/d2e/d33/c38 0 2024-04-22T23:46:01.620 INFO:tasks.workunit.client.1.smithi044.stdout:0/157: fsync d0/d2/f6 0 2024-04-22T23:46:01.620 INFO:tasks.workunit.client.1.smithi044.stdout:0/158: readlink d0/d2/l29 0 2024-04-22T23:46:01.622 INFO:tasks.workunit.client.1.smithi044.stdout:9/139: mkdir d2/dc/d1d/d34 0 2024-04-22T23:46:01.623 INFO:tasks.workunit.client.1.smithi044.stdout:9/140: write d2/d6/f33 [1000729,65709] 0 2024-04-22T23:46:01.623 INFO:tasks.workunit.client.1.smithi044.stdout:5/134: truncate ff 805479 0 2024-04-22T23:46:01.625 INFO:tasks.workunit.client.1.smithi044.stdout:5/135: truncate d15/d16/f1a 729373 0 2024-04-22T23:46:01.625 INFO:tasks.workunit.client.1.smithi044.stdout:9/141: truncate d2/d5/f1e 4715055 0 2024-04-22T23:46:01.626 INFO:tasks.workunit.client.1.smithi044.stdout:5/136: dread d15/f1e [0,4194304] 0 2024-04-22T23:46:01.627 INFO:tasks.workunit.client.1.smithi044.stdout:0/159: symlink d0/d2/l39 0 2024-04-22T23:46:01.630 INFO:tasks.workunit.client.1.smithi044.stdout:4/207: creat d5/d31/d24/d38/f40 x:0 0 0 2024-04-22T23:46:01.632 INFO:tasks.workunit.client.1.smithi044.stdout:0/160: dread d0/f16 [0,4194304] 0 2024-04-22T23:46:01.633 INFO:tasks.workunit.client.1.smithi044.stdout:4/208: write d5/db/d16/d18/d20/d22/f37 [822495,114495] 0 2024-04-22T23:46:01.635 INFO:tasks.workunit.client.1.smithi044.stdout:9/142: symlink d2/d6/l35 0 2024-04-22T23:46:01.636 INFO:tasks.workunit.client.1.smithi044.stdout:5/137: symlink d15/d16/d1d/d22/l2b 0 2024-04-22T23:46:01.639 INFO:tasks.workunit.client.1.smithi044.stdout:3/175: getdents d9 0 2024-04-22T23:46:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:0/161: creat d0/d2e/d33/f3a x:0 0 0 2024-04-22T23:46:01.642 INFO:tasks.workunit.client.1.smithi044.stdout:5/138: dwrite f11 [0,4194304] 0 2024-04-22T23:46:01.646 INFO:tasks.workunit.client.1.smithi044.stdout:3/176: rmdir d9/d14 39 2024-04-22T23:46:01.649 INFO:tasks.workunit.client.1.smithi044.stdout:4/209: symlink d5/db/d16/l41 0 2024-04-22T23:46:01.653 INFO:tasks.workunit.client.1.smithi044.stdout:0/162: creat d0/d2e/d2f/f3b x:0 0 0 2024-04-22T23:46:01.659 INFO:tasks.workunit.client.1.smithi044.stdout:4/210: mkdir d5/d42 0 2024-04-22T23:46:01.667 INFO:tasks.workunit.client.1.smithi044.stdout:4/211: dwrite d5/db/d16/d18/f29 [0,4194304] 0 2024-04-22T23:46:01.667 INFO:tasks.workunit.client.1.smithi044.stdout:5/139: dread ff [0,4194304] 0 2024-04-22T23:46:01.668 INFO:tasks.workunit.client.1.smithi044.stdout:0/163: dread d0/d2/f1e [0,4194304] 0 2024-04-22T23:46:01.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/177: rename d9/cc to d9/db/d15/d2e/c37 0 2024-04-22T23:46:01.673 INFO:tasks.workunit.client.1.smithi044.stdout:7/221: sync 2024-04-22T23:46:01.674 INFO:tasks.workunit.client.1.smithi044.stdout:5/140: write d15/d16/f28 [898644,40676] 0 2024-04-22T23:46:01.675 INFO:tasks.workunit.client.1.smithi044.stdout:0/164: dread d0/d2/dd/d19/f26 [0,4194304] 0 2024-04-22T23:46:01.677 INFO:tasks.workunit.client.1.smithi044.stdout:3/178: mkdir d9/db/d22/d38 0 2024-04-22T23:46:01.680 INFO:tasks.workunit.client.1.smithi044.stdout:4/212: dwrite d5/db/d16/d18/d20/d22/f35 [0,4194304] 0 2024-04-22T23:46:01.681 INFO:tasks.workunit.client.1.smithi044.stdout:5/141: dread fe [0,4194304] 0 2024-04-22T23:46:01.681 INFO:tasks.workunit.client.1.smithi044.stdout:7/222: fdatasync dd/d1f/d24/f30 0 2024-04-22T23:46:01.684 INFO:tasks.workunit.client.1.smithi044.stdout:3/179: rmdir d9/db/d15 39 2024-04-22T23:46:01.684 INFO:tasks.workunit.client.1.smithi044.stdout:4/213: chown d5/db/d16/f3e 798061 1 2024-04-22T23:46:01.685 INFO:tasks.workunit.client.1.smithi044.stdout:5/142: truncate d15/f18 843233 0 2024-04-22T23:46:01.686 INFO:tasks.workunit.client.1.smithi044.stdout:5/143: chown fa 21341 1 2024-04-22T23:46:01.687 INFO:tasks.workunit.client.1.smithi044.stdout:0/165: dread d0/d2/fa [0,4194304] 0 2024-04-22T23:46:01.691 INFO:tasks.workunit.client.1.smithi044.stdout:3/180: dread f8 [0,4194304] 0 2024-04-22T23:46:01.697 INFO:tasks.workunit.client.1.smithi044.stdout:4/214: rename d5/db/d16/d18/d20/c28 to d5/db/d3c/c43 0 2024-04-22T23:46:01.697 INFO:tasks.workunit.client.1.smithi044.stdout:0/166: dwrite d0/d2e/d2f/f3b [0,4194304] 0 2024-04-22T23:46:01.712 INFO:tasks.workunit.client.1.smithi044.stdout:7/223: link dd/c18 dd/d1a/c4f 0 2024-04-22T23:46:01.714 INFO:tasks.workunit.client.1.smithi044.stdout:5/144: dwrite ff [0,4194304] 0 2024-04-22T23:46:01.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/181: creat d9/db/d15/d2e/f39 x:0 0 0 2024-04-22T23:46:01.726 INFO:tasks.workunit.client.1.smithi044.stdout:4/215: rename d5/db/d16/d18 to d5/d31/d44 0 2024-04-22T23:46:01.726 INFO:tasks.workunit.client.1.smithi044.stdout:5/145: read f13 [377807,9136] 0 2024-04-22T23:46:01.727 INFO:tasks.workunit.client.1.smithi044.stdout:3/182: mkdir d9/d12/d3a 0 2024-04-22T23:46:01.728 INFO:tasks.workunit.client.1.smithi044.stdout:5/146: write d15/d16/f28 [998860,113408] 0 2024-04-22T23:46:01.728 INFO:tasks.workunit.client.1.smithi044.stdout:3/183: stat d9/db/cd 0 2024-04-22T23:46:01.730 INFO:tasks.workunit.client.1.smithi044.stdout:0/167: getdents d0/d2/dd/d19 0 2024-04-22T23:46:01.736 INFO:tasks.workunit.client.1.smithi044.stdout:3/184: write d9/f23 [667435,29796] 0 2024-04-22T23:46:01.737 INFO:tasks.workunit.client.1.smithi044.stdout:5/147: dwrite d15/d16/f17 [0,4194304] 0 2024-04-22T23:46:01.740 INFO:tasks.workunit.client.1.smithi044.stdout:0/168: dwrite d0/d2/dd/f36 [0,4194304] 0 2024-04-22T23:46:01.742 INFO:tasks.workunit.client.1.smithi044.stdout:0/169: chown d0/d2/dd/f23 2 1 2024-04-22T23:46:01.746 INFO:tasks.workunit.client.1.smithi044.stdout:0/170: dread d0/d2/dd/f12 [0,4194304] 0 2024-04-22T23:46:01.747 INFO:tasks.workunit.client.1.smithi044.stdout:0/171: chown d0/d2/dd/f13 4 1 2024-04-22T23:46:01.747 INFO:tasks.workunit.client.1.smithi044.stdout:3/185: unlink d9/db/d15/f1b 0 2024-04-22T23:46:01.748 INFO:tasks.workunit.client.1.smithi044.stdout:3/186: stat d9/db/d15/d25 0 2024-04-22T23:46:01.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/216: getdents d5/db/d3c 0 2024-04-22T23:46:01.751 INFO:tasks.workunit.client.1.smithi044.stdout:4/217: fdatasync d5/d31/d44/f30 0 2024-04-22T23:46:01.753 INFO:tasks.workunit.client.1.smithi044.stdout:5/148: rename d15/d16/d1d/d22/d27 to d15/d16/d2c 0 2024-04-22T23:46:01.755 INFO:tasks.workunit.client.1.smithi044.stdout:5/149: dread fe [0,4194304] 0 2024-04-22T23:46:01.757 INFO:tasks.workunit.client.1.smithi044.stdout:5/150: write d15/d16/d1d/d22/f23 [104238,64693] 0 2024-04-22T23:46:01.760 INFO:tasks.workunit.client.1.smithi044.stdout:4/218: symlink d5/d31/d44/d20/d22/l45 0 2024-04-22T23:46:01.766 INFO:tasks.workunit.client.1.smithi044.stdout:0/172: symlink d0/d2e/l3c 0 2024-04-22T23:46:01.768 INFO:tasks.workunit.client.1.smithi044.stdout:4/219: dwrite d5/db/f2e [0,4194304] 0 2024-04-22T23:46:01.769 INFO:tasks.workunit.client.1.smithi044.stdout:3/187: write d9/db/d15/f18 [523934,33548] 0 2024-04-22T23:46:01.773 INFO:tasks.workunit.client.1.smithi044.stdout:0/173: rename d0/d2/dd/d19 to d0/d2/d3d 0 2024-04-22T23:46:01.777 INFO:tasks.workunit.client.1.smithi044.stdout:5/151: link d15/d16/d1d/d22/f23 d15/d16/d1d/d22/f2d 0 2024-04-22T23:46:01.778 INFO:tasks.workunit.client.1.smithi044.stdout:5/152: chown l3 0 1 2024-04-22T23:46:01.780 INFO:tasks.workunit.client.1.smithi044.stdout:5/153: dread fe [0,4194304] 0 2024-04-22T23:46:01.781 INFO:tasks.workunit.client.1.smithi044.stdout:5/154: fdatasync d15/d16/f1a 0 2024-04-22T23:46:01.782 INFO:tasks.workunit.client.1.smithi044.stdout:5/155: fdatasync d15/d16/d1d/f26 0 2024-04-22T23:46:01.783 INFO:tasks.workunit.client.1.smithi044.stdout:5/156: fdatasync fa 0 2024-04-22T23:46:01.783 INFO:tasks.workunit.client.1.smithi044.stdout:0/174: rmdir d0/d2e/d2f 39 2024-04-22T23:46:01.787 INFO:tasks.workunit.client.1.smithi044.stdout:4/220: symlink d5/db/d16/l46 0 2024-04-22T23:46:01.789 INFO:tasks.workunit.client.1.smithi044.stdout:4/221: write d5/d31/d44/f30 [967898,41526] 0 2024-04-22T23:46:01.792 INFO:tasks.workunit.client.1.smithi044.stdout:0/175: dwrite d0/f17 [0,4194304] 0 2024-04-22T23:46:01.792 INFO:tasks.workunit.client.1.smithi044.stdout:3/188: link d9/db/f1a d9/db/d15/d25/f3b 0 2024-04-22T23:46:01.797 INFO:tasks.workunit.client.1.smithi044.stdout:0/176: read d0/d2e/f32 [2718407,94849] 0 2024-04-22T23:46:01.799 INFO:tasks.workunit.client.1.smithi044.stdout:4/222: fsync d5/db/f19 0 2024-04-22T23:46:01.801 INFO:tasks.workunit.client.1.smithi044.stdout:4/223: truncate d5/d31/f10 8451087 0 2024-04-22T23:46:01.802 INFO:tasks.workunit.client.1.smithi044.stdout:4/224: read d5/d31/f10 [3403000,8603] 0 2024-04-22T23:46:01.803 INFO:tasks.workunit.client.1.smithi044.stdout:4/225: chown d5/d31/d24/f27 454183 1 2024-04-22T23:46:01.804 INFO:tasks.workunit.client.1.smithi044.stdout:4/226: read d5/db/f2e [3908356,129084] 0 2024-04-22T23:46:01.804 INFO:tasks.workunit.client.1.smithi044.stdout:0/177: mkdir d0/d2/dd/d3e 0 2024-04-22T23:46:01.806 INFO:tasks.workunit.client.1.smithi044.stdout:0/178: write d0/d2/dd/f2a [998566,87323] 0 2024-04-22T23:46:01.818 INFO:tasks.workunit.client.1.smithi044.stdout:3/189: write d9/f34 [2513591,117052] 0 2024-04-22T23:46:01.820 INFO:tasks.workunit.client.1.smithi044.stdout:3/190: stat d9/db/d22/d38 0 2024-04-22T23:46:01.821 INFO:tasks.workunit.client.1.smithi044.stdout:5/157: getdents d15/d16 0 2024-04-22T23:46:01.823 INFO:tasks.workunit.client.1.smithi044.stdout:4/227: dwrite d5/d31/f1d [0,4194304] 0 2024-04-22T23:46:01.824 INFO:tasks.workunit.client.1.smithi044.stdout:0/179: creat d0/d2/dd/d3e/f3f x:0 0 0 2024-04-22T23:46:01.829 INFO:tasks.workunit.client.1.smithi044.stdout:4/228: write d5/d31/f10 [9476928,8099] 0 2024-04-22T23:46:01.829 INFO:tasks.workunit.client.1.smithi044.stdout:0/180: dread d0/d2/f6 [4194304,4194304] 0 2024-04-22T23:46:01.831 INFO:tasks.workunit.client.1.smithi044.stdout:0/181: fdatasync d0/d2/dd/f2a 0 2024-04-22T23:46:01.834 INFO:tasks.workunit.client.1.smithi044.stdout:0/182: chown d0/d2/cc 12 1 2024-04-22T23:46:01.838 INFO:tasks.workunit.client.1.smithi044.stdout:3/191: dwrite d9/db/f1a [0,4194304] 0 2024-04-22T23:46:01.840 INFO:tasks.workunit.client.1.smithi044.stdout:4/229: link d5/d31/d44/d20/d22/c2c d5/d31/d24/c47 0 2024-04-22T23:46:01.840 INFO:tasks.workunit.client.1.smithi044.stdout:0/183: mkdir d0/d2/dd/d3e/d40 0 2024-04-22T23:46:01.841 INFO:tasks.workunit.client.1.smithi044.stdout:4/230: chown c3 476438 1 2024-04-22T23:46:01.843 INFO:tasks.workunit.client.1.smithi044.stdout:4/231: fdatasync d5/d31/f10 0 2024-04-22T23:46:01.843 INFO:tasks.workunit.client.1.smithi044.stdout:0/184: chown d0/d2/dd/c35 2 1 2024-04-22T23:46:01.844 INFO:tasks.workunit.client.1.smithi044.stdout:3/192: mkdir d9/db/d15/d29/d3c 0 2024-04-22T23:46:01.849 INFO:tasks.workunit.client.1.smithi044.stdout:3/193: write d9/f19 [2869416,79235] 0 2024-04-22T23:46:01.850 INFO:tasks.workunit.client.1.smithi044.stdout:5/158: dread f13 [0,4194304] 0 2024-04-22T23:46:01.850 INFO:tasks.workunit.client.1.smithi044.stdout:4/232: creat d5/d42/f48 x:0 0 0 2024-04-22T23:46:01.851 INFO:tasks.workunit.client.1.smithi044.stdout:4/233: stat d5/db/d16/l46 0 2024-04-22T23:46:01.852 INFO:tasks.workunit.client.1.smithi044.stdout:0/185: creat d0/d2/dd/d3e/d40/f41 x:0 0 0 2024-04-22T23:46:01.854 INFO:tasks.workunit.client.1.smithi044.stdout:5/159: rename fa to d15/d16/d2c/f2e 0 2024-04-22T23:46:01.855 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:01 smithi044 ceph-mon[34463]: pgmap v4: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail 2024-04-22T23:46:01.856 INFO:tasks.workunit.client.1.smithi044.stdout:3/194: symlink d9/db/d33/l3d 0 2024-04-22T23:46:01.858 INFO:tasks.workunit.client.1.smithi044.stdout:5/160: mknod d15/d16/c2f 0 2024-04-22T23:46:01.859 INFO:tasks.workunit.client.1.smithi044.stdout:4/234: symlink d5/d31/l49 0 2024-04-22T23:46:01.865 INFO:tasks.workunit.client.1.smithi044.stdout:5/161: creat d15/f30 x:0 0 0 2024-04-22T23:46:01.866 INFO:tasks.workunit.client.1.smithi044.stdout:4/235: dwrite d5/d31/f17 [0,4194304] 0 2024-04-22T23:46:01.871 INFO:tasks.workunit.client.1.smithi044.stdout:3/195: mknod d9/db/d22/d38/c3e 0 2024-04-22T23:46:01.873 INFO:tasks.workunit.client.1.smithi044.stdout:4/236: dread d5/d31/d24/f3d [0,4194304] 0 2024-04-22T23:46:01.874 INFO:tasks.workunit.client.1.smithi044.stdout:4/237: write d5/d31/d24/f27 [665583,115869] 0 2024-04-22T23:46:01.876 INFO:tasks.workunit.client.1.smithi044.stdout:5/162: symlink d15/l31 0 2024-04-22T23:46:01.877 INFO:tasks.workunit.client.1.smithi044.stdout:5/163: fsync f11 0 2024-04-22T23:46:01.882 INFO:tasks.workunit.client.1.smithi044.stdout:0/186: getdents d0/d2e 0 2024-04-22T23:46:01.882 INFO:tasks.workunit.client.1.smithi044.stdout:3/196: creat d9/db/d15/d2e/f3f x:0 0 0 2024-04-22T23:46:01.883 INFO:tasks.workunit.client.1.smithi044.stdout:4/238: creat d5/d31/d44/f4a x:0 0 0 2024-04-22T23:46:01.888 INFO:tasks.workunit.client.1.smithi044.stdout:0/187: mknod d0/d2e/d2f/c42 0 2024-04-22T23:46:01.891 INFO:tasks.workunit.client.1.smithi044.stdout:3/197: dwrite d9/f19 [4194304,4194304] 0 2024-04-22T23:46:01.894 INFO:tasks.workunit.client.1.smithi044.stdout:4/239: dwrite d5/d31/d44/f4a [0,4194304] 0 2024-04-22T23:46:01.898 INFO:tasks.workunit.client.1.smithi044.stdout:0/188: mkdir d0/d2/dd/d3e/d40/d43 0 2024-04-22T23:46:01.903 INFO:tasks.workunit.client.1.smithi044.stdout:3/198: creat d9/d12/d3a/f40 x:0 0 0 2024-04-22T23:46:01.904 INFO:tasks.workunit.client.1.smithi044.stdout:4/240: dwrite d5/d42/f48 [0,4194304] 0 2024-04-22T23:46:01.906 INFO:tasks.workunit.client.1.smithi044.stdout:3/199: truncate d9/db/d15/d2e/f39 714842 0 2024-04-22T23:46:01.907 INFO:tasks.workunit.client.1.smithi044.stdout:0/189: creat d0/d2/dd/d3e/d40/f44 x:0 0 0 2024-04-22T23:46:01.912 INFO:tasks.workunit.client.1.smithi044.stdout:5/164: truncate d15/f18 229590 0 2024-04-22T23:46:01.916 INFO:tasks.workunit.client.1.smithi044.stdout:5/165: mknod d15/d16/d1d/c32 0 2024-04-22T23:46:01.917 INFO:tasks.workunit.client.1.smithi044.stdout:3/200: dwrite d9/db/f10 [4194304,4194304] 0 2024-04-22T23:46:01.919 INFO:tasks.workunit.client.1.smithi044.stdout:4/241: link d5/d42/f48 d5/d31/d44/d20/f4b 0 2024-04-22T23:46:01.920 INFO:tasks.workunit.client.1.smithi044.stdout:4/242: chown d5/db/d16/l41 10 1 2024-04-22T23:46:01.922 INFO:tasks.workunit.client.1.smithi044.stdout:5/166: chown d15/d16/d1d/d22/f2d 297 1 2024-04-22T23:46:01.923 INFO:tasks.workunit.client.1.smithi044.stdout:3/201: write d9/db/d15/d2e/f39 [29105,5237] 0 2024-04-22T23:46:01.925 INFO:tasks.workunit.client.1.smithi044.stdout:5/167: truncate d15/d16/d1d/d22/f23 663477 0 2024-04-22T23:46:01.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/243: creat d5/d31/d44/d20/d22/f4c x:0 0 0 2024-04-22T23:46:01.930 INFO:tasks.workunit.client.1.smithi044.stdout:3/202: creat d9/db/d22/d38/f41 x:0 0 0 2024-04-22T23:46:01.930 INFO:tasks.workunit.client.1.smithi044.stdout:5/168: dread ff [0,4194304] 0 2024-04-22T23:46:01.938 INFO:tasks.workunit.client.1.smithi044.stdout:3/203: symlink d9/db/d22/l42 0 2024-04-22T23:46:01.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/244: mkdir d5/db/d16/d4d 0 2024-04-22T23:46:01.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:01 smithi031 ceph-mon[26721]: pgmap v4: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail 2024-04-22T23:46:01.945 INFO:tasks.workunit.client.1.smithi044.stdout:3/204: dwrite d9/d12/f2d [0,4194304] 0 2024-04-22T23:46:01.946 INFO:tasks.workunit.client.1.smithi044.stdout:5/169: fsync f14 0 2024-04-22T23:46:01.947 INFO:tasks.workunit.client.1.smithi044.stdout:3/205: chown d9/db/d15/d25/l26 4610 1 2024-04-22T23:46:01.949 INFO:tasks.workunit.client.1.smithi044.stdout:3/206: read - d9/d12/d3a/f40 zero size 2024-04-22T23:46:01.954 INFO:tasks.workunit.client.1.smithi044.stdout:5/170: dwrite f11 [0,4194304] 0 2024-04-22T23:46:01.961 INFO:tasks.workunit.client.1.smithi044.stdout:4/245: link d5/d31/d44/d20/d22/l45 d5/db/l4e 0 2024-04-22T23:46:01.962 INFO:tasks.workunit.client.1.smithi044.stdout:4/246: fsync d5/d31/f10 0 2024-04-22T23:46:01.963 INFO:tasks.workunit.client.1.smithi044.stdout:3/207: link d9/db/d15/d2e/f32 d9/d14/f43 0 2024-04-22T23:46:01.965 INFO:tasks.workunit.client.1.smithi044.stdout:4/247: write d5/d42/f48 [2892902,61080] 0 2024-04-22T23:46:01.967 INFO:tasks.workunit.client.1.smithi044.stdout:4/248: fdatasync d5/d31/f17 0 2024-04-22T23:46:01.969 INFO:tasks.workunit.client.1.smithi044.stdout:3/208: dread d9/db/f1a [0,4194304] 0 2024-04-22T23:46:01.970 INFO:tasks.workunit.client.1.smithi044.stdout:5/171: link d15/d16/c1f d15/d16/d1d/c33 0 2024-04-22T23:46:01.983 INFO:tasks.workunit.client.1.smithi044.stdout:3/209: mknod d9/db/c44 0 2024-04-22T23:46:01.983 INFO:tasks.workunit.client.1.smithi044.stdout:3/210: chown d9/d12/d3a 87396 1 2024-04-22T23:46:01.983 INFO:tasks.workunit.client.1.smithi044.stdout:5/172: unlink d15/d16/c1f 0 2024-04-22T23:46:01.985 INFO:tasks.workunit.client.1.smithi044.stdout:4/249: link d5/db/l4e d5/d31/d24/l4f 0 2024-04-22T23:46:01.986 INFO:tasks.workunit.client.1.smithi044.stdout:5/173: write f8 [4837937,45586] 0 2024-04-22T23:46:01.986 INFO:tasks.workunit.client.1.smithi044.stdout:3/211: mknod d9/d14/c45 0 2024-04-22T23:46:01.991 INFO:tasks.workunit.client.1.smithi044.stdout:5/174: truncate f12 209696 0 2024-04-22T23:46:01.991 INFO:tasks.workunit.client.1.smithi044.stdout:4/250: mknod d5/d31/d24/c50 0 2024-04-22T23:46:01.991 INFO:tasks.workunit.client.1.smithi044.stdout:4/251: chown d5/d31/d44/d20/f23 51 1 2024-04-22T23:46:01.993 INFO:tasks.workunit.client.1.smithi044.stdout:5/175: symlink d15/d16/d1d/d22/l34 0 2024-04-22T23:46:01.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/252: creat d5/db/d34/f51 x:0 0 0 2024-04-22T23:46:01.995 INFO:tasks.workunit.client.1.smithi044.stdout:3/212: dread d9/f17 [0,4194304] 0 2024-04-22T23:46:01.996 INFO:tasks.workunit.client.1.smithi044.stdout:5/176: symlink d15/d16/l35 0 2024-04-22T23:46:01.998 INFO:tasks.workunit.client.1.smithi044.stdout:3/213: mknod d9/db/d15/d29/c46 0 2024-04-22T23:46:02.002 INFO:tasks.workunit.client.1.smithi044.stdout:4/253: mkdir d5/d52 0 2024-04-22T23:46:02.007 INFO:tasks.workunit.client.1.smithi044.stdout:4/254: unlink d5/db/f19 0 2024-04-22T23:46:02.010 INFO:tasks.workunit.client.1.smithi044.stdout:3/214: mknod d9/db/d36/c47 0 2024-04-22T23:46:02.012 INFO:tasks.workunit.client.1.smithi044.stdout:4/255: unlink d5/d31/lf 0 2024-04-22T23:46:02.013 INFO:tasks.workunit.client.1.smithi044.stdout:4/256: chown d5/db 1000158 1 2024-04-22T23:46:02.017 INFO:tasks.workunit.client.1.smithi044.stdout:3/215: mkdir d9/db/d15/d2e/d48 0 2024-04-22T23:46:02.018 INFO:tasks.workunit.client.1.smithi044.stdout:3/216: chown d9/db/d15/d25 27407635 1 2024-04-22T23:46:02.020 INFO:tasks.workunit.client.1.smithi044.stdout:4/257: creat d5/db/d3c/f53 x:0 0 0 2024-04-22T23:46:02.027 INFO:tasks.workunit.client.1.smithi044.stdout:4/258: dwrite d5/d31/d44/d20/d22/f35 [0,4194304] 0 2024-04-22T23:46:02.030 INFO:tasks.workunit.client.1.smithi044.stdout:5/177: dwrite ff [4194304,4194304] 0 2024-04-22T23:46:02.030 INFO:tasks.workunit.client.1.smithi044.stdout:4/259: write d5/db/d16/f3e [554495,90148] 0 2024-04-22T23:46:02.055 INFO:tasks.workunit.client.1.smithi044.stdout:5/178: creat d15/d16/f36 x:0 0 0 2024-04-22T23:46:02.055 INFO:tasks.workunit.client.1.smithi044.stdout:5/179: truncate f12 1245706 0 2024-04-22T23:46:02.063 INFO:tasks.workunit.client.1.smithi044.stdout:4/260: truncate d5/d31/d44/d20/d22/f37 422870 0 2024-04-22T23:46:02.067 INFO:tasks.workunit.client.1.smithi044.stdout:4/261: dread d5/d31/f1d [0,4194304] 0 2024-04-22T23:46:02.076 INFO:tasks.workunit.client.1.smithi044.stdout:1/216: sync 2024-04-22T23:46:02.077 INFO:tasks.workunit.client.1.smithi044.stdout:8/221: sync 2024-04-22T23:46:02.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/175: sync 2024-04-22T23:46:02.077 INFO:tasks.workunit.client.1.smithi044.stdout:2/167: sync 2024-04-22T23:46:02.078 INFO:tasks.workunit.client.1.smithi044.stdout:2/168: chown d8/d19/d1c 930935616 1 2024-04-22T23:46:02.082 INFO:tasks.workunit.client.1.smithi044.stdout:5/180: dwrite d15/d16/d1d/d22/f2d [0,4194304] 0 2024-04-22T23:46:02.091 INFO:tasks.workunit.client.1.smithi044.stdout:6/176: fsync d5/dd/d18/f2a 0 2024-04-22T23:46:02.092 INFO:tasks.workunit.client.1.smithi044.stdout:2/169: dwrite d8/d19/d1c/f1d [0,4194304] 0 2024-04-22T23:46:02.094 INFO:tasks.workunit.client.1.smithi044.stdout:1/217: mknod d1/d8/d24/d46/c4e 0 2024-04-22T23:46:02.097 INFO:tasks.workunit.client.1.smithi044.stdout:5/181: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:02.101 INFO:tasks.workunit.client.1.smithi044.stdout:4/262: dwrite d5/d31/d24/f2f [0,4194304] 0 2024-04-22T23:46:02.111 INFO:tasks.workunit.client.1.smithi044.stdout:5/182: mkdir d15/d16/d2c/d37 0 2024-04-22T23:46:02.112 INFO:tasks.workunit.client.1.smithi044.stdout:5/183: write d15/d16/d1d/f26 [3744003,109369] 0 2024-04-22T23:46:02.114 INFO:tasks.workunit.client.1.smithi044.stdout:5/184: write d15/d16/f28 [1621193,130484] 0 2024-04-22T23:46:02.115 INFO:tasks.workunit.client.1.smithi044.stdout:1/218: rename d1/d8/d1c/d23/f27 to d1/d8/d20/d45/f4f 0 2024-04-22T23:46:02.127 INFO:tasks.workunit.client.1.smithi044.stdout:5/185: mkdir d15/d16/d2c/d38 0 2024-04-22T23:46:02.131 INFO:tasks.workunit.client.1.smithi044.stdout:5/186: dread d15/d16/d1d/f26 [0,4194304] 0 2024-04-22T23:46:02.133 INFO:tasks.workunit.client.1.smithi044.stdout:1/219: symlink d1/l50 0 2024-04-22T23:46:02.134 INFO:tasks.workunit.client.1.smithi044.stdout:1/220: write d1/d8/d1c/d23/f42 [250605,40357] 0 2024-04-22T23:46:02.135 INFO:tasks.workunit.client.1.smithi044.stdout:1/221: fdatasync d1/d8/d20/f44 0 2024-04-22T23:46:02.137 INFO:tasks.workunit.client.1.smithi044.stdout:4/263: creat d5/d31/f54 x:0 0 0 2024-04-22T23:46:02.138 INFO:tasks.workunit.client.1.smithi044.stdout:5/187: creat d15/d16/f39 x:0 0 0 2024-04-22T23:46:02.143 INFO:tasks.workunit.client.1.smithi044.stdout:1/222: mknod d1/c51 0 2024-04-22T23:46:02.146 INFO:tasks.workunit.client.1.smithi044.stdout:5/188: mkdir d15/d16/d1d/d22/d2a/d3a 0 2024-04-22T23:46:02.146 INFO:tasks.workunit.client.1.smithi044.stdout:2/170: dwrite d8/f15 [0,4194304] 0 2024-04-22T23:46:02.148 INFO:tasks.workunit.client.1.smithi044.stdout:4/264: rename d5/d31/d24/f27 to d5/d31/d44/d20/f55 0 2024-04-22T23:46:02.148 INFO:tasks.workunit.client.1.smithi044.stdout:2/171: write d8/d19/d1c/f1d [1103470,104957] 0 2024-04-22T23:46:02.150 INFO:tasks.workunit.client.1.smithi044.stdout:1/223: fdatasync d1/d8/d20/d45/f4f 0 2024-04-22T23:46:02.153 INFO:tasks.workunit.client.1.smithi044.stdout:1/224: fdatasync d1/d8/f18 0 2024-04-22T23:46:02.156 INFO:tasks.workunit.client.1.smithi044.stdout:2/172: creat d8/d19/d1c/f33 x:0 0 0 2024-04-22T23:46:02.159 INFO:tasks.workunit.client.1.smithi044.stdout:4/265: symlink d5/d31/d24/l56 0 2024-04-22T23:46:02.159 INFO:tasks.workunit.client.1.smithi044.stdout:6/177: dwrite d5/dd/f1b [0,4194304] 0 2024-04-22T23:46:02.166 INFO:tasks.workunit.client.1.smithi044.stdout:1/225: creat d1/d14/f52 x:0 0 0 2024-04-22T23:46:02.168 INFO:tasks.workunit.client.1.smithi044.stdout:2/173: symlink d8/d19/d1c/l34 0 2024-04-22T23:46:02.169 INFO:tasks.workunit.client.1.smithi044.stdout:5/189: dread d15/d16/d2c/f2e [0,4194304] 0 2024-04-22T23:46:02.172 INFO:tasks.workunit.client.1.smithi044.stdout:4/266: dread d5/db/f21 [0,4194304] 0 2024-04-22T23:46:02.174 INFO:tasks.workunit.client.1.smithi044.stdout:1/226: rename d1/ld to d1/d8/d20/l53 0 2024-04-22T23:46:02.178 INFO:tasks.workunit.client.1.smithi044.stdout:6/178: creat d5/d30/f39 x:0 0 0 2024-04-22T23:46:02.179 INFO:tasks.workunit.client.1.smithi044.stdout:5/190: creat d15/d16/d2c/f3b x:0 0 0 2024-04-22T23:46:02.182 INFO:tasks.workunit.client.1.smithi044.stdout:4/267: symlink d5/d31/d44/d20/d22/l57 0 2024-04-22T23:46:02.185 INFO:tasks.workunit.client.1.smithi044.stdout:6/179: dwrite d5/dd/d18/f20 [0,4194304] 0 2024-04-22T23:46:02.188 INFO:tasks.workunit.client.1.smithi044.stdout:9/143: sync 2024-04-22T23:46:02.188 INFO:tasks.workunit.client.1.smithi044.stdout:2/174: getdents d8 0 2024-04-22T23:46:02.191 INFO:tasks.workunit.client.1.smithi044.stdout:6/180: creat d5/dd/d1d/f3a x:0 0 0 2024-04-22T23:46:02.192 INFO:tasks.workunit.client.1.smithi044.stdout:6/181: chown d5/dd/c37 1983417986 1 2024-04-22T23:46:02.193 INFO:tasks.workunit.client.1.smithi044.stdout:6/182: stat d5/dd/d28 0 2024-04-22T23:46:02.197 INFO:tasks.workunit.client.1.smithi044.stdout:2/175: dwrite d8/d19/d1c/f1d [0,4194304] 0 2024-04-22T23:46:02.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/144: rmdir d2/d6 39 2024-04-22T23:46:02.198 INFO:tasks.workunit.client.1.smithi044.stdout:2/176: chown d8/d19/f32 846 1 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.209+0000 7ff4755d3700 1 -- 172.21.15.31:0/3493667400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 msgr2=0x7ff470103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.209+0000 7ff4755d3700 1 --2- 172.21.15.31:0/3493667400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470103bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff460009910 tx=0x7ff460009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 -- 172.21.15.31:0/3493667400 shutdown_connections 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 --2- 172.21.15.31:0/3493667400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 0x7ff470106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 --2- 172.21.15.31:0/3493667400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470103bf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 -- 172.21.15.31:0/3493667400 >> 172.21.15.31:0/3493667400 conn(0x7ff4700fb190 msgr2=0x7ff4700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 -- 172.21.15.31:0/3493667400 shutdown_connections 2024-04-22T23:46:02.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 -- 172.21.15.31:0/3493667400 wait complete. 2024-04-22T23:46:02.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.210+0000 7ff4755d3700 1 Processor -- start 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff4755d3700 1 -- start start 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff4755d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff4755d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 0x7ff4700ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff4755d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700ffcf0 con 0x7ff470101800 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff4755d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700ffe30 con 0x7ff470104130 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37368/0 (socket says 172.21.15.31:37368) 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46effd700 1 -- 172.21.15.31:0/8347321 learned_addr learned my addr 172.21.15.31:0/8347321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46e7fc700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 0x7ff4700ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46effd700 1 -- 172.21.15.31:0/8347321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 msgr2=0x7ff4700ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46effd700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 0x7ff4700ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.211+0000 7ff46effd700 1 -- 172.21.15.31:0/8347321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600095f0 con 0x7ff470101800 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff46e7fc700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 0x7ff4700ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:46:02.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff46effd700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470101160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff46000bd70 tx=0x7ff46000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:02.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46001d070 con 0x7ff470101800 2024-04-22T23:46:02.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4701000b0 con 0x7ff470101800 2024-04-22T23:46:02.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff460022470 con 0x7ff470101800 2024-04-22T23:46:02.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46000f670 con 0x7ff470101800 2024-04-22T23:46:02.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.212+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4701005a0 con 0x7ff470101800 2024-04-22T23:46:02.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.213+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff470190690 con 0x7ff470101800 2024-04-22T23:46:02.206 INFO:tasks.workunit.client.1.smithi044.stdout:2/177: creat d8/d19/d1c/f35 x:0 0 0 2024-04-22T23:46:02.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/191: truncate f12 21847 0 2024-04-22T23:46:02.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.214+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 28) v1 ==== 46697+0+0 (secure 0 0 0) 0x7ff460022a50 con 0x7ff470101800 2024-04-22T23:46:02.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.214+0000 7ff457fff700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff4580398c0 0x7ff45803bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:02.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.214+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff46004fa90 con 0x7ff470101800 2024-04-22T23:46:02.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.215+0000 7ff46e7fc700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff4580398c0 0x7ff45803bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:02.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.215+0000 7ff46e7fc700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff4580398c0 0x7ff45803bd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff464007900 tx=0x7ff464008040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:02.208 INFO:tasks.workunit.client.1.smithi044.stdout:4/268: link d5/d31/c15 d5/d31/d44/d20/c58 0 2024-04-22T23:46:02.209 INFO:tasks.workunit.client.1.smithi044.stdout:4/269: read - d5/d31/d24/d38/f40 zero size 2024-04-22T23:46:02.210 INFO:tasks.workunit.client.1.smithi044.stdout:2/178: mkdir d8/d19/d36 0 2024-04-22T23:46:02.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.219+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7ff46002baa0 con 0x7ff470101800 2024-04-22T23:46:02.215 INFO:tasks.workunit.client.1.smithi044.stdout:4/270: rename d5/d31/f1d to d5/db/d34/f59 0 2024-04-22T23:46:02.216 INFO:tasks.workunit.client.1.smithi044.stdout:6/183: getdents d5/dd/d28 0 2024-04-22T23:46:02.219 INFO:tasks.workunit.client.1.smithi044.stdout:5/192: mkdir d15/d3c 0 2024-04-22T23:46:02.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/179: rename d8/f17 to d8/d19/f37 0 2024-04-22T23:46:02.232 INFO:tasks.workunit.client.1.smithi044.stdout:4/271: symlink d5/d31/d44/d20/l5a 0 2024-04-22T23:46:02.242 INFO:tasks.workunit.client.1.smithi044.stdout:4/272: truncate d5/d31/d24/f3d 523313 0 2024-04-22T23:46:02.242 INFO:tasks.workunit.client.1.smithi044.stdout:2/180: write f4 [285700,103846] 0 2024-04-22T23:46:02.242 INFO:tasks.workunit.client.1.smithi044.stdout:5/193: dwrite d15/d16/f28 [0,4194304] 0 2024-04-22T23:46:02.244 INFO:tasks.workunit.client.1.smithi044.stdout:6/184: dread d5/dd/f16 [0,4194304] 0 2024-04-22T23:46:02.248 INFO:tasks.workunit.client.1.smithi044.stdout:5/194: creat d15/d16/d1d/d22/f3d x:0 0 0 2024-04-22T23:46:02.258 INFO:tasks.workunit.client.1.smithi044.stdout:6/185: rename d5/f6 to d5/dd/d1d/d23/f3b 0 2024-04-22T23:46:02.261 INFO:tasks.workunit.client.1.smithi044.stdout:2/181: dread f6 [0,4194304] 0 2024-04-22T23:46:02.262 INFO:tasks.workunit.client.1.smithi044.stdout:2/182: fsync d8/d19/d1c/f35 0 2024-04-22T23:46:02.263 INFO:tasks.workunit.client.1.smithi044.stdout:5/195: rmdir d15/d16/d2c/d37 0 2024-04-22T23:46:02.264 INFO:tasks.workunit.client.1.smithi044.stdout:4/273: getdents d5/db/d3c 0 2024-04-22T23:46:02.267 INFO:tasks.workunit.client.1.smithi044.stdout:2/183: dread d8/d19/f2b [0,4194304] 0 2024-04-22T23:46:02.269 INFO:tasks.workunit.client.1.smithi044.stdout:5/196: dread d15/d16/f28 [0,4194304] 0 2024-04-22T23:46:02.270 INFO:tasks.workunit.client.1.smithi044.stdout:4/274: dread d5/d42/f48 [0,4194304] 0 2024-04-22T23:46:02.276 INFO:tasks.workunit.client.1.smithi044.stdout:4/275: dwrite d5/d31/f11 [0,4194304] 0 2024-04-22T23:46:02.281 INFO:tasks.workunit.client.1.smithi044.stdout:5/197: mknod d15/c3e 0 2024-04-22T23:46:02.282 INFO:tasks.workunit.client.1.smithi044.stdout:5/198: chown d15/d16/d1d/c32 169067 1 2024-04-22T23:46:02.286 INFO:tasks.workunit.client.1.smithi044.stdout:6/186: truncate d5/dd/f1e 439128 0 2024-04-22T23:46:02.290 INFO:tasks.workunit.client.1.smithi044.stdout:5/199: creat d15/d16/d1d/f3f x:0 0 0 2024-04-22T23:46:02.292 INFO:tasks.workunit.client.1.smithi044.stdout:4/276: symlink d5/db/l5b 0 2024-04-22T23:46:02.295 INFO:tasks.workunit.client.1.smithi044.stdout:6/187: mknod d5/dd/c3c 0 2024-04-22T23:46:02.297 INFO:tasks.workunit.client.1.smithi044.stdout:5/200: dwrite d15/d16/d1d/f3f [0,4194304] 0 2024-04-22T23:46:02.300 INFO:tasks.workunit.client.1.smithi044.stdout:4/277: symlink d5/l5c 0 2024-04-22T23:46:02.306 INFO:tasks.workunit.client.1.smithi044.stdout:5/201: fdatasync d15/f18 0 2024-04-22T23:46:02.318 INFO:tasks.workunit.client.1.smithi044.stdout:6/188: getdents d5/d30 0 2024-04-22T23:46:02.318 INFO:tasks.workunit.client.1.smithi044.stdout:6/189: chown d5/dd/d18 0 1 2024-04-22T23:46:02.319 INFO:tasks.workunit.client.1.smithi044.stdout:5/202: write f12 [748269,58123] 0 2024-04-22T23:46:02.320 INFO:tasks.workunit.client.1.smithi044.stdout:4/278: getdents d5/d31/d24/d38 0 2024-04-22T23:46:02.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/203: chown d15/d16/d1d 44369 1 2024-04-22T23:46:02.324 INFO:tasks.workunit.client.1.smithi044.stdout:5/204: mkdir d15/d16/d1d/d22/d40 0 2024-04-22T23:46:02.325 INFO:tasks.workunit.client.1.smithi044.stdout:6/190: dwrite d5/dd/f1b [0,4194304] 0 2024-04-22T23:46:02.348 INFO:tasks.workunit.client.1.smithi044.stdout:7/224: sync 2024-04-22T23:46:02.349 INFO:tasks.workunit.client.1.smithi044.stdout:0/190: sync 2024-04-22T23:46:02.351 INFO:tasks.workunit.client.1.smithi044.stdout:7/225: fdatasync fc 0 2024-04-22T23:46:02.351 INFO:tasks.workunit.client.1.smithi044.stdout:4/279: write d5/db/d34/f59 [4698588,37248] 0 2024-04-22T23:46:02.355 INFO:tasks.workunit.client.1.smithi044.stdout:0/191: creat d0/d2e/d33/f45 x:0 0 0 2024-04-22T23:46:02.356 INFO:tasks.workunit.client.1.smithi044.stdout:7/226: write fa [797427,110763] 0 2024-04-22T23:46:02.359 INFO:tasks.workunit.client.1.smithi044.stdout:5/205: dwrite d15/d16/f39 [0,4194304] 0 2024-04-22T23:46:02.361 INFO:tasks.workunit.client.1.smithi044.stdout:7/227: truncate dd/d1f/f34 4201809 0 2024-04-22T23:46:02.361 INFO:tasks.workunit.client.1.smithi044.stdout:0/192: rename d0/d2/cf to d0/d2/c46 0 2024-04-22T23:46:02.364 INFO:tasks.workunit.client.1.smithi044.stdout:5/206: dread ff [0,4194304] 0 2024-04-22T23:46:02.365 INFO:tasks.workunit.client.1.smithi044.stdout:5/207: chown f11 181662037 1 2024-04-22T23:46:02.366 INFO:tasks.workunit.client.1.smithi044.stdout:4/280: rename d5/db/l12 to d5/db/l5d 0 2024-04-22T23:46:02.371 INFO:tasks.workunit.client.1.smithi044.stdout:0/193: symlink d0/d2e/d2f/l47 0 2024-04-22T23:46:02.373 INFO:tasks.workunit.client.1.smithi044.stdout:4/281: dwrite d5/d31/f17 [0,4194304] 0 2024-04-22T23:46:02.373 INFO:tasks.workunit.client.1.smithi044.stdout:0/194: write d0/d2/dd/f11 [3212319,13807] 0 2024-04-22T23:46:02.375 INFO:tasks.workunit.client.1.smithi044.stdout:4/282: write d5/d31/f10 [9144011,119264] 0 2024-04-22T23:46:02.377 INFO:tasks.workunit.client.1.smithi044.stdout:4/283: write d5/d31/f11 [1868530,78911] 0 2024-04-22T23:46:02.377 INFO:tasks.workunit.client.1.smithi044.stdout:7/228: rename dd/d1f/d24/l40 to dd/d1f/l50 0 2024-04-22T23:46:02.379 INFO:tasks.workunit.client.1.smithi044.stdout:7/229: read dd/d1f/d24/f30 [8065511,26124] 0 2024-04-22T23:46:02.380 INFO:tasks.workunit.client.1.smithi044.stdout:0/195: rmdir d0/d2/dd/d3e/d40 39 2024-04-22T23:46:02.385 INFO:tasks.workunit.client.1.smithi044.stdout:4/284: rename d5/d31/l49 to d5/d31/d24/l5e 0 2024-04-22T23:46:02.389 INFO:tasks.workunit.client.1.smithi044.stdout:0/196: dwrite d0/d2/d3d/f20 [0,4194304] 0 2024-04-22T23:46:02.391 INFO:tasks.workunit.client.1.smithi044.stdout:4/285: creat d5/d31/d44/d20/f5f x:0 0 0 2024-04-22T23:46:02.393 INFO:tasks.workunit.client.1.smithi044.stdout:0/197: mknod d0/c48 0 2024-04-22T23:46:02.398 INFO:tasks.workunit.client.1.smithi044.stdout:0/198: dread - d0/d2/dd/d3e/d40/f44 zero size 2024-04-22T23:46:02.401 INFO:tasks.workunit.client.1.smithi044.stdout:4/286: dwrite d5/d31/f54 [0,4194304] 0 2024-04-22T23:46:02.437 INFO:tasks.workunit.client.1.smithi044.stdout:0/199: link d0/d2/dd/f13 d0/d2/f49 0 2024-04-22T23:46:02.437 INFO:tasks.workunit.client.1.smithi044.stdout:0/200: readlink d0/l1c 0 2024-04-22T23:46:02.437 INFO:tasks.workunit.client.1.smithi044.stdout:7/230: dwrite dd/ff [0,4194304] 0 2024-04-22T23:46:02.437 INFO:tasks.workunit.client.1.smithi044.stdout:7/231: chown dd/d1f/d24/f1c 1950495 1 2024-04-22T23:46:02.437 INFO:tasks.workunit.client.1.smithi044.stdout:4/287: dwrite d5/db/d34/f51 [0,4194304] 0 2024-04-22T23:46:02.437 INFO:tasks.workunit.client.1.smithi044.stdout:0/201: creat d0/d2/dd/d3e/d40/d43/f4a x:0 0 0 2024-04-22T23:46:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:0/202: readlink d0/d2/l3 0 2024-04-22T23:46:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:7/232: link dd/d1a/l45 dd/d27/l51 0 2024-04-22T23:46:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:7/233: truncate dd/d1f/d24/f3d 879721 0 2024-04-22T23:46:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/288: dread d5/d31/d44/f30 [0,4194304] 0 2024-04-22T23:46:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/289: dread - d5/d31/d44/d20/f5f zero size 2024-04-22T23:46:02.438 INFO:tasks.workunit.client.1.smithi044.stdout:4/290: creat d5/d52/f60 x:0 0 0 2024-04-22T23:46:02.439 INFO:tasks.workunit.client.1.smithi044.stdout:4/291: chown d5/db/l4e 1354030 1 2024-04-22T23:46:02.440 INFO:tasks.workunit.client.1.smithi044.stdout:0/203: rmdir d0/d2/dd 39 2024-04-22T23:46:02.446 INFO:tasks.workunit.client.1.smithi044.stdout:0/204: dread d0/d2e/d2f/f3b [0,4194304] 0 2024-04-22T23:46:02.447 INFO:tasks.workunit.client.1.smithi044.stdout:7/234: dwrite dd/d27/f32 [0,4194304] 0 2024-04-22T23:46:02.448 INFO:tasks.workunit.client.1.smithi044.stdout:0/205: chown d0/d2e/d2f/l47 1595 1 2024-04-22T23:46:02.451 INFO:tasks.workunit.client.1.smithi044.stdout:7/235: creat dd/d1a/d3c/f52 x:0 0 0 2024-04-22T23:46:02.455 INFO:tasks.workunit.client.1.smithi044.stdout:0/206: mkdir d0/d2/d4b 0 2024-04-22T23:46:02.458 INFO:tasks.workunit.client.1.smithi044.stdout:0/207: fdatasync d0/f8 0 2024-04-22T23:46:02.461 INFO:tasks.workunit.client.1.smithi044.stdout:7/236: getdents dd/d1f 0 2024-04-22T23:46:02.462 INFO:tasks.workunit.client.1.smithi044.stdout:7/237: fdatasync dd/d1a/d3c/f52 0 2024-04-22T23:46:02.462 INFO:tasks.workunit.client.1.smithi044.stdout:0/208: mkdir d0/d4c 0 2024-04-22T23:46:02.466 INFO:tasks.workunit.client.1.smithi044.stdout:7/238: rename dd/d1a/c4f to dd/d1a/c53 0 2024-04-22T23:46:02.469 INFO:tasks.workunit.client.1.smithi044.stdout:0/209: link d0/d2e/d33/f3a d0/d2e/d2f/f4d 0 2024-04-22T23:46:02.482 INFO:tasks.workunit.client.1.smithi044.stdout:7/239: creat dd/f54 x:0 0 0 2024-04-22T23:46:02.486 INFO:tasks.workunit.client.1.smithi044.stdout:7/240: mkdir dd/d55 0 2024-04-22T23:46:02.486 INFO:tasks.workunit.client.1.smithi044.stdout:7/241: rename dd to dd/d1a/d56 22 2024-04-22T23:46:02.487 INFO:tasks.workunit.client.1.smithi044.stdout:0/210: getdents d0/d2/dd/d3e/d40/d43 0 2024-04-22T23:46:02.488 INFO:tasks.workunit.client.1.smithi044.stdout:7/242: write dd/d1f/d24/f1c [882122,27928] 0 2024-04-22T23:46:02.489 INFO:tasks.workunit.client.1.smithi044.stdout:0/211: symlink d0/d2e/d33/l4e 0 2024-04-22T23:46:02.494 INFO:tasks.workunit.client.1.smithi044.stdout:7/243: truncate fc 1014662 0 2024-04-22T23:46:02.501 INFO:tasks.workunit.client.1.smithi044.stdout:0/212: creat d0/d2/dd/d3e/f4f x:0 0 0 2024-04-22T23:46:02.508 INFO:tasks.workunit.client.1.smithi044.stdout:0/213: truncate d0/d2/f6 3922821 0 2024-04-22T23:46:02.508 INFO:tasks.workunit.client.1.smithi044.stdout:7/244: dwrite dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:02.509 INFO:tasks.workunit.client.1.smithi044.stdout:0/214: dread - d0/d2/dd/d3e/f4f zero size 2024-04-22T23:46:02.509 INFO:tasks.workunit.client.1.smithi044.stdout:7/245: dread - dd/d1f/d24/f28 zero size 2024-04-22T23:46:02.514 INFO:tasks.workunit.client.1.smithi044.stdout:7/246: symlink dd/d1a/d3c/l57 0 2024-04-22T23:46:02.517 INFO:tasks.workunit.client.1.smithi044.stdout:0/215: link d0/d2/fa d0/d2e/d2f/f50 0 2024-04-22T23:46:02.522 INFO:tasks.workunit.client.1.smithi044.stdout:0/216: creat d0/d2e/d2f/f51 x:0 0 0 2024-04-22T23:46:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.531+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4700611d0 con 0x7ff4580398c0 2024-04-22T23:46:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.533+0000 7ff457fff700 1 -- 172.21.15.31:0/8347321 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff4700611d0 con 0x7ff4580398c0 2024-04-22T23:46:02.526 INFO:tasks.workunit.client.1.smithi044.stdout:0/217: creat d0/d4c/f52 x:0 0 0 2024-04-22T23:46:02.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff4580398c0 msgr2=0x7ff45803bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff4580398c0 0x7ff45803bd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff464007900 tx=0x7ff464008040 comp rx=0 tx=0).stop 2024-04-22T23:46:02.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 msgr2=0x7ff470101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470101160 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff46000bd70 tx=0x7ff46000be50 comp rx=0 tx=0).stop 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 shutdown_connections 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff4580398c0 0x7ff45803bd80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff470101800 0x7ff470101160 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 --2- 172.21.15.31:0/8347321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff470104130 0x7ff4700ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.535+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 >> 172.21.15.31:0/8347321 conn(0x7ff4700fb190 msgr2=0x7ff4700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.536+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 shutdown_connections 2024-04-22T23:46:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.536+0000 7ff4755d3700 1 -- 172.21.15.31:0/8347321 wait complete. 2024-04-22T23:46:02.529 INFO:tasks.workunit.client.1.smithi044.stdout:0/218: creat d0/d2e/f53 x:0 0 0 2024-04-22T23:46:02.532 INFO:tasks.workunit.client.1.smithi044.stdout:7/247: truncate dd/f20 3913087 0 2024-04-22T23:46:02.538 INFO:tasks.workunit.client.1.smithi044.stdout:7/248: creat dd/d55/f58 x:0 0 0 2024-04-22T23:46:02.539 INFO:tasks.workunit.client.1.smithi044.stdout:7/249: chown dd/d1f/f2c 262031209 1 2024-04-22T23:46:02.540 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:46:02.544 INFO:tasks.workunit.client.1.smithi044.stdout:7/250: creat dd/f59 x:0 0 0 2024-04-22T23:46:02.545 INFO:tasks.workunit.client.1.smithi044.stdout:0/219: getdents d0/d2e/d2f 0 2024-04-22T23:46:02.552 INFO:tasks.workunit.client.1.smithi044.stdout:7/251: dwrite dd/d27/f37 [0,4194304] 0 2024-04-22T23:46:02.553 INFO:tasks.workunit.client.1.smithi044.stdout:0/220: mknod d0/d2/c54 0 2024-04-22T23:46:02.554 INFO:tasks.workunit.client.1.smithi044.stdout:7/252: dread - dd/d1f/d24/f13 zero size 2024-04-22T23:46:02.554 INFO:tasks.workunit.client.1.smithi044.stdout:0/221: chown d0/d2e/d2f/f51 6608125 1 2024-04-22T23:46:02.558 INFO:tasks.workunit.client.1.smithi044.stdout:7/253: symlink dd/d1f/l5a 0 2024-04-22T23:46:02.566 INFO:tasks.workunit.client.1.smithi044.stdout:0/222: dwrite d0/d2e/d33/f45 [0,4194304] 0 2024-04-22T23:46:02.570 INFO:tasks.workunit.client.1.smithi044.stdout:0/223: creat d0/d2/d3d/f55 x:0 0 0 2024-04-22T23:46:02.573 INFO:tasks.workunit.client.1.smithi044.stdout:0/224: unlink d0/c48 0 2024-04-22T23:46:02.580 INFO:tasks.workunit.client.1.smithi044.stdout:0/225: fsync d0/d2e/f32 0 2024-04-22T23:46:02.581 INFO:tasks.workunit.client.1.smithi044.stdout:0/226: write d0/d2/dd/f36 [1630297,106242] 0 2024-04-22T23:46:02.583 INFO:tasks.workunit.client.1.smithi044.stdout:0/227: dread - d0/d2/dd/d3e/f4f zero size 2024-04-22T23:46:02.591 INFO:tasks.workunit.client.1.smithi044.stdout:0/228: mknod d0/d4c/c56 0 2024-04-22T23:46:02.595 INFO:tasks.workunit.client.1.smithi044.stdout:7/254: fdatasync dd/d1f/d24/f28 0 2024-04-22T23:46:02.596 INFO:tasks.workunit.client.1.smithi044.stdout:7/255: readlink dd/d1a/l1b 0 2024-04-22T23:46:02.601 INFO:tasks.workunit.client.1.smithi044.stdout:7/256: dread dd/d27/f37 [0,4194304] 0 2024-04-22T23:46:02.613 INFO:tasks.workunit.client.1.smithi044.stdout:7/257: unlink dd/d1a/l4c 0 2024-04-22T23:46:02.613 INFO:tasks.workunit.client.1.smithi044.stdout:7/258: readlink dd/l12 0 2024-04-22T23:46:02.618 INFO:tasks.workunit.client.1.smithi044.stdout:7/259: rename dd/f44 to dd/d27/f5b 0 2024-04-22T23:46:02.620 INFO:tasks.workunit.client.1.smithi044.stdout:3/217: sync 2024-04-22T23:46:02.622 INFO:tasks.workunit.client.1.smithi044.stdout:7/260: symlink dd/l5c 0 2024-04-22T23:46:02.623 INFO:tasks.workunit.client.1.smithi044.stdout:3/218: symlink d9/d14/l49 0 2024-04-22T23:46:02.626 INFO:tasks.workunit.client.1.smithi044.stdout:7/261: unlink dd/d1a/d3c/f52 0 2024-04-22T23:46:02.627 INFO:tasks.workunit.client.1.smithi044.stdout:3/219: dread d9/f19 [4194304,4194304] 0 2024-04-22T23:46:02.628 INFO:tasks.workunit.client.1.smithi044.stdout:3/220: readlink d9/db/d15/d29/l2c 0 2024-04-22T23:46:02.629 INFO:tasks.workunit.client.1.smithi044.stdout:3/221: write f5 [3567249,128543] 0 2024-04-22T23:46:02.632 INFO:tasks.workunit.client.1.smithi044.stdout:0/229: dwrite d0/d2e/f32 [0,4194304] 0 2024-04-22T23:46:02.633 INFO:tasks.workunit.client.1.smithi044.stdout:0/230: chown d0/c31 29958380 1 2024-04-22T23:46:02.638 INFO:tasks.workunit.client.1.smithi044.stdout:7/262: rename lb to dd/d1f/d24/l5d 0 2024-04-22T23:46:02.643 INFO:tasks.workunit.client.1.smithi044.stdout:3/222: chown d9/c28 42215236 1 2024-04-22T23:46:02.647 INFO:tasks.workunit.client.1.smithi044.stdout:0/231: symlink d0/d2/dd/d3e/d40/l57 0 2024-04-22T23:46:02.648 INFO:tasks.workunit.client.1.smithi044.stdout:0/232: dread - d0/d2e/f53 zero size 2024-04-22T23:46:02.657 INFO:tasks.workunit.client.1.smithi044.stdout:0/233: creat d0/d2/d3d/f58 x:0 0 0 2024-04-22T23:46:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 -- 172.21.15.31:0/2918980464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 msgr2=0x7fddd4100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 --2- 172.21.15.31:0/2918980464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd4100030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009910 tx=0x7fddc8009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 -- 172.21.15.31:0/2918980464 shutdown_connections 2024-04-22T23:46:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 --2- 172.21.15.31:0/2918980464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd4100e10 0x7fddd4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 --2- 172.21.15.31:0/2918980464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd4100030 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 -- 172.21.15.31:0/2918980464 >> 172.21.15.31:0/2918980464 conn(0x7fddd40fb190 msgr2=0x7fddd40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 -- 172.21.15.31:0/2918980464 shutdown_connections 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 -- 172.21.15.31:0/2918980464 wait complete. 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 Processor -- start 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.666+0000 7fdddace4700 1 -- start start 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fdddace4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd40690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fdddace4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd4100e10 0x7fddd419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fdddace4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419b800 con 0x7fddd40ffc10 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fdddace4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419b940 con 0x7fddd4100e10 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd40690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:02.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd40690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37384/0 (socket says 172.21.15.31:37384) 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 -- 172.21.15.31:0/962278102 learned_addr learned my addr 172.21.15.31:0/962278102 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd3fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd4100e10 0x7fddd419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 -- 172.21.15.31:0/962278102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd4100e10 msgr2=0x7fddd419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd4100e10 0x7fddd419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 -- 172.21.15.31:0/962278102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80095f0 con 0x7fddd40ffc10 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.667+0000 7fddd8a80700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd40690e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fddc8000c00 tx=0x7fddc800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.668+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddc801d070 con 0x7fddd40ffc10 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.668+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd419bbc0 con 0x7fddd40ffc10 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.668+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fddc8005dc0 con 0x7fddd40ffc10 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.668+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddc8022470 con 0x7fddd40ffc10 2024-04-22T23:46:02.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.668+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd419c0b0 con 0x7fddd40ffc10 2024-04-22T23:46:02.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/223: dread f5 [0,4194304] 0 2024-04-22T23:46:02.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/224: write f7 [3277431,3619] 0 2024-04-22T23:46:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.669+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 28) v1 ==== 46697+0+0 (secure 0 0 0) 0x7fddc8017400 con 0x7fddd40ffc10 2024-04-22T23:46:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.669+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd4108700 con 0x7fddd40ffc10 2024-04-22T23:46:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.669+0000 7fddd1ffb700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fddc40398c0 0x7fddc403bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.669+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fddc804f2e0 con 0x7fddd40ffc10 2024-04-22T23:46:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.669+0000 7fddd3fff700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fddc40398c0 0x7fddc403bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:02.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.670+0000 7fddd3fff700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fddc40398c0 0x7fddc403bd80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddd4069660 tx=0x7fddc0009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:02.664 INFO:tasks.workunit.client.1.smithi044.stdout:0/234: unlink d0/d2/c28 0 2024-04-22T23:46:02.665 INFO:tasks.workunit.client.1.smithi044.stdout:0/235: read d0/d2/fa [2386899,37611] 0 2024-04-22T23:46:02.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.675+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fddc8020a60 con 0x7fddd40ffc10 2024-04-22T23:46:02.669 INFO:tasks.workunit.client.1.smithi044.stdout:0/236: mkdir d0/d4c/d59 0 2024-04-22T23:46:02.670 INFO:tasks.workunit.client.1.smithi044.stdout:7/263: dwrite dd/f20 [0,4194304] 0 2024-04-22T23:46:02.672 INFO:tasks.workunit.client.1.smithi044.stdout:3/225: link f7 d9/db/d22/d38/f4a 0 2024-04-22T23:46:02.674 INFO:tasks.workunit.client.1.smithi044.stdout:7/264: mkdir dd/d55/d5e 0 2024-04-22T23:46:02.675 INFO:tasks.workunit.client.1.smithi044.stdout:0/237: mkdir d0/d4c/d59/d5a 0 2024-04-22T23:46:02.675 INFO:tasks.workunit.client.1.smithi044.stdout:0/238: stat d0/d2/c54 0 2024-04-22T23:46:02.677 INFO:tasks.workunit.client.1.smithi044.stdout:7/265: chown dd/d1f/l42 14927 1 2024-04-22T23:46:02.678 INFO:tasks.workunit.client.1.smithi044.stdout:0/239: mknod d0/c5b 0 2024-04-22T23:46:02.681 INFO:tasks.workunit.client.1.smithi044.stdout:0/240: creat d0/d2/d3d/f5c x:0 0 0 2024-04-22T23:46:02.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/266: link dd/l43 dd/d55/l5f 0 2024-04-22T23:46:02.687 INFO:tasks.workunit.client.1.smithi044.stdout:7/267: truncate dd/d27/f5b 913456 0 2024-04-22T23:46:02.698 INFO:tasks.workunit.client.1.smithi044.stdout:3/226: dread d9/db/f13 [0,4194304] 0 2024-04-22T23:46:02.699 INFO:tasks.workunit.client.1.smithi044.stdout:3/227: creat d9/db/d15/d29/f4b x:0 0 0 2024-04-22T23:46:02.707 INFO:tasks.workunit.client.1.smithi044.stdout:0/241: dwrite d0/d2/dd/f13 [0,4194304] 0 2024-04-22T23:46:02.707 INFO:tasks.workunit.client.1.smithi044.stdout:3/228: getdents d9/db/d15/d2e 0 2024-04-22T23:46:02.708 INFO:tasks.workunit.client.1.smithi044.stdout:3/229: chown d9/d14/f43 20481 1 2024-04-22T23:46:02.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:02 smithi044 ceph-mon[34463]: mgrmap e28: smithi031.fazdcv(active, since 3s) 2024-04-22T23:46:02.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/230: mkdir d9/db/d15/d25/d4c 0 2024-04-22T23:46:02.897 INFO:tasks.workunit.client.1.smithi044.stdout:8/222: sync 2024-04-22T23:46:02.901 INFO:tasks.workunit.client.1.smithi044.stdout:8/223: dread db/d14/f3e [0,4194304] 0 2024-04-22T23:46:02.903 INFO:tasks.workunit.client.1.smithi044.stdout:8/224: link db/d14/d24/f30 db/d14/d24/f59 0 2024-04-22T23:46:02.904 INFO:tasks.workunit.client.1.smithi044.stdout:8/225: write db/f3d [799524,61904] 0 2024-04-22T23:46:02.907 INFO:tasks.workunit.client.1.smithi044.stdout:8/226: creat db/d14/f5a x:0 0 0 2024-04-22T23:46:02.909 INFO:tasks.workunit.client.1.smithi044.stdout:8/227: mknod db/d14/c5b 0 2024-04-22T23:46:02.911 INFO:tasks.workunit.client.1.smithi044.stdout:8/228: rename db/d14/d24/f55 to db/d14/d24/f5c 0 2024-04-22T23:46:02.913 INFO:tasks.workunit.client.1.smithi044.stdout:8/229: fsync db/d14/f3e 0 2024-04-22T23:46:02.916 INFO:tasks.workunit.client.1.smithi044.stdout:8/230: unlink db/d14/d24/f39 0 2024-04-22T23:46:02.928 INFO:tasks.workunit.client.1.smithi044.stdout:8/231: getdents db/d52 0 2024-04-22T23:46:02.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.938+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd4105760 con 0x7fddc40398c0 2024-04-22T23:46:02.930 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:02 smithi031 ceph-mon[26721]: mgrmap e28: smithi031.fazdcv(active, since 3s) 2024-04-22T23:46:02.931 INFO:tasks.workunit.client.1.smithi044.stdout:8/232: fdatasync db/f10 0 2024-04-22T23:46:02.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.940+0000 7fddd1ffb700 1 -- 172.21.15.31:0/962278102 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fddd4105760 con 0x7fddc40398c0 2024-04-22T23:46:02.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fddc40398c0 msgr2=0x7fddc403bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fddc40398c0 0x7fddc403bd80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddd4069660 tx=0x7fddc0009260 comp rx=0 tx=0).stop 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 msgr2=0x7fddd40690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd40690e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fddc8000c00 tx=0x7fddc800f690 comp rx=0 tx=0).stop 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 shutdown_connections 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fddc40398c0 0x7fddc403bd80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddd40ffc10 0x7fddd40690e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 --2- 172.21.15.31:0/962278102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddd4100e10 0x7fddd419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 >> 172.21.15.31:0/962278102 conn(0x7fddd40fb190 msgr2=0x7fddd4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 shutdown_connections 2024-04-22T23:46:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:02.942+0000 7fdddace4700 1 -- 172.21.15.31:0/962278102 wait complete. 2024-04-22T23:46:02.936 INFO:tasks.workunit.client.1.smithi044.stdout:8/233: mknod db/d14/c5d 0 2024-04-22T23:46:02.939 INFO:tasks.workunit.client.1.smithi044.stdout:8/234: creat db/d52/f5e x:0 0 0 2024-04-22T23:46:02.939 INFO:tasks.workunit.client.1.smithi044.stdout:8/235: readlink db/l4d 0 2024-04-22T23:46:02.945 INFO:tasks.workunit.client.1.smithi044.stdout:8/236: dwrite db/f22 [0,4194304] 0 2024-04-22T23:46:02.963 INFO:tasks.workunit.client.1.smithi044.stdout:8/237: dread db/d14/d24/f43 [0,4194304] 0 2024-04-22T23:46:02.964 INFO:tasks.workunit.client.1.smithi044.stdout:8/238: dread - db/d14/f36 zero size 2024-04-22T23:46:02.970 INFO:tasks.workunit.client.1.smithi044.stdout:8/239: getdents db/d14/d24 0 2024-04-22T23:46:02.972 INFO:tasks.workunit.client.1.smithi044.stdout:8/240: fsync db/f27 0 2024-04-22T23:46:02.974 INFO:tasks.workunit.client.1.smithi044.stdout:8/241: symlink db/d52/l5f 0 2024-04-22T23:46:02.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/242: readlink db/d52/l5f 0 2024-04-22T23:46:02.978 INFO:tasks.workunit.client.1.smithi044.stdout:8/243: unlink db/f1d 0 2024-04-22T23:46:02.979 INFO:tasks.workunit.client.1.smithi044.stdout:8/244: dread - db/d14/f36 zero size 2024-04-22T23:46:02.980 INFO:tasks.workunit.client.1.smithi044.stdout:8/245: dread - db/d14/d24/f49 zero size 2024-04-22T23:46:02.981 INFO:tasks.workunit.client.1.smithi044.stdout:8/246: write db/d14/f25 [506505,22969] 0 2024-04-22T23:46:02.981 INFO:tasks.workunit.client.1.smithi044.stdout:8/247: readlink db/l4d 0 2024-04-22T23:46:02.984 INFO:tasks.workunit.client.1.smithi044.stdout:8/248: rename db/f2c to db/d14/f60 0 2024-04-22T23:46:02.986 INFO:tasks.workunit.client.1.smithi044.stdout:8/249: symlink db/d14/d24/l61 0 2024-04-22T23:46:02.988 INFO:tasks.workunit.client.1.smithi044.stdout:8/250: rename db/d14/c5d to db/d14/d24/c62 0 2024-04-22T23:46:02.990 INFO:tasks.workunit.client.1.smithi044.stdout:8/251: creat db/d14/f63 x:0 0 0 2024-04-22T23:46:02.997 INFO:tasks.workunit.client.1.smithi044.stdout:8/252: dwrite db/f3d [0,4194304] 0 2024-04-22T23:46:02.998 INFO:tasks.workunit.client.1.smithi044.stdout:8/253: write db/d14/d24/f51 [949558,15813] 0 2024-04-22T23:46:02.999 INFO:tasks.workunit.client.1.smithi044.stdout:8/254: readlink db/d52/l5f 0 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:7/268: sync 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:1/227: sync 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:2/184: sync 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:9/145: sync 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:6/191: sync 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:5/208: sync 2024-04-22T23:46:03.000 INFO:tasks.workunit.client.1.smithi044.stdout:4/292: sync 2024-04-22T23:46:03.002 INFO:tasks.workunit.client.1.smithi044.stdout:8/255: symlink db/d14/d24/l64 0 2024-04-22T23:46:03.002 INFO:tasks.workunit.client.1.smithi044.stdout:9/146: write d2/f2c [2244171,56655] 0 2024-04-22T23:46:03.004 INFO:tasks.workunit.client.1.smithi044.stdout:7/269: rmdir dd/d1a 39 2024-04-22T23:46:03.008 INFO:tasks.workunit.client.1.smithi044.stdout:4/293: creat d5/d42/f61 x:0 0 0 2024-04-22T23:46:03.010 INFO:tasks.workunit.client.1.smithi044.stdout:6/192: mkdir d5/dd/d1d/d23/d3d 0 2024-04-22T23:46:03.016 INFO:tasks.workunit.client.1.smithi044.stdout:4/294: truncate d5/d31/d44/d20/f23 4682486 0 2024-04-22T23:46:03.016 INFO:tasks.workunit.client.1.smithi044.stdout:5/209: rename d15/d16/d1d/d22/f3d to d15/d16/d1d/d22/d2a/d3a/f41 0 2024-04-22T23:46:03.017 INFO:tasks.workunit.client.1.smithi044.stdout:8/256: dread db/d14/d24/f4e [0,4194304] 0 2024-04-22T23:46:03.019 INFO:tasks.workunit.client.1.smithi044.stdout:2/185: dwrite d8/d27/f2a [0,4194304] 0 2024-04-22T23:46:03.019 INFO:tasks.workunit.client.1.smithi044.stdout:9/147: dwrite d2/d5/f1f [0,4194304] 0 2024-04-22T23:46:03.019 INFO:tasks.workunit.client.1.smithi044.stdout:1/228: creat d1/d8/d20/d47/d4a/f54 x:0 0 0 2024-04-22T23:46:03.024 INFO:tasks.workunit.client.1.smithi044.stdout:5/210: stat d15/d16/d1d/d22/d2a 0 2024-04-22T23:46:03.024 INFO:tasks.workunit.client.1.smithi044.stdout:9/148: readlink d2/le 0 2024-04-22T23:46:03.026 INFO:tasks.workunit.client.1.smithi044.stdout:4/295: dread d5/d31/d44/f30 [0,4194304] 0 2024-04-22T23:46:03.027 INFO:tasks.workunit.client.1.smithi044.stdout:2/186: creat d8/f38 x:0 0 0 2024-04-22T23:46:03.029 INFO:tasks.workunit.client.1.smithi044.stdout:8/257: rmdir db/d14/d24 39 2024-04-22T23:46:03.038 INFO:tasks.workunit.client.1.smithi044.stdout:5/211: dread d15/f1e [0,4194304] 0 2024-04-22T23:46:03.038 INFO:tasks.workunit.client.1.smithi044.stdout:4/296: dread d5/d31/f17 [0,4194304] 0 2024-04-22T23:46:03.039 INFO:tasks.workunit.client.1.smithi044.stdout:2/187: creat d8/d19/f39 x:0 0 0 2024-04-22T23:46:03.042 INFO:tasks.workunit.client.1.smithi044.stdout:2/188: read d8/fd [3130872,32075] 0 2024-04-22T23:46:03.042 INFO:tasks.workunit.client.1.smithi044.stdout:9/149: dwrite d2/d6/f33 [0,4194304] 0 2024-04-22T23:46:03.044 INFO:tasks.workunit.client.1.smithi044.stdout:9/150: write d2/d5/f30 [162802,36775] 0 2024-04-22T23:46:03.050 INFO:tasks.workunit.client.1.smithi044.stdout:7/270: dread dd/d1f/f34 [0,4194304] 0 2024-04-22T23:46:03.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/271: fsync dd/d1f/d24/f3f 0 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.069+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3681123223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4100ca0 msgr2=0x7fc7e41010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.069+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3681123223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41010c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009910 tx=0x7fc7d4009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.069+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3681123223 shutdown_connections 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.069+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3681123223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.070+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3681123223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41010c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.070+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3681123223 >> 172.21.15.31:0/3681123223 conn(0x7fc7e40fc260 msgr2=0x7fc7e40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:03.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.070+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3681123223 shutdown_connections 2024-04-22T23:46:03.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.070+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3681123223 wait complete. 2024-04-22T23:46:03.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.071+0000 7fc7ea9d2700 1 Processor -- start 2024-04-22T23:46:03.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.071+0000 7fc7ea9d2700 1 -- start start 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.071+0000 7fc7ea9d2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.071+0000 7fc7ea9d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7ea9d2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4197030 con 0x7fc7e4101ea0 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7ea9d2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e4197170 con 0x7fc7e4100ca0 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37394/0 (socket says 172.21.15.31:37394) 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e37fe700 1 -- 172.21.15.31:0/3624347210 learned_addr learned my addr 172.21.15.31:0/3624347210 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e3fff700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e3fff700 1 -- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4101ea0 msgr2=0x7fc7e4196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e3fff700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.072+0000 7fc7e3fff700 1 -- 172.21.15.31:0/3624347210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40095f0 con 0x7fc7e4100ca0 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.073+0000 7fc7e3fff700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41964d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.073+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7d401d070 con 0x7fc7e4100ca0 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.073+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc7d4005dc0 con 0x7fc7e4100ca0 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.073+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7d4017400 con 0x7fc7e4100ca0 2024-04-22T23:46:03.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.073+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e419bbc0 con 0x7fc7e4100ca0 2024-04-22T23:46:03.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.073+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e419bff0 con 0x7fc7e4100ca0 2024-04-22T23:46:03.066 INFO:tasks.workunit.client.1.smithi044.stdout:2/189: symlink d8/d19/d1c/l3a 0 2024-04-22T23:46:03.067 INFO:tasks.workunit.client.1.smithi044.stdout:2/190: stat d8/ff 0 2024-04-22T23:46:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.074+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 28) v1 ==== 46697+0+0 (secure 0 0 0) 0x7fc7d4017580 con 0x7fc7e4100ca0 2024-04-22T23:46:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.074+0000 7fc7e17fa700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7cc0398c0 0x7fc7cc03bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.074+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc7d404ec90 con 0x7fc7e4100ca0 2024-04-22T23:46:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.074+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e4109800 con 0x7fc7e4100ca0 2024-04-22T23:46:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.075+0000 7fc7e37fe700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7cc0398c0 0x7fc7cc03bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:03.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.075+0000 7fc7e37fe700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7cc0398c0 0x7fc7cc03bd80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009d20 tx=0x7fc7d8009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:03.068 INFO:tasks.workunit.client.1.smithi044.stdout:5/212: write f5 [2783186,34407] 0 2024-04-22T23:46:03.070 INFO:tasks.workunit.client.1.smithi044.stdout:5/213: write d15/d16/d1d/f3f [4342432,88509] 0 2024-04-22T23:46:03.074 INFO:tasks.workunit.client.1.smithi044.stdout:9/151: write d2/f19 [905858,48279] 0 2024-04-22T23:46:03.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.084+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fc7d4027070 con 0x7fc7e4100ca0 2024-04-22T23:46:03.077 INFO:tasks.workunit.client.1.smithi044.stdout:1/229: getdents d1/d8/d20/d47/d4a 0 2024-04-22T23:46:03.078 INFO:tasks.workunit.client.1.smithi044.stdout:2/191: unlink d8/d19/d1c/f35 0 2024-04-22T23:46:03.080 INFO:tasks.workunit.client.1.smithi044.stdout:7/272: mkdir dd/d55/d5e/d60 0 2024-04-22T23:46:03.081 INFO:tasks.workunit.client.1.smithi044.stdout:5/214: creat d15/d16/d1d/f42 x:0 0 0 2024-04-22T23:46:03.082 INFO:tasks.workunit.client.1.smithi044.stdout:9/152: mkdir d2/d5/d36 0 2024-04-22T23:46:03.085 INFO:tasks.workunit.client.1.smithi044.stdout:8/258: rename db/d14/d24/c56 to db/c65 0 2024-04-22T23:46:03.085 INFO:tasks.workunit.client.1.smithi044.stdout:5/215: dread f5 [0,4194304] 0 2024-04-22T23:46:03.087 INFO:tasks.workunit.client.1.smithi044.stdout:8/259: truncate db/f3d 4565028 0 2024-04-22T23:46:03.087 INFO:tasks.workunit.client.1.smithi044.stdout:8/260: chown db/f10 24499267 1 2024-04-22T23:46:03.089 INFO:tasks.workunit.client.1.smithi044.stdout:2/192: symlink d8/l3b 0 2024-04-22T23:46:03.090 INFO:tasks.workunit.client.1.smithi044.stdout:2/193: chown d8/l3b 33 1 2024-04-22T23:46:03.092 INFO:tasks.workunit.client.1.smithi044.stdout:9/153: mkdir d2/d6/d37 0 2024-04-22T23:46:03.094 INFO:tasks.workunit.client.1.smithi044.stdout:5/216: dwrite f11 [4194304,4194304] 0 2024-04-22T23:46:03.095 INFO:tasks.workunit.client.1.smithi044.stdout:1/230: symlink d1/d8/d1c/d23/d48/l55 0 2024-04-22T23:46:03.095 INFO:tasks.workunit.client.1.smithi044.stdout:5/217: stat d15 0 2024-04-22T23:46:03.097 INFO:tasks.workunit.client.1.smithi044.stdout:7/273: dread f7 [0,4194304] 0 2024-04-22T23:46:03.097 INFO:tasks.workunit.client.1.smithi044.stdout:1/231: write d1/d8/f26 [1359531,126420] 0 2024-04-22T23:46:03.103 INFO:tasks.workunit.client.1.smithi044.stdout:2/194: symlink d8/d19/d36/l3c 0 2024-04-22T23:46:03.104 INFO:tasks.workunit.client.1.smithi044.stdout:5/218: mkdir d15/d16/d1d/d22/d2a/d3a/d43 0 2024-04-22T23:46:03.105 INFO:tasks.workunit.client.1.smithi044.stdout:7/274: dwrite dd/d1f/d24/f39 [0,4194304] 0 2024-04-22T23:46:03.105 INFO:tasks.workunit.client.1.smithi044.stdout:5/219: chown f5 98 1 2024-04-22T23:46:03.107 INFO:tasks.workunit.client.1.smithi044.stdout:1/232: symlink d1/d8/d20/d45/l56 0 2024-04-22T23:46:03.107 INFO:tasks.workunit.client.1.smithi044.stdout:1/233: chown d1/d8/d1c 84 1 2024-04-22T23:46:03.111 INFO:tasks.workunit.client.1.smithi044.stdout:8/261: link db/f3c db/d14/f66 0 2024-04-22T23:46:03.112 INFO:tasks.workunit.client.1.smithi044.stdout:8/262: fdatasync db/ff 0 2024-04-22T23:46:03.113 INFO:tasks.workunit.client.1.smithi044.stdout:2/195: creat d8/d19/d36/f3d x:0 0 0 2024-04-22T23:46:03.114 INFO:tasks.workunit.client.1.smithi044.stdout:8/263: write db/d14/d24/f49 [422952,102514] 0 2024-04-22T23:46:03.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/264: write db/f48 [898106,128750] 0 2024-04-22T23:46:03.120 INFO:tasks.workunit.client.1.smithi044.stdout:5/220: dread ff [0,4194304] 0 2024-04-22T23:46:03.126 INFO:tasks.workunit.client.1.smithi044.stdout:7/275: mkdir dd/d1a/d61 0 2024-04-22T23:46:03.130 INFO:tasks.workunit.client.1.smithi044.stdout:9/154: getdents d2 0 2024-04-22T23:46:03.131 INFO:tasks.workunit.client.1.smithi044.stdout:2/196: dwrite d8/fd [0,4194304] 0 2024-04-22T23:46:03.134 INFO:tasks.workunit.client.1.smithi044.stdout:5/221: chown l9 27302378 1 2024-04-22T23:46:03.135 INFO:tasks.workunit.client.1.smithi044.stdout:1/234: symlink d1/d8/d1c/l57 0 2024-04-22T23:46:03.139 INFO:tasks.workunit.client.1.smithi044.stdout:7/276: creat dd/d27/f62 x:0 0 0 2024-04-22T23:46:03.140 INFO:tasks.workunit.client.1.smithi044.stdout:9/155: dwrite d2/f3 [4194304,4194304] 0 2024-04-22T23:46:03.141 INFO:tasks.workunit.client.1.smithi044.stdout:8/265: mknod db/d14/c67 0 2024-04-22T23:46:03.146 INFO:tasks.workunit.client.1.smithi044.stdout:1/235: creat d1/d8/d1c/d23/d3e/f58 x:0 0 0 2024-04-22T23:46:03.148 INFO:tasks.workunit.client.1.smithi044.stdout:1/236: write d1/d14/f52 [356044,105454] 0 2024-04-22T23:46:03.150 INFO:tasks.workunit.client.1.smithi044.stdout:1/237: chown d1/d8/d24/d4b 1046 1 2024-04-22T23:46:03.151 INFO:tasks.workunit.client.1.smithi044.stdout:9/156: dwrite d2/f19 [0,4194304] 0 2024-04-22T23:46:03.153 INFO:tasks.workunit.client.1.smithi044.stdout:8/266: unlink db/d14/c5b 0 2024-04-22T23:46:03.153 INFO:tasks.workunit.client.1.smithi044.stdout:5/222: mknod d15/d16/d1d/d22/d2a/d3a/d43/c44 0 2024-04-22T23:46:03.154 INFO:tasks.workunit.client.1.smithi044.stdout:8/267: chown db/d14/d24/l40 19149722 1 2024-04-22T23:46:03.155 INFO:tasks.workunit.client.1.smithi044.stdout:8/268: chown db/d14/f15 12800367 1 2024-04-22T23:46:03.157 INFO:tasks.workunit.client.1.smithi044.stdout:8/269: chown db/d14/d24/f3b 443 1 2024-04-22T23:46:03.157 INFO:tasks.workunit.client.1.smithi044.stdout:9/157: rename d2/d5/d36 to d2/dc/d1d/d34/d38 0 2024-04-22T23:46:03.158 INFO:tasks.workunit.client.1.smithi044.stdout:0/242: sync 2024-04-22T23:46:03.161 INFO:tasks.workunit.client.1.smithi044.stdout:9/158: symlink d2/d6/l39 0 2024-04-22T23:46:03.162 INFO:tasks.workunit.client.1.smithi044.stdout:8/270: dread db/f3d [0,4194304] 0 2024-04-22T23:46:03.163 INFO:tasks.workunit.client.1.smithi044.stdout:1/238: creat d1/d8/d1c/f59 x:0 0 0 2024-04-22T23:46:03.164 INFO:tasks.workunit.client.1.smithi044.stdout:0/243: mkdir d0/d2e/d33/d5d 0 2024-04-22T23:46:03.164 INFO:tasks.workunit.client.1.smithi044.stdout:8/271: write db/d14/d24/f58 [330140,66401] 0 2024-04-22T23:46:03.166 INFO:tasks.workunit.client.1.smithi044.stdout:5/223: symlink d15/l45 0 2024-04-22T23:46:03.166 INFO:tasks.workunit.client.1.smithi044.stdout:9/159: rmdir d2 39 2024-04-22T23:46:03.167 INFO:tasks.workunit.client.1.smithi044.stdout:8/272: write db/f38 [892039,68688] 0 2024-04-22T23:46:03.168 INFO:tasks.workunit.client.1.smithi044.stdout:1/239: truncate d1/d8/d1c/f59 486405 0 2024-04-22T23:46:03.169 INFO:tasks.workunit.client.1.smithi044.stdout:0/244: creat d0/d2e/d2f/f5e x:0 0 0 2024-04-22T23:46:03.169 INFO:tasks.workunit.client.1.smithi044.stdout:2/197: dread f7 [0,4194304] 0 2024-04-22T23:46:03.171 INFO:tasks.workunit.client.1.smithi044.stdout:0/245: write d0/d2/dd/f2a [314732,37113] 0 2024-04-22T23:46:03.172 INFO:tasks.workunit.client.1.smithi044.stdout:0/246: chown d0/d2/d3d/c34 6 1 2024-04-22T23:46:03.173 INFO:tasks.workunit.client.1.smithi044.stdout:0/247: chown d0/d2e/d33/c38 42828 1 2024-04-22T23:46:03.174 INFO:tasks.workunit.client.1.smithi044.stdout:5/224: symlink d15/d16/d2c/l46 0 2024-04-22T23:46:03.180 INFO:tasks.workunit.client.1.smithi044.stdout:0/248: dwrite d0/d2/f25 [4194304,4194304] 0 2024-04-22T23:46:03.182 INFO:tasks.workunit.client.1.smithi044.stdout:0/249: chown d0/d2/dd/d3e/d40/l57 1038164 1 2024-04-22T23:46:03.185 INFO:tasks.workunit.client.1.smithi044.stdout:2/198: creat d8/d19/d36/f3e x:0 0 0 2024-04-22T23:46:03.190 INFO:tasks.workunit.client.1.smithi044.stdout:5/225: mkdir d15/d16/d1d/d22/d2a/d3a/d43/d47 0 2024-04-22T23:46:03.192 INFO:tasks.workunit.client.1.smithi044.stdout:2/199: dwrite d8/fd [0,4194304] 0 2024-04-22T23:46:03.193 INFO:tasks.workunit.client.1.smithi044.stdout:8/273: mkdir db/d68 0 2024-04-22T23:46:03.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/160: getdents d2/dc/d1d/d34/d38 0 2024-04-22T23:46:03.202 INFO:tasks.workunit.client.1.smithi044.stdout:2/200: unlink d8/l3b 0 2024-04-22T23:46:03.205 INFO:tasks.workunit.client.1.smithi044.stdout:9/161: creat d2/dc/f3a x:0 0 0 2024-04-22T23:46:03.206 INFO:tasks.workunit.client.1.smithi044.stdout:9/162: chown d2/d6/l35 30591291 1 2024-04-22T23:46:03.207 INFO:tasks.workunit.client.1.smithi044.stdout:5/226: symlink d15/d3c/l48 0 2024-04-22T23:46:03.209 INFO:tasks.workunit.client.1.smithi044.stdout:2/201: symlink d8/d19/d1c/l3f 0 2024-04-22T23:46:03.210 INFO:tasks.workunit.client.1.smithi044.stdout:0/250: dread d0/d2/f6 [0,4194304] 0 2024-04-22T23:46:03.211 INFO:tasks.workunit.client.1.smithi044.stdout:9/163: creat d2/d5/f3b x:0 0 0 2024-04-22T23:46:03.213 INFO:tasks.workunit.client.1.smithi044.stdout:0/251: truncate d0/d2/d3d/f58 551136 0 2024-04-22T23:46:03.214 INFO:tasks.workunit.client.1.smithi044.stdout:8/274: rename db/f4b to db/f69 0 2024-04-22T23:46:03.218 INFO:tasks.workunit.client.1.smithi044.stdout:2/202: mkdir d8/d19/d1c/d40 0 2024-04-22T23:46:03.218 INFO:tasks.workunit.client.1.smithi044.stdout:5/227: dread f5 [4194304,4194304] 0 2024-04-22T23:46:03.220 INFO:tasks.workunit.client.1.smithi044.stdout:9/164: dwrite d2/d5/f1f [0,4194304] 0 2024-04-22T23:46:03.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/252: rename d0/d2/d3d/f20 to d0/d2/dd/d3e/d40/d43/f5f 0 2024-04-22T23:46:03.226 INFO:tasks.workunit.client.1.smithi044.stdout:1/240: write d1/d8/f1e [664966,81535] 0 2024-04-22T23:46:03.227 INFO:tasks.workunit.client.1.smithi044.stdout:2/203: creat d8/f41 x:0 0 0 2024-04-22T23:46:03.229 INFO:tasks.workunit.client.1.smithi044.stdout:5/228: creat d15/d16/d1d/f49 x:0 0 0 2024-04-22T23:46:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:2/204: truncate d8/d27/f2c 152558 0 2024-04-22T23:46:03.230 INFO:tasks.workunit.client.1.smithi044.stdout:5/229: read f5 [643439,71580] 0 2024-04-22T23:46:03.238 INFO:tasks.workunit.client.1.smithi044.stdout:5/230: dwrite f11 [4194304,4194304] 0 2024-04-22T23:46:03.244 INFO:tasks.workunit.client.1.smithi044.stdout:2/205: fdatasync d8/d19/f37 0 2024-04-22T23:46:03.247 INFO:tasks.workunit.client.1.smithi044.stdout:8/275: write db/f12 [4899510,110484] 0 2024-04-22T23:46:03.248 INFO:tasks.workunit.client.1.smithi044.stdout:5/231: symlink d15/d16/d1d/l4a 0 2024-04-22T23:46:03.251 INFO:tasks.workunit.client.1.smithi044.stdout:1/241: creat d1/d8/d24/d4b/f5a x:0 0 0 2024-04-22T23:46:03.254 INFO:tasks.workunit.client.1.smithi044.stdout:9/165: rmdir d2/d6/d37 0 2024-04-22T23:46:03.261 INFO:tasks.workunit.client.1.smithi044.stdout:0/253: dread d0/d2/dd/f11 [0,4194304] 0 2024-04-22T23:46:03.267 INFO:tasks.workunit.client.1.smithi044.stdout:5/232: dread d15/d16/d1d/d22/f23 [0,4194304] 0 2024-04-22T23:46:03.268 INFO:tasks.workunit.client.1.smithi044.stdout:2/206: rmdir d8/d19/d1c/d40 0 2024-04-22T23:46:03.272 INFO:tasks.workunit.client.1.smithi044.stdout:8/276: creat db/f6a x:0 0 0 2024-04-22T23:46:03.273 INFO:tasks.workunit.client.1.smithi044.stdout:8/277: chown db/f21 2537 1 2024-04-22T23:46:03.275 INFO:tasks.workunit.client.1.smithi044.stdout:5/233: fsync d15/d16/d1d/f26 0 2024-04-22T23:46:03.278 INFO:tasks.workunit.client.1.smithi044.stdout:0/254: getdents d0/d4c/d59/d5a 0 2024-04-22T23:46:03.279 INFO:tasks.workunit.client.1.smithi044.stdout:1/242: rename d1/d8/d20/d45/f2c to d1/d8/d1c/f5b 0 2024-04-22T23:46:03.284 INFO:tasks.workunit.client.1.smithi044.stdout:5/234: rmdir d15 39 2024-04-22T23:46:03.286 INFO:tasks.workunit.client.1.smithi044.stdout:0/255: mkdir d0/d60 0 2024-04-22T23:46:03.288 INFO:tasks.workunit.client.1.smithi044.stdout:9/166: link d2/d9/cb d2/d9/c3c 0 2024-04-22T23:46:03.289 INFO:tasks.workunit.client.1.smithi044.stdout:8/278: symlink db/d68/l6b 0 2024-04-22T23:46:03.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/279: chown f5 2 1 2024-04-22T23:46:03.292 INFO:tasks.workunit.client.1.smithi044.stdout:2/207: dwrite d8/d19/f20 [0,4194304] 0 2024-04-22T23:46:03.294 INFO:tasks.workunit.client.1.smithi044.stdout:1/243: rename d1/d8/d20/d45/c37 to d1/d8/d24/c5c 0 2024-04-22T23:46:03.295 INFO:tasks.workunit.client.1.smithi044.stdout:5/235: chown d15/d16/d1d/d22/d2a/d3a/f41 5 1 2024-04-22T23:46:03.295 INFO:tasks.workunit.client.1.smithi044.stdout:8/280: fsync db/f3d 0 2024-04-22T23:46:03.297 INFO:tasks.workunit.client.1.smithi044.stdout:0/256: dwrite d0/d2e/d2f/f51 [0,4194304] 0 2024-04-22T23:46:03.301 INFO:tasks.workunit.client.1.smithi044.stdout:1/244: mkdir d1/d8/d20/d45/d5d 0 2024-04-22T23:46:03.302 INFO:tasks.workunit.client.1.smithi044.stdout:1/245: write d1/d8/f12 [2554855,116322] 0 2024-04-22T23:46:03.310 INFO:tasks.workunit.client.1.smithi044.stdout:0/257: mkdir d0/d2e/d2f/d61 0 2024-04-22T23:46:03.311 INFO:tasks.workunit.client.1.smithi044.stdout:8/281: mkdir db/d6c 0 2024-04-22T23:46:03.317 INFO:tasks.workunit.client.1.smithi044.stdout:8/282: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:03.319 INFO:tasks.workunit.client.1.smithi044.stdout:0/258: symlink d0/d2/dd/d3e/d40/l62 0 2024-04-22T23:46:03.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/259: chown d0/d2/dd/f1d 244 1 2024-04-22T23:46:03.321 INFO:tasks.workunit.client.1.smithi044.stdout:3/231: sync 2024-04-22T23:46:03.325 INFO:tasks.workunit.client.1.smithi044.stdout:1/246: symlink d1/d8/d1c/d23/d3e/d4d/l5e 0 2024-04-22T23:46:03.327 INFO:tasks.workunit.client.1.smithi044.stdout:8/283: chown db/fd 1755976 1 2024-04-22T23:46:03.330 INFO:tasks.workunit.client.1.smithi044.stdout:3/232: write f5 [3387580,38169] 0 2024-04-22T23:46:03.331 INFO:tasks.workunit.client.1.smithi044.stdout:3/233: dread - d9/db/d15/d2e/f32 zero size 2024-04-22T23:46:03.332 INFO:tasks.workunit.client.1.smithi044.stdout:1/247: unlink d1/d14/c31 0 2024-04-22T23:46:03.334 INFO:tasks.workunit.client.1.smithi044.stdout:1/248: write d1/d8/d20/d47/d4a/f54 [148060,108226] 0 2024-04-22T23:46:03.335 INFO:tasks.workunit.client.1.smithi044.stdout:2/208: truncate f6 413528 0 2024-04-22T23:46:03.336 INFO:tasks.workunit.client.1.smithi044.stdout:8/284: fsync db/d14/f60 0 2024-04-22T23:46:03.340 INFO:tasks.workunit.client.1.smithi044.stdout:5/236: dwrite d15/f1e [0,4194304] 0 2024-04-22T23:46:03.347 INFO:tasks.workunit.client.1.smithi044.stdout:3/234: creat d9/db/d15/d25/d4c/f4d x:0 0 0 2024-04-22T23:46:03.350 INFO:tasks.workunit.client.1.smithi044.stdout:8/285: dwrite db/f12 [0,4194304] 0 2024-04-22T23:46:03.351 INFO:tasks.workunit.client.1.smithi044.stdout:8/286: chown db/f10 64641 1 2024-04-22T23:46:03.351 INFO:tasks.workunit.client.1.smithi044.stdout:1/249: dwrite d1/d8/d1c/d23/d3e/f58 [0,4194304] 0 2024-04-22T23:46:03.353 INFO:tasks.workunit.client.1.smithi044.stdout:3/235: mknod d9/d14/c4e 0 2024-04-22T23:46:03.357 INFO:tasks.workunit.client.1.smithi044.stdout:5/237: rename d15/d16/d1d/d22/d2a/d3a/f41 to d15/d16/d1d/d22/d2a/d3a/f4b 0 2024-04-22T23:46:03.357 INFO:tasks.workunit.client.1.smithi044.stdout:5/238: chown d15/d16/l35 51338 1 2024-04-22T23:46:03.361 INFO:tasks.workunit.client.1.smithi044.stdout:1/250: creat d1/d8/d1c/d23/f5f x:0 0 0 2024-04-22T23:46:03.362 INFO:tasks.workunit.client.1.smithi044.stdout:1/251: chown d1/d8/f18 63007 1 2024-04-22T23:46:03.363 INFO:tasks.workunit.client.1.smithi044.stdout:5/239: dread d15/d16/f39 [0,4194304] 0 2024-04-22T23:46:03.363 INFO:tasks.workunit.client.1.smithi044.stdout:3/236: chown d9/l35 1 1 2024-04-22T23:46:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.372+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7e41067f0 con 0x7fc7cc0398c0 2024-04-22T23:46:03.364 INFO:tasks.workunit.client.1.smithi044.stdout:5/240: chown d15/c3e 306 1 2024-04-22T23:46:03.365 INFO:tasks.workunit.client.1.smithi044.stdout:3/237: chown d9/db/d15/d29 1246950556 1 2024-04-22T23:46:03.367 INFO:tasks.workunit.client.1.smithi044.stdout:8/287: symlink db/d14/l6d 0 2024-04-22T23:46:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.384+0000 7fc7e17fa700 1 -- 172.21.15.31:0/3624347210 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc7e41067f0 con 0x7fc7cc0398c0 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 16s ago 8m 25.6M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (9m) 16s ago 9m 7688k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (3m) 57s ago 8m 7520k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (9m) 16s ago 9m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (8m) 57s ago 8m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 16s ago 8m 87.4M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (3m) 16s ago 3m 13.3M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (3m) 16s ago 3m 188M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (3m) 57s ago 3m 108M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (3m) 57s ago 3m 16.9M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:46:03.377 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (35s) 16s ago 10m 65.5M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (94s) 57s ago 7m 491M - 19.0.0-3252-g43be0201 a0434699c1d7 05ec5942bb78 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (10m) 16s ago 10m 53.5M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (7m) 57s ago 7m 40.1M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 16s ago 9m 16.6M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 57s ago 7m 16.5M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 16s ago 7m 86.5M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 16s ago 6m 94.3M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 16s ago 6m 78.7M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (5m) 57s ago 5m 71.1M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (5m) 57s ago 5m 66.8M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (4m) 57s ago 4m 65.7M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:46:03.378 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (42s) 16s ago 8m 40.3M - 2.43.0 a07b618ecd1d 4f6e096a9f9a 2024-04-22T23:46:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7cc0398c0 msgr2=0x7fc7cc03bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7cc0398c0 0x7fc7cc03bd80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009d20 tx=0x7fc7d8009260 comp rx=0 tx=0).stop 2024-04-22T23:46:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4100ca0 msgr2=0x7fc7e41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41964d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4000c00 tx=0x7fc7d400f690 comp rx=0 tx=0).stop 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 shutdown_connections 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e4100ca0 0x7fc7e41964d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7cc0398c0 0x7fc7cc03bd80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 --2- 172.21.15.31:0/3624347210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7e4101ea0 0x7fc7e4196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.387+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 >> 172.21.15.31:0/3624347210 conn(0x7fc7e40fc260 msgr2=0x7fc7e41050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.388+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 shutdown_connections 2024-04-22T23:46:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.388+0000 7fc7ea9d2700 1 -- 172.21.15.31:0/3624347210 wait complete. 2024-04-22T23:46:03.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/288: mknod db/d6c/c6e 0 2024-04-22T23:46:03.381 INFO:tasks.workunit.client.1.smithi044.stdout:3/238: link d9/db/cd d9/db/d15/d25/c4f 0 2024-04-22T23:46:03.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/289: dread - db/d14/f41 zero size 2024-04-22T23:46:03.382 INFO:tasks.workunit.client.1.smithi044.stdout:8/290: write db/f21 [34039,67904] 0 2024-04-22T23:46:03.383 INFO:tasks.workunit.client.1.smithi044.stdout:5/241: rmdir d15/d16/d1d/d22/d2a/d3a/d43/d47 0 2024-04-22T23:46:03.384 INFO:tasks.workunit.client.1.smithi044.stdout:8/291: write db/f44 [443382,25810] 0 2024-04-22T23:46:03.385 INFO:tasks.workunit.client.1.smithi044.stdout:3/239: rmdir d9/d12 39 2024-04-22T23:46:03.386 INFO:tasks.workunit.client.1.smithi044.stdout:5/242: dread - d15/f30 zero size 2024-04-22T23:46:03.388 INFO:tasks.workunit.client.1.smithi044.stdout:3/240: write d9/db/d15/d25/f3b [771347,65678] 0 2024-04-22T23:46:03.390 INFO:tasks.workunit.client.1.smithi044.stdout:5/243: creat d15/d16/d1d/d22/d2a/d3a/d43/f4c x:0 0 0 2024-04-22T23:46:03.393 INFO:tasks.workunit.client.1.smithi044.stdout:5/244: symlink d15/d16/d1d/l4d 0 2024-04-22T23:46:03.393 INFO:tasks.workunit.client.1.smithi044.stdout:8/292: link db/d14/l6d db/d14/l6f 0 2024-04-22T23:46:03.394 INFO:tasks.workunit.client.1.smithi044.stdout:5/245: chown d15/d16/d1d/d22/f2d 5881903 1 2024-04-22T23:46:03.394 INFO:tasks.workunit.client.1.smithi044.stdout:5/246: fdatasync f14 0 2024-04-22T23:46:03.395 INFO:tasks.workunit.client.1.smithi044.stdout:3/241: symlink d9/db/d15/d2e/d48/l50 0 2024-04-22T23:46:03.395 INFO:tasks.workunit.client.1.smithi044.stdout:5/247: dread - d15/d16/d2c/f3b zero size 2024-04-22T23:46:03.397 INFO:tasks.workunit.client.1.smithi044.stdout:8/293: unlink db/d14/f3e 0 2024-04-22T23:46:03.397 INFO:tasks.workunit.client.1.smithi044.stdout:8/294: chown db/f11 490 1 2024-04-22T23:46:03.402 INFO:tasks.workunit.client.1.smithi044.stdout:3/242: symlink d9/l51 0 2024-04-22T23:46:03.404 INFO:tasks.workunit.client.1.smithi044.stdout:5/248: dwrite d15/d16/d1d/d22/d2a/d3a/f4b [0,4194304] 0 2024-04-22T23:46:03.407 INFO:tasks.workunit.client.1.smithi044.stdout:3/243: dread d9/f34 [0,4194304] 0 2024-04-22T23:46:03.409 INFO:tasks.workunit.client.1.smithi044.stdout:5/249: unlink d15/c24 0 2024-04-22T23:46:03.411 INFO:tasks.workunit.client.1.smithi044.stdout:5/250: truncate d15/f18 1121711 0 2024-04-22T23:46:03.412 INFO:tasks.workunit.client.1.smithi044.stdout:5/251: write f12 [1782903,39046] 0 2024-04-22T23:46:03.412 INFO:tasks.workunit.client.1.smithi044.stdout:3/244: mkdir d9/d12/d3a/d52 0 2024-04-22T23:46:03.413 INFO:tasks.workunit.client.1.smithi044.stdout:3/245: dread - d9/db/d15/d2e/f3f zero size 2024-04-22T23:46:03.414 INFO:tasks.workunit.client.1.smithi044.stdout:3/246: write f5 [4740605,41648] 0 2024-04-22T23:46:03.414 INFO:tasks.workunit.client.1.smithi044.stdout:5/252: symlink d15/d3c/l4e 0 2024-04-22T23:46:03.416 INFO:tasks.workunit.client.1.smithi044.stdout:5/253: unlink d15/l45 0 2024-04-22T23:46:03.418 INFO:tasks.workunit.client.1.smithi044.stdout:3/247: rename d9/db/d15/d25/l26 to d9/db/d33/l53 0 2024-04-22T23:46:03.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/254: mkdir d15/d16/d1d/d22/d4f 0 2024-04-22T23:46:03.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/255: truncate d15/d16/f1a 1560230 0 2024-04-22T23:46:03.421 INFO:tasks.workunit.client.1.smithi044.stdout:3/248: rename d9/db/d15/d29 to d9/db/d15/d54 0 2024-04-22T23:46:03.422 INFO:tasks.workunit.client.1.smithi044.stdout:3/249: rename d9 to d9/db/d15/d25/d55 22 2024-04-22T23:46:03.422 INFO:tasks.workunit.client.1.smithi044.stdout:5/256: creat d15/d16/d2c/d38/f50 x:0 0 0 2024-04-22T23:46:03.423 INFO:tasks.workunit.client.1.smithi044.stdout:5/257: write d15/f1e [3393837,46208] 0 2024-04-22T23:46:03.424 INFO:tasks.workunit.client.1.smithi044.stdout:5/258: chown d15/d16/d1d/d22/l2b 155971 1 2024-04-22T23:46:03.428 INFO:tasks.workunit.client.1.smithi044.stdout:3/250: read d9/f23 [51006,105028] 0 2024-04-22T23:46:03.430 INFO:tasks.workunit.client.1.smithi044.stdout:3/251: dread - d9/db/d22/f24 zero size 2024-04-22T23:46:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:5/259: dread fe [0,4194304] 0 2024-04-22T23:46:03.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/252: dread - d9/db/d22/f24 zero size 2024-04-22T23:46:03.439 INFO:tasks.workunit.client.1.smithi044.stdout:2/209: dwrite d8/fe [0,4194304] 0 2024-04-22T23:46:03.439 INFO:tasks.workunit.client.1.smithi044.stdout:3/253: rename d9/f19 to d9/db/d22/f56 0 2024-04-22T23:46:03.443 INFO:tasks.workunit.client.1.smithi044.stdout:2/210: symlink d8/l42 0 2024-04-22T23:46:03.445 INFO:tasks.workunit.client.1.smithi044.stdout:2/211: write d8/d19/f37 [993375,38664] 0 2024-04-22T23:46:03.446 INFO:tasks.workunit.client.1.smithi044.stdout:1/252: dread d1/d8/d20/d45/f2a [0,4194304] 0 2024-04-22T23:46:03.447 INFO:tasks.workunit.client.1.smithi044.stdout:1/253: chown d1/d8/d1c 6 1 2024-04-22T23:46:03.449 INFO:tasks.workunit.client.1.smithi044.stdout:2/212: creat d8/f43 x:0 0 0 2024-04-22T23:46:03.451 INFO:tasks.workunit.client.1.smithi044.stdout:1/254: mkdir d1/d8/d20/d47/d4a/d60 0 2024-04-22T23:46:03.452 INFO:tasks.workunit.client.1.smithi044.stdout:2/213: mkdir d8/d19/d36/d44 0 2024-04-22T23:46:03.455 INFO:tasks.workunit.client.1.smithi044.stdout:1/255: creat d1/d8/d20/d45/d5d/f61 x:0 0 0 2024-04-22T23:46:03.456 INFO:tasks.workunit.client.1.smithi044.stdout:1/256: fdatasync d1/d14/f36 0 2024-04-22T23:46:03.457 INFO:tasks.workunit.client.1.smithi044.stdout:2/214: dread d8/f11 [0,4194304] 0 2024-04-22T23:46:03.458 INFO:tasks.workunit.client.1.smithi044.stdout:1/257: write d1/d8/f1e [176059,7900] 0 2024-04-22T23:46:03.458 INFO:tasks.workunit.client.1.smithi044.stdout:2/215: chown d8/d19/d1c/l21 3542814 1 2024-04-22T23:46:03.460 INFO:tasks.workunit.client.1.smithi044.stdout:2/216: write d8/d19/d1c/f24 [3990061,130899] 0 2024-04-22T23:46:03.460 INFO:tasks.workunit.client.1.smithi044.stdout:1/258: write d1/d8/d20/d47/d4a/f54 [564279,103393] 0 2024-04-22T23:46:03.467 INFO:tasks.workunit.client.1.smithi044.stdout:2/217: symlink d8/l45 0 2024-04-22T23:46:03.471 INFO:tasks.workunit.client.1.smithi044.stdout:2/218: unlink d8/l42 0 2024-04-22T23:46:03.471 INFO:tasks.workunit.client.1.smithi044.stdout:8/295: write db/d14/d24/f4e [430491,70975] 0 2024-04-22T23:46:03.476 INFO:tasks.workunit.client.1.smithi044.stdout:2/219: mkdir d8/d19/d36/d46 0 2024-04-22T23:46:03.477 INFO:tasks.workunit.client.1.smithi044.stdout:4/297: sync 2024-04-22T23:46:03.477 INFO:tasks.workunit.client.1.smithi044.stdout:6/193: sync 2024-04-22T23:46:03.480 INFO:tasks.workunit.client.1.smithi044.stdout:8/296: symlink db/d52/l70 0 2024-04-22T23:46:03.483 INFO:tasks.workunit.client.1.smithi044.stdout:4/298: dwrite d5/d31/d44/f4a [0,4194304] 0 2024-04-22T23:46:03.484 INFO:tasks.workunit.client.1.smithi044.stdout:2/220: truncate d8/ff 4748233 0 2024-04-22T23:46:03.484 INFO:tasks.workunit.client.1.smithi044.stdout:4/299: fdatasync d5/d31/f11 0 2024-04-22T23:46:03.487 INFO:tasks.workunit.client.1.smithi044.stdout:6/194: chown d5/dd/d1d/d23/f3b 496529039 1 2024-04-22T23:46:03.495 INFO:tasks.workunit.client.1.smithi044.stdout:5/260: rmdir d15/d16 39 2024-04-22T23:46:03.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3980142444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 msgr2=0x7f7a0c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3980142444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c102300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009910 tx=0x7f7a00009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:03.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3980142444 shutdown_connections 2024-04-22T23:46:03.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3980142444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c102300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3980142444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a0c100ca0 0x7f7a0c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3980142444 >> 172.21.15.31:0/3980142444 conn(0x7f7a0c0fc260 msgr2=0x7f7a0c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:03.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3980142444 shutdown_connections 2024-04-22T23:46:03.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.506+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3980142444 wait complete. 2024-04-22T23:46:03.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.507+0000 7f7a11dc8700 1 Processor -- start 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.507+0000 7f7a11dc8700 1 -- start start 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.507+0000 7f7a11dc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a0c100ca0 0x7f7a0c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.507+0000 7f7a11dc8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.507+0000 7f7a11dc8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0736c0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.507+0000 7f7a11dc8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c073800 con 0x7f7a0c100ca0 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37412/0 (socket says 172.21.15.31:37412) 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 -- 172.21.15.31:0/3580776039 learned_addr learned my addr 172.21.15.31:0/3580776039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0b7fe700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a0c100ca0 0x7f7a0c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 -- 172.21.15.31:0/3580776039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a0c100ca0 msgr2=0x7f7a0c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a0c100ca0 0x7f7a0c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 -- 172.21.15.31:0/3580776039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000095f0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a0affd700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c073180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000c00 tx=0x7f7a0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:03.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.508+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a0001d070 con 0x7f7a0c101ea0 2024-04-22T23:46:03.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.509+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7a00005dc0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.509+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c073a80 con 0x7f7a0c101ea0 2024-04-22T23:46:03.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.509+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a000226a0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.509+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c073f20 con 0x7f7a0c101ea0 2024-04-22T23:46:03.503 INFO:tasks.workunit.client.1.smithi044.stdout:8/297: dwrite db/d14/d24/f5c [0,4194304] 0 2024-04-22T23:46:03.503 INFO:tasks.workunit.client.1.smithi044.stdout:2/221: unlink d8/d19/d1c/l34 0 2024-04-22T23:46:03.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.510+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f8005320 con 0x7f7a0c101ea0 2024-04-22T23:46:03.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.510+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 28) v1 ==== 46697+0+0 (secure 0 0 0) 0x7f7a00022800 con 0x7f7a0c101ea0 2024-04-22T23:46:03.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.510+0000 7f7a08ff9700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f79f40398c0 0x7f79f403bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:03.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.510+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7a0004f0e0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.510+0000 7f7a0b7fe700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f79f40398c0 0x7f79f403bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:03.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.511+0000 7f7a0b7fe700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f79f40398c0 0x7f79f403bd80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c101d00 tx=0x7f79fc009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:03.504 INFO:tasks.workunit.client.1.smithi044.stdout:4/300: creat d5/d31/d44/d20/d22/f62 x:0 0 0 2024-04-22T23:46:03.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.517+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f7a00020de0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/254: truncate d9/f34 2268677 0 2024-04-22T23:46:03.509 INFO:tasks.workunit.client.1.smithi044.stdout:5/261: write d15/d16/d1d/f42 [732303,46141] 0 2024-04-22T23:46:03.514 INFO:tasks.workunit.client.1.smithi044.stdout:6/195: dread d5/d30/f31 [0,4194304] 0 2024-04-22T23:46:03.514 INFO:tasks.workunit.client.1.smithi044.stdout:8/298: creat db/d52/f71 x:0 0 0 2024-04-22T23:46:03.518 INFO:tasks.workunit.client.1.smithi044.stdout:4/301: creat d5/d31/d24/d38/f63 x:0 0 0 2024-04-22T23:46:03.519 INFO:tasks.workunit.client.1.smithi044.stdout:5/262: rmdir d15/d16/d1d 39 2024-04-22T23:46:03.520 INFO:tasks.workunit.client.1.smithi044.stdout:4/302: truncate d5/d31/d44/d20/f5f 504707 0 2024-04-22T23:46:03.521 INFO:tasks.workunit.client.1.smithi044.stdout:4/303: chown d5/db/d3c 16559 1 2024-04-22T23:46:03.521 INFO:tasks.workunit.client.1.smithi044.stdout:4/304: chown d5/db/d16/d4d 310520 1 2024-04-22T23:46:03.522 INFO:tasks.workunit.client.1.smithi044.stdout:4/305: write d5/d31/d44/f4a [4327209,92067] 0 2024-04-22T23:46:03.523 INFO:tasks.workunit.client.1.smithi044.stdout:3/255: symlink d9/db/d15/l57 0 2024-04-22T23:46:03.534 INFO:tasks.workunit.client.1.smithi044.stdout:5/263: unlink f8 0 2024-04-22T23:46:03.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/259: write d1/d8/d1c/f5b [3760119,53864] 0 2024-04-22T23:46:03.539 INFO:tasks.workunit.client.1.smithi044.stdout:4/306: mkdir d5/db/d34/d64 0 2024-04-22T23:46:03.540 INFO:tasks.workunit.client.1.smithi044.stdout:4/307: chown d5/l3f 8 1 2024-04-22T23:46:03.543 INFO:tasks.workunit.client.1.smithi044.stdout:4/308: write d5/db/d34/f59 [508656,115933] 0 2024-04-22T23:46:03.544 INFO:tasks.workunit.client.1.smithi044.stdout:3/256: dwrite f8 [0,4194304] 0 2024-04-22T23:46:03.545 INFO:tasks.workunit.client.1.smithi044.stdout:4/309: write d5/d31/f10 [6522678,35130] 0 2024-04-22T23:46:03.547 INFO:tasks.workunit.client.1.smithi044.stdout:8/299: mkdir db/d14/d72 0 2024-04-22T23:46:03.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/257: write d9/db/f1a [2461042,57175] 0 2024-04-22T23:46:03.549 INFO:tasks.workunit.client.1.smithi044.stdout:3/258: write d9/d12/f1f [1863539,99393] 0 2024-04-22T23:46:03.553 INFO:tasks.workunit.client.1.smithi044.stdout:7/277: sync 2024-04-22T23:46:03.559 INFO:tasks.workunit.client.1.smithi044.stdout:5/264: write d15/d16/d1d/f3f [3490840,16559] 0 2024-04-22T23:46:03.568 INFO:tasks.workunit.client.1.smithi044.stdout:6/196: rename d5/dd/d1d/f25 to d5/dd/f3e 0 2024-04-22T23:46:03.578 INFO:tasks.workunit.client.1.smithi044.stdout:2/222: getdents d8/d19 0 2024-04-22T23:46:03.579 INFO:tasks.workunit.client.1.smithi044.stdout:2/223: write d8/d19/d36/f3e [47386,106273] 0 2024-04-22T23:46:03.586 INFO:tasks.workunit.client.1.smithi044.stdout:7/278: creat dd/d1f/d24/f63 x:0 0 0 2024-04-22T23:46:03.589 INFO:tasks.workunit.client.1.smithi044.stdout:1/260: mknod d1/c62 0 2024-04-22T23:46:03.591 INFO:tasks.workunit.client.1.smithi044.stdout:5/265: creat d15/d16/d1d/f51 x:0 0 0 2024-04-22T23:46:03.594 INFO:tasks.workunit.client.1.smithi044.stdout:6/197: write d5/d30/f31 [1502397,62949] 0 2024-04-22T23:46:03.596 INFO:tasks.workunit.client.1.smithi044.stdout:6/198: truncate d5/dd/d18/f24 1304274 0 2024-04-22T23:46:03.597 INFO:tasks.workunit.client.1.smithi044.stdout:8/300: mknod db/c73 0 2024-04-22T23:46:03.605 INFO:tasks.workunit.client.1.smithi044.stdout:2/224: creat d8/d19/f47 x:0 0 0 2024-04-22T23:46:03.606 INFO:tasks.workunit.client.1.smithi044.stdout:2/225: read - d8/d19/f39 zero size 2024-04-22T23:46:03.612 INFO:tasks.workunit.client.1.smithi044.stdout:5/266: fdatasync d15/d16/d1d/d22/d2a/d3a/f4b 0 2024-04-22T23:46:03.616 INFO:tasks.workunit.client.1.smithi044.stdout:4/310: link d5/d42/f48 d5/d31/d44/d20/d22/f65 0 2024-04-22T23:46:03.619 INFO:tasks.workunit.client.1.smithi044.stdout:2/226: mknod d8/d19/d36/c48 0 2024-04-22T23:46:03.624 INFO:tasks.workunit.client.1.smithi044.stdout:7/279: symlink dd/d55/d5e/d60/l64 0 2024-04-22T23:46:03.626 INFO:tasks.workunit.client.1.smithi044.stdout:0/260: sync 2024-04-22T23:46:03.626 INFO:tasks.workunit.client.1.smithi044.stdout:9/167: sync 2024-04-22T23:46:03.626 INFO:tasks.workunit.client.1.smithi044.stdout:1/261: unlink d1/d8/d1c/l33 0 2024-04-22T23:46:03.627 INFO:tasks.workunit.client.1.smithi044.stdout:0/261: fsync d0/d2e/d2f/f51 0 2024-04-22T23:46:03.637 INFO:tasks.workunit.client.1.smithi044.stdout:2/227: creat d8/d19/d1c/f49 x:0 0 0 2024-04-22T23:46:03.638 INFO:tasks.workunit.client.1.smithi044.stdout:8/301: rmdir db 39 2024-04-22T23:46:03.639 INFO:tasks.workunit.client.1.smithi044.stdout:3/259: write d9/db/f13 [1564069,105060] 0 2024-04-22T23:46:03.642 INFO:tasks.workunit.client.1.smithi044.stdout:9/168: rmdir d2 39 2024-04-22T23:46:03.645 INFO:tasks.workunit.client.1.smithi044.stdout:4/311: dwrite d5/d31/d44/f30 [0,4194304] 0 2024-04-22T23:46:03.647 INFO:tasks.workunit.client.1.smithi044.stdout:3/260: dread d9/f17 [0,4194304] 0 2024-04-22T23:46:03.649 INFO:tasks.workunit.client.1.smithi044.stdout:1/262: rmdir d1/d8/d24/d4b 39 2024-04-22T23:46:03.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/312: dread d5/d31/d44/f4a [0,4194304] 0 2024-04-22T23:46:03.653 INFO:tasks.workunit.client.1.smithi044.stdout:5/267: creat d15/d16/d1d/d22/d4f/f52 x:0 0 0 2024-04-22T23:46:03.662 INFO:tasks.workunit.client.1.smithi044.stdout:3/261: creat d9/db/d15/d54/f58 x:0 0 0 2024-04-22T23:46:03.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/262: chown d9/db/d15/f18 674319 1 2024-04-22T23:46:03.663 INFO:tasks.workunit.client.1.smithi044.stdout:0/262: creat d0/d4c/d59/d5a/f63 x:0 0 0 2024-04-22T23:46:03.664 INFO:tasks.workunit.client.1.smithi044.stdout:3/263: write d9/db/d22/f24 [304948,68037] 0 2024-04-22T23:46:03.665 INFO:tasks.workunit.client.1.smithi044.stdout:0/263: truncate d0/f22 150711 0 2024-04-22T23:46:03.665 INFO:tasks.workunit.client.1.smithi044.stdout:1/263: rmdir d1/d8/d24/d46 39 2024-04-22T23:46:03.667 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:03 smithi044 ceph-mon[34463]: pgmap v5: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail 2024-04-22T23:46:03.668 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:03 smithi044 ceph-mon[34463]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:03.668 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:03 smithi044 ceph-mon[34463]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:03.668 INFO:tasks.workunit.client.1.smithi044.stdout:3/264: dread d9/f34 [0,4194304] 0 2024-04-22T23:46:03.669 INFO:tasks.workunit.client.1.smithi044.stdout:6/199: rename d5/d8 to d5/d3f 0 2024-04-22T23:46:03.672 INFO:tasks.workunit.client.1.smithi044.stdout:4/313: mknod d5/d31/d44/d20/d22/c66 0 2024-04-22T23:46:03.673 INFO:tasks.workunit.client.1.smithi044.stdout:4/314: fsync d5/d31/d44/d20/d22/f35 0 2024-04-22T23:46:03.674 INFO:tasks.workunit.client.1.smithi044.stdout:5/268: creat d15/d3c/f53 x:0 0 0 2024-04-22T23:46:03.676 INFO:tasks.workunit.client.1.smithi044.stdout:8/302: mkdir db/d68/d74 0 2024-04-22T23:46:03.678 INFO:tasks.workunit.client.1.smithi044.stdout:8/303: truncate db/d14/d24/f4e 1516028 0 2024-04-22T23:46:03.678 INFO:tasks.workunit.client.1.smithi044.stdout:7/280: truncate f8 1636558 0 2024-04-22T23:46:03.683 INFO:tasks.workunit.client.1.smithi044.stdout:0/264: fdatasync d0/d2/f1e 0 2024-04-22T23:46:03.684 INFO:tasks.workunit.client.1.smithi044.stdout:1/264: mknod d1/d8/d1c/d23/d3e/c63 0 2024-04-22T23:46:03.688 INFO:tasks.workunit.client.1.smithi044.stdout:3/265: rename d9/d14/f43 to d9/d12/d3a/d52/f59 0 2024-04-22T23:46:03.690 INFO:tasks.workunit.client.1.smithi044.stdout:4/315: creat d5/d31/d44/d20/d22/f67 x:0 0 0 2024-04-22T23:46:03.691 INFO:tasks.workunit.client.1.smithi044.stdout:4/316: fsync d5/d31/d44/d20/d22/f35 0 2024-04-22T23:46:03.691 INFO:tasks.workunit.client.1.smithi044.stdout:2/228: truncate d8/f11 2607795 0 2024-04-22T23:46:03.692 INFO:tasks.workunit.client.1.smithi044.stdout:4/317: readlink d5/l3f 0 2024-04-22T23:46:03.693 INFO:tasks.workunit.client.1.smithi044.stdout:8/304: unlink db/l13 0 2024-04-22T23:46:03.694 INFO:tasks.workunit.client.1.smithi044.stdout:7/281: stat dd/d1f/f38 0 2024-04-22T23:46:03.695 INFO:tasks.workunit.client.1.smithi044.stdout:7/282: truncate dd/d27/f48 311719 0 2024-04-22T23:46:03.696 INFO:tasks.workunit.client.1.smithi044.stdout:8/305: dread db/fd [0,4194304] 0 2024-04-22T23:46:03.697 INFO:tasks.workunit.client.1.smithi044.stdout:8/306: chown db/d14/f46 488908129 1 2024-04-22T23:46:03.697 INFO:tasks.workunit.client.1.smithi044.stdout:8/307: fsync db/d14/d24/f49 0 2024-04-22T23:46:03.698 INFO:tasks.workunit.client.1.smithi044.stdout:9/169: link d2/dc/f27 d2/f3d 0 2024-04-22T23:46:03.700 INFO:tasks.workunit.client.1.smithi044.stdout:1/265: symlink d1/d8/d20/d45/l64 0 2024-04-22T23:46:03.702 INFO:tasks.workunit.client.1.smithi044.stdout:3/266: mknod d9/db/d15/d2e/c5a 0 2024-04-22T23:46:03.704 INFO:tasks.workunit.client.1.smithi044.stdout:2/229: chown d8/d19/d1c/l2e 503 1 2024-04-22T23:46:03.707 INFO:tasks.workunit.client.1.smithi044.stdout:8/308: creat db/d14/d24/f75 x:0 0 0 2024-04-22T23:46:03.707 INFO:tasks.workunit.client.1.smithi044.stdout:8/309: chown db/d14/l18 128435655 1 2024-04-22T23:46:03.710 INFO:tasks.workunit.client.1.smithi044.stdout:6/200: mkdir d5/dd/d1d/d40 0 2024-04-22T23:46:03.711 INFO:tasks.workunit.client.1.smithi044.stdout:1/266: creat d1/d8/d1c/d23/d3e/d4d/f65 x:0 0 0 2024-04-22T23:46:03.713 INFO:tasks.workunit.client.1.smithi044.stdout:5/269: rename f13 to d15/d16/d2c/f54 0 2024-04-22T23:46:03.715 INFO:tasks.workunit.client.1.smithi044.stdout:4/318: symlink d5/db/d16/l68 0 2024-04-22T23:46:03.717 INFO:tasks.workunit.client.1.smithi044.stdout:3/267: rmdir d9/db/d22/d38 39 2024-04-22T23:46:03.719 INFO:tasks.workunit.client.1.smithi044.stdout:2/230: symlink d8/d19/d36/l4a 0 2024-04-22T23:46:03.719 INFO:tasks.workunit.client.1.smithi044.stdout:5/270: dwrite d15/f30 [0,4194304] 0 2024-04-22T23:46:03.721 INFO:tasks.workunit.client.1.smithi044.stdout:0/265: creat d0/d2/f64 x:0 0 0 2024-04-22T23:46:03.729 INFO:tasks.workunit.client.1.smithi044.stdout:7/283: rename dd/d1f/d24/f3d to dd/d1a/f65 0 2024-04-22T23:46:03.731 INFO:tasks.workunit.client.1.smithi044.stdout:4/319: stat d5/d31/d44/d20/d22/f37 0 2024-04-22T23:46:03.733 INFO:tasks.workunit.client.1.smithi044.stdout:4/320: write d5/d31/d44/f30 [2132958,38612] 0 2024-04-22T23:46:03.735 INFO:tasks.workunit.client.1.smithi044.stdout:2/231: write d8/d19/f2b [563768,49523] 0 2024-04-22T23:46:03.737 INFO:tasks.workunit.client.1.smithi044.stdout:5/271: mkdir d15/d16/d1d/d22/d2a/d3a/d43/d55 0 2024-04-22T23:46:03.738 INFO:tasks.workunit.client.1.smithi044.stdout:7/284: dwrite dd/d1f/d24/f13 [0,4194304] 0 2024-04-22T23:46:03.739 INFO:tasks.workunit.client.1.smithi044.stdout:5/272: chown d15/d16/f28 0 1 2024-04-22T23:46:03.740 INFO:tasks.workunit.client.1.smithi044.stdout:0/266: unlink d0/c30 0 2024-04-22T23:46:03.743 INFO:tasks.workunit.client.1.smithi044.stdout:6/201: creat d5/dd/d1d/d40/f41 x:0 0 0 2024-04-22T23:46:03.748 INFO:tasks.workunit.client.1.smithi044.stdout:7/285: dwrite f5 [0,4194304] 0 2024-04-22T23:46:03.750 INFO:tasks.workunit.client.1.smithi044.stdout:5/273: dwrite d15/d16/d1d/f3f [0,4194304] 0 2024-04-22T23:46:03.755 INFO:tasks.workunit.client.1.smithi044.stdout:1/267: rename d1/d14/f3b to d1/d8/d1c/d23/d3e/d4d/f66 0 2024-04-22T23:46:03.765 INFO:tasks.workunit.client.1.smithi044.stdout:1/268: dwrite d1/d8/d1c/f5b [0,4194304] 0 2024-04-22T23:46:03.768 INFO:tasks.workunit.client.1.smithi044.stdout:0/267: creat d0/d2e/d2f/f65 x:0 0 0 2024-04-22T23:46:03.770 INFO:tasks.workunit.client.1.smithi044.stdout:8/310: dwrite db/d14/d24/f26 [4194304,4194304] 0 2024-04-22T23:46:03.770 INFO:tasks.workunit.client.1.smithi044.stdout:1/269: dread d1/d14/f52 [0,4194304] 0 2024-04-22T23:46:03.771 INFO:tasks.workunit.client.1.smithi044.stdout:9/170: symlink d2/dc/l3e 0 2024-04-22T23:46:03.774 INFO:tasks.workunit.client.1.smithi044.stdout:5/274: mknod d15/d16/d1d/d22/d2a/d3a/c56 0 2024-04-22T23:46:03.783 INFO:tasks.workunit.client.1.smithi044.stdout:3/268: rename d9/d12/d3a to d9/db/d15/d25/d5b 0 2024-04-22T23:46:03.784 INFO:tasks.workunit.client.1.smithi044.stdout:3/269: read d9/f23 [246901,58324] 0 2024-04-22T23:46:03.791 INFO:tasks.workunit.client.1.smithi044.stdout:3/270: dwrite d9/db/d15/f21 [0,4194304] 0 2024-04-22T23:46:03.799 INFO:tasks.workunit.client.1.smithi044.stdout:8/311: mknod db/d6c/c76 0 2024-04-22T23:46:03.800 INFO:tasks.workunit.client.1.smithi044.stdout:8/312: write db/f21 [120410,98246] 0 2024-04-22T23:46:03.802 INFO:tasks.workunit.client.1.smithi044.stdout:8/313: write db/f50 [769055,27610] 0 2024-04-22T23:46:03.804 INFO:tasks.workunit.client.1.smithi044.stdout:9/171: write d2/d5/f1e [5363867,75847] 0 2024-04-22T23:46:03.805 INFO:tasks.workunit.client.1.smithi044.stdout:0/268: creat d0/d2/dd/d3e/d40/d43/f66 x:0 0 0 2024-04-22T23:46:03.805 INFO:tasks.workunit.client.1.smithi044.stdout:9/172: write d2/dc/f32 [664481,128485] 0 2024-04-22T23:46:03.806 INFO:tasks.workunit.client.1.smithi044.stdout:0/269: write d0/f17 [142199,102498] 0 2024-04-22T23:46:03.807 INFO:tasks.workunit.client.1.smithi044.stdout:0/270: chown d0/d2/l39 0 1 2024-04-22T23:46:03.808 INFO:tasks.workunit.client.1.smithi044.stdout:5/275: symlink d15/d16/d1d/d22/d4f/l57 0 2024-04-22T23:46:03.809 INFO:tasks.workunit.client.1.smithi044.stdout:0/271: write d0/d2/f64 [505335,58650] 0 2024-04-22T23:46:03.814 INFO:tasks.workunit.client.1.smithi044.stdout:4/321: write d5/d31/d44/d20/d22/f65 [396259,119237] 0 2024-04-22T23:46:03.818 INFO:tasks.workunit.client.1.smithi044.stdout:0/272: dwrite d0/d2/dd/f1d [0,4194304] 0 2024-04-22T23:46:03.819 INFO:tasks.workunit.client.1.smithi044.stdout:0/273: write d0/d2/f1e [1103277,8856] 0 2024-04-22T23:46:03.821 INFO:tasks.workunit.client.1.smithi044.stdout:3/271: rmdir d9/db/d15 39 2024-04-22T23:46:03.823 INFO:tasks.workunit.client.1.smithi044.stdout:3/272: truncate d9/f17 1061984 0 2024-04-22T23:46:03.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/202: link d5/dd/c37 d5/dd/d1d/d23/c42 0 2024-04-22T23:46:03.830 INFO:tasks.workunit.client.1.smithi044.stdout:4/322: dwrite d5/d31/f10 [4194304,4194304] 0 2024-04-22T23:46:03.832 INFO:tasks.workunit.client.1.smithi044.stdout:0/274: dwrite d0/d2/dd/d3e/f4f [0,4194304] 0 2024-04-22T23:46:03.834 INFO:tasks.workunit.client.1.smithi044.stdout:6/203: dread - d5/dd/d1d/d40/f41 zero size 2024-04-22T23:46:03.834 INFO:tasks.workunit.client.1.smithi044.stdout:8/314: creat db/d68/f77 x:0 0 0 2024-04-22T23:46:03.840 INFO:tasks.workunit.client.1.smithi044.stdout:8/315: dread db/f22 [4194304,4194304] 0 2024-04-22T23:46:03.841 INFO:tasks.workunit.client.1.smithi044.stdout:7/286: creat dd/d1f/f66 x:0 0 0 2024-04-22T23:46:03.842 INFO:tasks.workunit.client.1.smithi044.stdout:8/316: write db/d68/f77 [974511,50720] 0 2024-04-22T23:46:03.843 INFO:tasks.workunit.client.1.smithi044.stdout:9/173: mkdir d2/dc/d1d/d3f 0 2024-04-22T23:46:03.849 INFO:tasks.workunit.client.1.smithi044.stdout:5/276: chown fe 948 1 2024-04-22T23:46:03.851 INFO:tasks.workunit.client.1.smithi044.stdout:2/232: write d8/d19/f20 [1890708,394] 0 2024-04-22T23:46:03.853 INFO:tasks.workunit.client.1.smithi044.stdout:3/273: chown d9/db/cd 2 1 2024-04-22T23:46:03.854 INFO:tasks.workunit.client.1.smithi044.stdout:1/270: creat d1/f67 x:0 0 0 2024-04-22T23:46:03.856 INFO:tasks.workunit.client.1.smithi044.stdout:4/323: rmdir d5/db/d3c 39 2024-04-22T23:46:03.859 INFO:tasks.workunit.client.1.smithi044.stdout:1/271: write d1/d8/d20/d45/f29 [4106246,115787] 0 2024-04-22T23:46:03.862 INFO:tasks.workunit.client.1.smithi044.stdout:6/204: mknod d5/dd/d18/c43 0 2024-04-22T23:46:03.864 INFO:tasks.workunit.client.1.smithi044.stdout:2/233: dwrite d8/f38 [0,4194304] 0 2024-04-22T23:46:03.867 INFO:tasks.workunit.client.1.smithi044.stdout:0/275: mkdir d0/d2/dd/d3e/d40/d67 0 2024-04-22T23:46:03.867 INFO:tasks.workunit.client.1.smithi044.stdout:7/287: symlink dd/d55/d5e/l67 0 2024-04-22T23:46:03.870 INFO:tasks.workunit.client.1.smithi044.stdout:8/317: unlink c7 0 2024-04-22T23:46:03.871 INFO:tasks.workunit.client.1.smithi044.stdout:1/272: dwrite d1/d8/d20/d45/f39 [0,4194304] 0 2024-04-22T23:46:03.873 INFO:tasks.workunit.client.1.smithi044.stdout:9/174: symlink d2/d5/l40 0 2024-04-22T23:46:03.876 INFO:tasks.workunit.client.1.smithi044.stdout:5/277: creat d15/d16/d1d/d22/d2a/d3a/d43/f58 x:0 0 0 2024-04-22T23:46:03.876 INFO:tasks.workunit.client.1.smithi044.stdout:0/276: truncate d0/d2e/d2f/f4d 261870 0 2024-04-22T23:46:03.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:03 smithi031 ceph-mon[26721]: pgmap v5: 65 pgs: 65 active+clean; 2.3 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail 2024-04-22T23:46:03.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:03 smithi031 ceph-mon[26721]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:03.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:03 smithi031 ceph-mon[26721]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:03.883 INFO:tasks.workunit.client.1.smithi044.stdout:2/234: creat d8/d27/f4b x:0 0 0 2024-04-22T23:46:03.885 INFO:tasks.workunit.client.1.smithi044.stdout:7/288: mkdir dd/d55/d5e/d60/d68 0 2024-04-22T23:46:03.890 INFO:tasks.workunit.client.1.smithi044.stdout:0/277: rename d0/d2/dd/d3e/d40/f41 to d0/d2/d4b/f68 0 2024-04-22T23:46:03.893 INFO:tasks.workunit.client.1.smithi044.stdout:6/205: creat d5/d30/f44 x:0 0 0 2024-04-22T23:46:03.896 INFO:tasks.workunit.client.1.smithi044.stdout:0/278: dread d0/d2e/d2f/f4d [0,4194304] 0 2024-04-22T23:46:03.896 INFO:tasks.workunit.client.1.smithi044.stdout:1/273: dwrite d1/d8/d20/d45/f2a [0,4194304] 0 2024-04-22T23:46:03.898 INFO:tasks.workunit.client.1.smithi044.stdout:7/289: dread f7 [0,4194304] 0 2024-04-22T23:46:03.899 INFO:tasks.workunit.client.1.smithi044.stdout:2/235: creat d8/d19/f4c x:0 0 0 2024-04-22T23:46:03.900 INFO:tasks.workunit.client.1.smithi044.stdout:7/290: chown dd/f20 969090626 1 2024-04-22T23:46:03.906 INFO:tasks.workunit.client.1.smithi044.stdout:8/318: dread db/ff [0,4194304] 0 2024-04-22T23:46:03.909 INFO:tasks.workunit.client.1.smithi044.stdout:3/274: dwrite d9/f34 [0,4194304] 0 2024-04-22T23:46:03.909 INFO:tasks.workunit.client.1.smithi044.stdout:8/319: dread - db/d14/d24/f30 zero size 2024-04-22T23:46:03.912 INFO:tasks.workunit.client.1.smithi044.stdout:6/206: readlink d5/d3f/lb 0 2024-04-22T23:46:03.912 INFO:tasks.workunit.client.1.smithi044.stdout:6/207: chown d5 3456 1 2024-04-22T23:46:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.923+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f79f8005cc0 con 0x7f7a0c101ea0 2024-04-22T23:46:03.915 INFO:tasks.workunit.client.1.smithi044.stdout:2/236: unlink d8/d19/d36/c48 0 2024-04-22T23:46:03.916 INFO:tasks.workunit.client.1.smithi044.stdout:6/208: write d5/dd/d1d/d23/f26 [2819221,125493] 0 2024-04-22T23:46:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.924+0000 7f7a08ff9700 1 -- 172.21.15.31:0/3580776039 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f7a00003e60 con 0x7f7a0c101ea0 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:03.917 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:46:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:46:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:46:03.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:46:03.918 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:46:03.918 INFO:tasks.workunit.client.1.smithi044.stdout:6/209: stat d5/dd/f3e 0 2024-04-22T23:46:03.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f79f40398c0 msgr2=0x7f79f403bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f79f40398c0 0x7f79f403bd80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c101d00 tx=0x7f79fc009260 comp rx=0 tx=0).stop 2024-04-22T23:46:03.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 msgr2=0x7f7a0c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c073180 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7a00000c00 tx=0x7f7a0000f690 comp rx=0 tx=0).stop 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 shutdown_connections 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a0c100ca0 0x7f7a0c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f79f40398c0 0x7f79f403bd80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 --2- 172.21.15.31:0/3580776039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a0c101ea0 0x7f7a0c073180 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 >> 172.21.15.31:0/3580776039 conn(0x7f7a0c0fc260 msgr2=0x7f7a0c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 shutdown_connections 2024-04-22T23:46:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:03.927+0000 7f7a11dc8700 1 -- 172.21.15.31:0/3580776039 wait complete. 2024-04-22T23:46:03.928 INFO:tasks.workunit.client.1.smithi044.stdout:3/275: dread d9/db/f13 [0,4194304] 0 2024-04-22T23:46:03.929 INFO:tasks.workunit.client.1.smithi044.stdout:7/291: dread dd/d1f/f34 [0,4194304] 0 2024-04-22T23:46:03.930 INFO:tasks.workunit.client.1.smithi044.stdout:0/279: dread d0/d2e/d2f/f50 [0,4194304] 0 2024-04-22T23:46:03.935 INFO:tasks.workunit.client.1.smithi044.stdout:5/278: getdents d15/d3c 0 2024-04-22T23:46:03.937 INFO:tasks.workunit.client.1.smithi044.stdout:5/279: write d15/d16/d1d/f51 [553934,61433] 0 2024-04-22T23:46:03.938 INFO:tasks.workunit.client.1.smithi044.stdout:5/280: chown d15/d16/d2c 104 1 2024-04-22T23:46:03.939 INFO:tasks.workunit.client.1.smithi044.stdout:2/237: rename d8/f15 to d8/d27/f4d 0 2024-04-22T23:46:03.956 INFO:tasks.workunit.client.1.smithi044.stdout:3/276: creat d9/db/d33/f5c x:0 0 0 2024-04-22T23:46:03.960 INFO:tasks.workunit.client.1.smithi044.stdout:0/280: mknod d0/d2/d3d/c69 0 2024-04-22T23:46:03.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/281: dread d0/d2/dd/f1d [0,4194304] 0 2024-04-22T23:46:03.969 INFO:tasks.workunit.client.1.smithi044.stdout:5/281: creat d15/d16/d1d/d22/d4f/f59 x:0 0 0 2024-04-22T23:46:03.970 INFO:tasks.workunit.client.1.smithi044.stdout:5/282: chown d15/d16/c2f 38 1 2024-04-22T23:46:03.972 INFO:tasks.workunit.client.1.smithi044.stdout:2/238: rename d8/d19/d1c/f49 to d8/d19/d36/d46/f4e 0 2024-04-22T23:46:03.974 INFO:tasks.workunit.client.1.smithi044.stdout:2/239: dread d8/d27/f2c [0,4194304] 0 2024-04-22T23:46:03.981 INFO:tasks.workunit.client.1.smithi044.stdout:5/283: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f4c [0,4194304] 0 2024-04-22T23:46:03.984 INFO:tasks.workunit.client.1.smithi044.stdout:7/292: fdatasync f8 0 2024-04-22T23:46:03.986 INFO:tasks.workunit.client.1.smithi044.stdout:3/277: write d9/db/d15/f18 [1965167,19156] 0 2024-04-22T23:46:03.986 INFO:tasks.workunit.client.1.smithi044.stdout:2/240: dwrite d8/d19/d36/f3d [0,4194304] 0 2024-04-22T23:46:03.986 INFO:tasks.workunit.client.1.smithi044.stdout:5/284: write d15/d16/d1d/f49 [203792,115465] 0 2024-04-22T23:46:03.991 INFO:tasks.workunit.client.1.smithi044.stdout:3/278: read d9/db/d22/f24 [303940,113413] 0 2024-04-22T23:46:03.994 INFO:tasks.workunit.client.1.smithi044.stdout:1/274: getdents d1/d8/d20/d45/d5d 0 2024-04-22T23:46:04.000 INFO:tasks.workunit.client.1.smithi044.stdout:5/285: dwrite d15/d3c/f53 [0,4194304] 0 2024-04-22T23:46:04.002 INFO:tasks.workunit.client.1.smithi044.stdout:5/286: write d15/d16/d1d/f51 [569096,60636] 0 2024-04-22T23:46:04.005 INFO:tasks.workunit.client.1.smithi044.stdout:6/210: link d5/d3f/c36 d5/d30/c45 0 2024-04-22T23:46:04.005 INFO:tasks.workunit.client.1.smithi044.stdout:6/211: read - d5/d3f/f35 zero size 2024-04-22T23:46:04.006 INFO:tasks.workunit.client.1.smithi044.stdout:8/320: getdents db/d68 0 2024-04-22T23:46:04.019 INFO:tasks.workunit.client.1.smithi044.stdout:2/241: creat d8/d27/f4f x:0 0 0 2024-04-22T23:46:04.021 INFO:tasks.workunit.client.1.smithi044.stdout:2/242: dread - d8/d19/d36/d46/f4e zero size 2024-04-22T23:46:04.021 INFO:tasks.workunit.client.1.smithi044.stdout:2/243: readlink d8/l12 0 2024-04-22T23:46:04.022 INFO:tasks.workunit.client.1.smithi044.stdout:3/279: dread d9/db/d22/f56 [0,4194304] 0 2024-04-22T23:46:04.024 INFO:tasks.workunit.client.1.smithi044.stdout:0/282: mkdir d0/d2e/d2f/d61/d6a 0 2024-04-22T23:46:04.030 INFO:tasks.workunit.client.1.smithi044.stdout:6/212: creat d5/dd/d1d/f46 x:0 0 0 2024-04-22T23:46:04.033 INFO:tasks.workunit.client.1.smithi044.stdout:2/244: unlink d8/f38 0 2024-04-22T23:46:04.034 INFO:tasks.workunit.client.1.smithi044.stdout:2/245: chown d8/f10 219197279 1 2024-04-22T23:46:04.035 INFO:tasks.workunit.client.1.smithi044.stdout:1/275: creat d1/d8/d24/d46/f68 x:0 0 0 2024-04-22T23:46:04.037 INFO:tasks.workunit.client.1.smithi044.stdout:0/283: mkdir d0/d2/d4b/d6b 0 2024-04-22T23:46:04.039 INFO:tasks.workunit.client.1.smithi044.stdout:0/284: dread - d0/d2/d3d/f5c zero size 2024-04-22T23:46:04.042 INFO:tasks.workunit.client.1.smithi044.stdout:2/246: mknod d8/d19/d36/d46/c50 0 2024-04-22T23:46:04.042 INFO:tasks.workunit.client.1.smithi044.stdout:1/276: dwrite d1/d8/f12 [0,4194304] 0 2024-04-22T23:46:04.047 INFO:tasks.workunit.client.1.smithi044.stdout:8/321: creat db/d14/f78 x:0 0 0 2024-04-22T23:46:04.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 -- 172.21.15.31:0/1694056370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0104130 msgr2=0x7f6fd0106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:04.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 --2- 172.21.15.31:0/1694056370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0104130 0x7f6fd0106520 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009910 tx=0x7f6fc0009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:04.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 -- 172.21.15.31:0/1694056370 shutdown_connections 2024-04-22T23:46:04.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 --2- 172.21.15.31:0/1694056370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0104130 0x7f6fd0106520 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 --2- 172.21.15.31:0/1694056370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fd0101800 0x7f6fd0103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 -- 172.21.15.31:0/1694056370 >> 172.21.15.31:0/1694056370 conn(0x7f6fd00fb190 msgr2=0x7f6fd00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 -- 172.21.15.31:0/1694056370 shutdown_connections 2024-04-22T23:46:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.057+0000 7f6fd7443700 1 -- 172.21.15.31:0/1694056370 wait complete. 2024-04-22T23:46:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd7443700 1 Processor -- start 2024-04-22T23:46:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd7443700 1 -- start start 2024-04-22T23:46:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd7443700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 0x7f6fd0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd7443700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fd0104130 0x7f6fd0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd7443700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01991d0 con 0x7f6fd0101800 2024-04-22T23:46:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd7443700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0199310 con 0x7f6fd0104130 2024-04-22T23:46:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd51df700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 0x7f6fd0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd51df700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 0x7f6fd0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37422/0 (socket says 172.21.15.31:37422) 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd51df700 1 -- 172.21.15.31:0/3777826994 learned_addr learned my addr 172.21.15.31:0/3777826994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.058+0000 7f6fd51df700 1 -- 172.21.15.31:0/3777826994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fd0104130 msgr2=0x7f6fd0198bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fd51df700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fd0104130 0x7f6fd0198bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fd51df700 1 -- 172.21.15.31:0/3777826994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00095f0 con 0x7f6fd0101800 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fd51df700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 0x7f6fd0198670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00d8d0 tx=0x7f6fcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fcc009940 con 0x7f6fd0101800 2024-04-22T23:46:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6fcc010460 con 0x7f6fd0101800 2024-04-22T23:46:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd019ddc0 con 0x7f6fd0101800 2024-04-22T23:46:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fcc00f5f0 con 0x7f6fd0101800 2024-04-22T23:46:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.059+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd019e290 con 0x7f6fd0101800 2024-04-22T23:46:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.060+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 28) v1 ==== 46697+0+0 (secure 0 0 0) 0x7f6fcc00f7e0 con 0x7f6fd0101800 2024-04-22T23:46:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.060+0000 7f6fc67fc700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6fbc0398c0 0x7f6fbc03bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.060+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6fcc04ece0 con 0x7f6fd0101800 2024-04-22T23:46:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.061+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd00ffe80 con 0x7f6fd0101800 2024-04-22T23:46:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.061+0000 7f6fd49de700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6fbc0398c0 0x7f6fbc03bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.061+0000 7f6fd49de700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6fbc0398c0 0x7f6fbc03bd80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0000c00 tx=0x7f6fc0005fb0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:04.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.066+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f6fcc020020 con 0x7f6fd0101800 2024-04-22T23:46:04.061 INFO:tasks.workunit.client.1.smithi044.stdout:7/293: truncate dd/d1f/f31 766926 0 2024-04-22T23:46:04.061 INFO:tasks.workunit.client.1.smithi044.stdout:2/247: creat d8/d19/f51 x:0 0 0 2024-04-22T23:46:04.061 INFO:tasks.workunit.client.1.smithi044.stdout:2/248: chown d8/d27/f4d 3 1 2024-04-22T23:46:04.061 INFO:tasks.workunit.client.1.smithi044.stdout:3/280: write d9/db/d22/d38/f4a [3562115,92311] 0 2024-04-22T23:46:04.062 INFO:tasks.workunit.client.1.smithi044.stdout:0/285: mknod d0/d2/c6c 0 2024-04-22T23:46:04.063 INFO:tasks.workunit.client.1.smithi044.stdout:7/294: dread - dd/d1f/f38 zero size 2024-04-22T23:46:04.064 INFO:tasks.workunit.client.1.smithi044.stdout:1/277: mknod d1/d8/d24/d4b/c69 0 2024-04-22T23:46:04.068 INFO:tasks.workunit.client.1.smithi044.stdout:6/213: getdents d5/dd/d1d 0 2024-04-22T23:46:04.070 INFO:tasks.workunit.client.1.smithi044.stdout:3/281: rename d9/db/d36 to d9/db/d33/d5d 0 2024-04-22T23:46:04.076 INFO:tasks.workunit.client.1.smithi044.stdout:7/295: dread - dd/d1a/f2f zero size 2024-04-22T23:46:04.104 INFO:tasks.workunit.client.1.smithi044.stdout:1/278: symlink d1/d8/d1c/d23/l6a 0 2024-04-22T23:46:04.104 INFO:tasks.workunit.client.1.smithi044.stdout:2/249: write d8/f10 [109478,118609] 0 2024-04-22T23:46:04.105 INFO:tasks.workunit.client.1.smithi044.stdout:6/214: creat d5/dd/d18/f47 x:0 0 0 2024-04-22T23:46:04.105 INFO:tasks.workunit.client.1.smithi044.stdout:3/282: unlink d9/db/d33/f5c 0 2024-04-22T23:46:04.108 INFO:tasks.workunit.client.1.smithi044.stdout:1/279: unlink d1/le 0 2024-04-22T23:46:04.109 INFO:tasks.workunit.client.1.smithi044.stdout:7/296: write dd/d1f/d24/f1c [958998,39685] 0 2024-04-22T23:46:04.110 INFO:tasks.workunit.client.1.smithi044.stdout:3/283: truncate d9/db/d15/d25/d5b/f40 874879 0 2024-04-22T23:46:04.112 INFO:tasks.workunit.client.1.smithi044.stdout:0/286: dwrite d0/d2/dd/f12 [0,4194304] 0 2024-04-22T23:46:04.113 INFO:tasks.workunit.client.1.smithi044.stdout:0/287: write d0/d2/d3d/f5c [511587,69814] 0 2024-04-22T23:46:04.147 INFO:tasks.workunit.client.1.smithi044.stdout:0/288: write d0/d2/dd/d3e/d40/d43/f4a [39976,42282] 0 2024-04-22T23:46:04.151 INFO:tasks.workunit.client.1.smithi044.stdout:1/280: creat d1/d8/d24/d4b/f6b x:0 0 0 2024-04-22T23:46:04.151 INFO:tasks.workunit.client.1.smithi044.stdout:7/297: rmdir dd/d1a 39 2024-04-22T23:46:04.151 INFO:tasks.workunit.client.1.smithi044.stdout:1/281: fdatasync d1/d8/f1e 0 2024-04-22T23:46:04.182 INFO:tasks.workunit.client.1.smithi044.stdout:6/215: dread d5/dd/d1d/d23/f29 [0,4194304] 0 2024-04-22T23:46:04.183 INFO:tasks.workunit.client.1.smithi044.stdout:3/284: symlink d9/db/d33/l5e 0 2024-04-22T23:46:04.185 INFO:tasks.workunit.client.1.smithi044.stdout:1/282: dwrite d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:04.186 INFO:tasks.workunit.client.1.smithi044.stdout:2/250: creat d8/d19/d36/d44/f52 x:0 0 0 2024-04-22T23:46:04.188 INFO:tasks.workunit.client.1.smithi044.stdout:1/283: write d1/d8/d20/d45/f4f [4517432,71862] 0 2024-04-22T23:46:04.194 INFO:tasks.workunit.client.1.smithi044.stdout:3/285: symlink d9/db/d22/d38/l5f 0 2024-04-22T23:46:04.197 INFO:tasks.workunit.client.1.smithi044.stdout:0/289: mknod d0/d2/d4b/d6b/c6d 0 2024-04-22T23:46:04.200 INFO:tasks.workunit.client.1.smithi044.stdout:1/284: dwrite d1/d8/d24/d4b/f6b [0,4194304] 0 2024-04-22T23:46:04.204 INFO:tasks.workunit.client.1.smithi044.stdout:2/251: rmdir d8/d19/d36 39 2024-04-22T23:46:04.205 INFO:tasks.workunit.client.1.smithi044.stdout:1/285: chown d1/d8/f19 170 1 2024-04-22T23:46:04.205 INFO:tasks.workunit.client.1.smithi044.stdout:6/216: dwrite d5/f12 [0,4194304] 0 2024-04-22T23:46:04.208 INFO:tasks.workunit.client.1.smithi044.stdout:7/298: rename dd/d55/d5e/d60/l64 to dd/d1a/l69 0 2024-04-22T23:46:04.210 INFO:tasks.workunit.client.1.smithi044.stdout:7/299: fdatasync dd/d1f/d24/f3f 0 2024-04-22T23:46:04.213 INFO:tasks.workunit.client.1.smithi044.stdout:7/300: dread - dd/d1f/f66 zero size 2024-04-22T23:46:04.213 INFO:tasks.workunit.client.1.smithi044.stdout:3/286: dwrite d9/db/d15/d25/d4c/f4d [0,4194304] 0 2024-04-22T23:46:04.216 INFO:tasks.workunit.client.1.smithi044.stdout:6/217: mknod d5/d3f/c48 0 2024-04-22T23:46:04.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/286: dread d1/d8/f26 [0,4194304] 0 2024-04-22T23:46:04.227 INFO:tasks.workunit.client.1.smithi044.stdout:2/252: mkdir d8/d19/d53 0 2024-04-22T23:46:04.229 INFO:tasks.workunit.client.1.smithi044.stdout:6/218: fsync d5/dd/f27 0 2024-04-22T23:46:04.231 INFO:tasks.workunit.client.1.smithi044.stdout:6/219: readlink d5/dd/d1d/l2d 0 2024-04-22T23:46:04.232 INFO:tasks.workunit.client.1.smithi044.stdout:3/287: dwrite d9/f34 [0,4194304] 0 2024-04-22T23:46:04.234 INFO:tasks.workunit.client.1.smithi044.stdout:6/220: chown d5/dd/d1d/d23/f3b 9 1 2024-04-22T23:46:04.235 INFO:tasks.workunit.client.1.smithi044.stdout:0/290: link d0/f17 d0/d2/dd/d3e/f6e 0 2024-04-22T23:46:04.237 INFO:tasks.workunit.client.1.smithi044.stdout:0/291: readlink d0/d2/dd/d3e/d40/l57 0 2024-04-22T23:46:04.242 INFO:tasks.workunit.client.1.smithi044.stdout:3/288: creat d9/d12/f60 x:0 0 0 2024-04-22T23:46:04.244 INFO:tasks.workunit.client.1.smithi044.stdout:6/221: symlink d5/d3f/l49 0 2024-04-22T23:46:04.245 INFO:tasks.workunit.client.1.smithi044.stdout:7/301: creat dd/d1f/f6a x:0 0 0 2024-04-22T23:46:04.246 INFO:tasks.workunit.client.1.smithi044.stdout:7/302: write dd/f20 [1927783,94529] 0 2024-04-22T23:46:04.250 INFO:tasks.workunit.client.1.smithi044.stdout:0/292: rmdir d0/d2/d3d 39 2024-04-22T23:46:04.251 INFO:tasks.workunit.client.1.smithi044.stdout:2/253: mkdir d8/d19/d36/d46/d54 0 2024-04-22T23:46:04.252 INFO:tasks.workunit.client.1.smithi044.stdout:2/254: fsync d8/f43 0 2024-04-22T23:46:04.252 INFO:tasks.workunit.client.1.smithi044.stdout:2/255: chown d8/d19 577 1 2024-04-22T23:46:04.260 INFO:tasks.workunit.client.1.smithi044.stdout:1/287: link d1/d14/l30 d1/d8/d20/d45/l6c 0 2024-04-22T23:46:04.264 INFO:tasks.workunit.client.1.smithi044.stdout:7/303: dread dd/d1f/f31 [0,4194304] 0 2024-04-22T23:46:04.265 INFO:tasks.workunit.client.1.smithi044.stdout:6/222: creat d5/dd/d1d/d23/d3d/f4a x:0 0 0 2024-04-22T23:46:04.267 INFO:tasks.workunit.client.1.smithi044.stdout:0/293: symlink d0/d2e/l6f 0 2024-04-22T23:46:04.269 INFO:tasks.workunit.client.1.smithi044.stdout:1/288: rename d1/d8/d1c/d23/d3e/d4d/l5e to d1/d8/d20/d47/l6d 0 2024-04-22T23:46:04.272 INFO:tasks.workunit.client.1.smithi044.stdout:3/289: link d9/db/d15/d2e/c5a d9/d14/c61 0 2024-04-22T23:46:04.286 INFO:tasks.workunit.client.1.smithi044.stdout:0/294: rmdir d0 39 2024-04-22T23:46:04.286 INFO:tasks.workunit.client.1.smithi044.stdout:3/290: dread f7 [0,4194304] 0 2024-04-22T23:46:04.287 INFO:tasks.workunit.client.1.smithi044.stdout:1/289: dwrite d1/d8/d1c/d23/d3e/d4d/f66 [4194304,4194304] 0 2024-04-22T23:46:04.293 INFO:tasks.workunit.client.1.smithi044.stdout:6/223: rename d5/c32 to d5/dd/d28/c4b 0 2024-04-22T23:46:04.294 INFO:tasks.workunit.client.1.smithi044.stdout:7/304: truncate dd/d1a/f1d 2651382 0 2024-04-22T23:46:04.294 INFO:tasks.workunit.client.1.smithi044.stdout:7/305: stat dd/d27 0 2024-04-22T23:46:04.295 INFO:tasks.workunit.client.1.smithi044.stdout:6/224: dread d5/dd/f3e [0,4194304] 0 2024-04-22T23:46:04.301 INFO:tasks.workunit.client.1.smithi044.stdout:1/290: rmdir d1/d8/d24/d46 39 2024-04-22T23:46:04.303 INFO:tasks.workunit.client.1.smithi044.stdout:3/291: rename d9/db/d22/l42 to d9/db/d15/d2e/d48/l62 0 2024-04-22T23:46:04.306 INFO:tasks.workunit.client.1.smithi044.stdout:7/306: symlink dd/d27/l6b 0 2024-04-22T23:46:04.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/291: dread - d1/d8/d24/d46/f68 zero size 2024-04-22T23:46:04.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.326+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fd00fc5f0 con 0x7f6fbc0398c0 2024-04-22T23:46:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.328+0000 7f6fc67fc700 1 -- 172.21.15.31:0/3777826994 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f6fd00fc5f0 con 0x7f6fbc0398c0 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:46:04.320 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:46:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6fbc0398c0 msgr2=0x7f6fbc03bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6fbc0398c0 0x7f6fbc03bd80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0000c00 tx=0x7f6fc0005fb0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 msgr2=0x7f6fd0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 0x7f6fd0198670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00d8d0 tx=0x7f6fcc00dc90 comp rx=0 tx=0).stop 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 shutdown_connections 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6fbc0398c0 0x7f6fbc03bd80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fd0101800 0x7f6fd0198670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 --2- 172.21.15.31:0/3777826994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fd0104130 0x7f6fd0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 >> 172.21.15.31:0/3777826994 conn(0x7f6fd00fb190 msgr2=0x7f6fd0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 shutdown_connections 2024-04-22T23:46:04.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:04.330+0000 7f6fd7443700 1 -- 172.21.15.31:0/3777826994 wait complete. 2024-04-22T23:46:04.394 INFO:tasks.workunit.client.1.smithi044.stdout:3/292: creat d9/db/d15/f63 x:0 0 0 2024-04-22T23:46:04.398 INFO:tasks.workunit.client.1.smithi044.stdout:1/292: rmdir d1/d8/d24 39 2024-04-22T23:46:04.421 INFO:tasks.workunit.client.1.smithi044.stdout:3/293: mkdir d9/db/d33/d5d/d64 0 2024-04-22T23:46:04.423 INFO:tasks.workunit.client.1.smithi044.stdout:3/294: chown d9/db/d15/d2e/d48/l50 45 1 2024-04-22T23:46:04.427 INFO:tasks.workunit.client.1.smithi044.stdout:1/293: rename d1/d8/d34/l41 to d1/d8/d20/d47/d4a/l6e 0 2024-04-22T23:46:04.427 INFO:tasks.workunit.client.1.smithi044.stdout:0/295: dwrite d0/d2e/d33/f3a [0,4194304] 0 2024-04-22T23:46:04.428 INFO:tasks.workunit.client.1.smithi044.stdout:6/225: getdents d5 0 2024-04-22T23:46:04.434 INFO:tasks.workunit.client.1.smithi044.stdout:7/307: getdents dd/d55/d5e 0 2024-04-22T23:46:04.434 INFO:tasks.workunit.client.1.smithi044.stdout:1/294: symlink d1/d8/d1c/l6f 0 2024-04-22T23:46:04.443 INFO:tasks.workunit.client.1.smithi044.stdout:7/308: symlink dd/d1f/d24/l6c 0 2024-04-22T23:46:04.443 INFO:tasks.workunit.client.1.smithi044.stdout:1/295: fdatasync d1/d8/d20/d47/d4a/f54 0 2024-04-22T23:46:04.445 INFO:tasks.workunit.client.1.smithi044.stdout:6/226: dwrite d5/d30/f39 [0,4194304] 0 2024-04-22T23:46:04.446 INFO:tasks.workunit.client.1.smithi044.stdout:1/296: dread - d1/f67 zero size 2024-04-22T23:46:04.447 INFO:tasks.workunit.client.1.smithi044.stdout:0/296: dwrite d0/d2/dd/d3e/d40/d43/f4a [0,4194304] 0 2024-04-22T23:46:04.447 INFO:tasks.workunit.client.1.smithi044.stdout:3/295: dwrite d9/f17 [0,4194304] 0 2024-04-22T23:46:04.451 INFO:tasks.workunit.client.1.smithi044.stdout:6/227: chown d5/dd/d1d/d23/d3d/f4a 6486 1 2024-04-22T23:46:04.462 INFO:tasks.workunit.client.1.smithi044.stdout:1/297: dwrite d1/d8/d20/f44 [0,4194304] 0 2024-04-22T23:46:04.464 INFO:tasks.workunit.client.1.smithi044.stdout:6/228: rename d5/dd/d1d to d5/d3f/d4c 0 2024-04-22T23:46:04.466 INFO:tasks.workunit.client.1.smithi044.stdout:0/297: dwrite d0/d2/dd/d3e/d40/f44 [0,4194304] 0 2024-04-22T23:46:04.472 INFO:tasks.workunit.client.1.smithi044.stdout:1/298: creat d1/d8/d20/d45/f70 x:0 0 0 2024-04-22T23:46:04.473 INFO:tasks.workunit.client.1.smithi044.stdout:6/229: rmdir d5/d3f/d4c/d23 39 2024-04-22T23:46:04.474 INFO:tasks.workunit.client.1.smithi044.stdout:1/299: chown d1/d8/d34/f40 362654 1 2024-04-22T23:46:04.474 INFO:tasks.workunit.client.1.smithi044.stdout:6/230: dread - d5/d3f/d4c/f3a zero size 2024-04-22T23:46:04.475 INFO:tasks.workunit.client.1.smithi044.stdout:6/231: chown d5/f38 0 1 2024-04-22T23:46:04.478 INFO:tasks.workunit.client.1.smithi044.stdout:7/309: write dd/d1f/f31 [647845,90647] 0 2024-04-22T23:46:04.480 INFO:tasks.workunit.client.1.smithi044.stdout:0/298: rename d0/d2e/d33/f45 to d0/d2e/f70 0 2024-04-22T23:46:04.481 INFO:tasks.workunit.client.1.smithi044.stdout:6/232: creat d5/f4d x:0 0 0 2024-04-22T23:46:04.483 INFO:tasks.workunit.client.1.smithi044.stdout:7/310: write dd/d27/f5b [8768,113394] 0 2024-04-22T23:46:04.483 INFO:tasks.workunit.client.1.smithi044.stdout:7/311: fdatasync dd/d27/f62 0 2024-04-22T23:46:04.484 INFO:tasks.workunit.client.1.smithi044.stdout:1/300: rename d1/c10 to d1/d8/d24/d4b/c71 0 2024-04-22T23:46:04.486 INFO:tasks.workunit.client.1.smithi044.stdout:7/312: creat dd/d27/f6d x:0 0 0 2024-04-22T23:46:04.488 INFO:tasks.workunit.client.1.smithi044.stdout:7/313: mknod dd/d55/c6e 0 2024-04-22T23:46:04.490 INFO:tasks.workunit.client.1.smithi044.stdout:7/314: fsync dd/d27/f62 0 2024-04-22T23:46:04.490 INFO:tasks.workunit.client.1.smithi044.stdout:0/299: link d0/d2/d3d/f55 d0/d4c/f71 0 2024-04-22T23:46:04.491 INFO:tasks.workunit.client.1.smithi044.stdout:1/301: dwrite d1/d8/f12 [4194304,4194304] 0 2024-04-22T23:46:04.492 INFO:tasks.workunit.client.1.smithi044.stdout:7/315: write dd/ff [5772737,2024] 0 2024-04-22T23:46:04.492 INFO:tasks.workunit.client.1.smithi044.stdout:1/302: readlink d1/l2b 0 2024-04-22T23:46:04.496 INFO:tasks.workunit.client.1.smithi044.stdout:1/303: mkdir d1/d8/d1c/d23/d48/d72 0 2024-04-22T23:46:04.497 INFO:tasks.workunit.client.1.smithi044.stdout:1/304: fdatasync d1/f7 0 2024-04-22T23:46:04.499 INFO:tasks.workunit.client.1.smithi044.stdout:1/305: stat d1/d8/d1c/d23/d3e/c63 0 2024-04-22T23:46:04.499 INFO:tasks.workunit.client.1.smithi044.stdout:7/316: unlink dd/d55/l5f 0 2024-04-22T23:46:04.499 INFO:tasks.workunit.client.1.smithi044.stdout:1/306: readlink d1/l2b 0 2024-04-22T23:46:04.500 INFO:tasks.workunit.client.1.smithi044.stdout:1/307: dread - d1/d8/d20/d45/f70 zero size 2024-04-22T23:46:04.502 INFO:tasks.workunit.client.1.smithi044.stdout:7/317: unlink dd/d1a/f2f 0 2024-04-22T23:46:04.503 INFO:tasks.workunit.client.1.smithi044.stdout:3/296: truncate d9/d12/f2d 1389651 0 2024-04-22T23:46:04.505 INFO:tasks.workunit.client.1.smithi044.stdout:1/308: unlink d1/d8/d1c/l6f 0 2024-04-22T23:46:04.518 INFO:tasks.workunit.client.1.smithi044.stdout:1/309: mknod d1/d8/d24/c73 0 2024-04-22T23:46:04.521 INFO:tasks.workunit.client.1.smithi044.stdout:7/318: dread dd/d1f/d24/f39 [0,4194304] 0 2024-04-22T23:46:04.521 INFO:tasks.workunit.client.1.smithi044.stdout:6/233: read d5/dd/f1e [128321,112260] 0 2024-04-22T23:46:04.522 INFO:tasks.workunit.client.1.smithi044.stdout:3/297: creat d9/db/f65 x:0 0 0 2024-04-22T23:46:04.523 INFO:tasks.workunit.client.1.smithi044.stdout:3/298: write f8 [3236606,20400] 0 2024-04-22T23:46:04.528 INFO:tasks.workunit.client.1.smithi044.stdout:7/319: fsync dd/d1a/d3c/f49 0 2024-04-22T23:46:04.532 INFO:tasks.workunit.client.1.smithi044.stdout:0/300: dwrite d0/d2/dd/f1d [0,4194304] 0 2024-04-22T23:46:04.533 INFO:tasks.workunit.client.1.smithi044.stdout:4/324: sync 2024-04-22T23:46:04.533 INFO:tasks.workunit.client.1.smithi044.stdout:2/256: sync 2024-04-22T23:46:04.533 INFO:tasks.workunit.client.1.smithi044.stdout:8/322: sync 2024-04-22T23:46:04.534 INFO:tasks.workunit.client.1.smithi044.stdout:9/175: sync 2024-04-22T23:46:04.534 INFO:tasks.workunit.client.1.smithi044.stdout:5/287: sync 2024-04-22T23:46:04.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/310: mkdir d1/d8/d1c/d74 0 2024-04-22T23:46:04.535 INFO:tasks.workunit.client.1.smithi044.stdout:2/257: write f3 [2267249,42686] 0 2024-04-22T23:46:04.538 INFO:tasks.workunit.client.1.smithi044.stdout:2/258: fsync d8/d27/f4b 0 2024-04-22T23:46:04.538 INFO:tasks.workunit.client.1.smithi044.stdout:9/176: truncate d2/f3d 4048438 0 2024-04-22T23:46:04.538 INFO:tasks.workunit.client.1.smithi044.stdout:2/259: chown d8/d27/f28 72550494 1 2024-04-22T23:46:04.538 INFO:tasks.workunit.client.1.smithi044.stdout:8/323: write db/d14/f5a [407636,54447] 0 2024-04-22T23:46:04.539 INFO:tasks.workunit.client.1.smithi044.stdout:2/260: chown d8/d19/c31 7577 1 2024-04-22T23:46:04.541 INFO:tasks.workunit.client.1.smithi044.stdout:2/261: write d8/f43 [859953,49309] 0 2024-04-22T23:46:04.541 INFO:tasks.workunit.client.1.smithi044.stdout:5/288: dread d15/d16/d1d/f49 [0,4194304] 0 2024-04-22T23:46:04.544 INFO:tasks.workunit.client.1.smithi044.stdout:2/262: write d8/d19/d36/f3e [1147333,128777] 0 2024-04-22T23:46:04.549 INFO:tasks.workunit.client.1.smithi044.stdout:4/325: write d5/db/d3c/f53 [890874,6791] 0 2024-04-22T23:46:04.552 INFO:tasks.workunit.client.1.smithi044.stdout:6/234: symlink d5/d3f/d4c/d23/l4e 0 2024-04-22T23:46:04.554 INFO:tasks.workunit.client.1.smithi044.stdout:1/311: mkdir d1/d8/d20/d45/d75 0 2024-04-22T23:46:04.555 INFO:tasks.workunit.client.1.smithi044.stdout:9/177: mkdir d2/dc/d1d/d34/d41 0 2024-04-22T23:46:04.563 INFO:tasks.workunit.client.1.smithi044.stdout:8/324: rename db/d14/f60 to db/d52/f79 0 2024-04-22T23:46:04.564 INFO:tasks.workunit.client.1.smithi044.stdout:8/325: fsync db/d14/d24/f58 0 2024-04-22T23:46:04.565 INFO:tasks.workunit.client.1.smithi044.stdout:7/320: symlink dd/d1f/l6f 0 2024-04-22T23:46:04.569 INFO:tasks.workunit.client.1.smithi044.stdout:3/299: write d9/db/d22/f24 [854608,8632] 0 2024-04-22T23:46:04.574 INFO:tasks.workunit.client.1.smithi044.stdout:4/326: mkdir d5/d31/d24/d69 0 2024-04-22T23:46:04.575 INFO:tasks.workunit.client.1.smithi044.stdout:4/327: truncate d5/d42/f61 515339 0 2024-04-22T23:46:04.580 INFO:tasks.workunit.client.1.smithi044.stdout:8/326: creat db/d68/f7a x:0 0 0 2024-04-22T23:46:04.581 INFO:tasks.workunit.client.1.smithi044.stdout:7/321: readlink dd/d1f/l50 0 2024-04-22T23:46:04.584 INFO:tasks.workunit.client.1.smithi044.stdout:5/289: link d15/d16/d1d/f3f d15/d16/d2c/f5a 0 2024-04-22T23:46:04.585 INFO:tasks.workunit.client.1.smithi044.stdout:9/178: rmdir d2/dc 39 2024-04-22T23:46:04.590 INFO:tasks.workunit.client.1.smithi044.stdout:2/263: link d8/d19/d36/d46/f4e d8/d19/d36/d44/f55 0 2024-04-22T23:46:04.591 INFO:tasks.workunit.client.1.smithi044.stdout:4/328: dread d5/d31/d44/d20/d22/f65 [0,4194304] 0 2024-04-22T23:46:04.591 INFO:tasks.workunit.client.1.smithi044.stdout:3/300: creat d9/db/d33/d5d/f66 x:0 0 0 2024-04-22T23:46:04.592 INFO:tasks.workunit.client.1.smithi044.stdout:3/301: chown d9/db/d33/d5d/d64 2611 1 2024-04-22T23:46:04.593 INFO:tasks.workunit.client.1.smithi044.stdout:4/329: read d5/d31/d44/d20/d22/f35 [3662389,49043] 0 2024-04-22T23:46:04.595 INFO:tasks.workunit.client.1.smithi044.stdout:6/235: mkdir d5/d3f/d4c/d4f 0 2024-04-22T23:46:04.596 INFO:tasks.workunit.client.1.smithi044.stdout:4/330: write d5/d31/d44/d20/d22/f35 [4142269,114323] 0 2024-04-22T23:46:04.597 INFO:tasks.workunit.client.1.smithi044.stdout:3/302: dread d9/db/f13 [0,4194304] 0 2024-04-22T23:46:04.600 INFO:tasks.workunit.client.1.smithi044.stdout:7/322: creat dd/d1f/d24/f70 x:0 0 0 2024-04-22T23:46:04.601 INFO:tasks.workunit.client.1.smithi044.stdout:5/290: symlink d15/d16/d1d/l5b 0 2024-04-22T23:46:04.606 INFO:tasks.workunit.client.1.smithi044.stdout:0/301: getdents d0/d2/d3d 0 2024-04-22T23:46:04.611 INFO:tasks.workunit.client.1.smithi044.stdout:6/236: mkdir d5/d3f/d50 0 2024-04-22T23:46:04.614 INFO:tasks.workunit.client.1.smithi044.stdout:3/303: mkdir d9/db/d15/d25/d67 0 2024-04-22T23:46:04.616 INFO:tasks.workunit.client.1.smithi044.stdout:4/331: mkdir d5/d31/d24/d38/d6a 0 2024-04-22T23:46:04.617 INFO:tasks.workunit.client.1.smithi044.stdout:7/323: mkdir dd/d55/d71 0 2024-04-22T23:46:04.617 INFO:tasks.workunit.client.1.smithi044.stdout:7/324: fsync dd/f59 0 2024-04-22T23:46:04.623 INFO:tasks.workunit.client.1.smithi044.stdout:0/302: creat d0/d2/d4b/f72 x:0 0 0 2024-04-22T23:46:04.627 INFO:tasks.workunit.client.1.smithi044.stdout:9/179: dwrite d2/d5/f30 [0,4194304] 0 2024-04-22T23:46:04.630 INFO:tasks.workunit.client.1.smithi044.stdout:1/312: getdents d1/d8/d24/d4b 0 2024-04-22T23:46:04.632 INFO:tasks.workunit.client.1.smithi044.stdout:1/313: write d1/d8/f1a [1163760,58801] 0 2024-04-22T23:46:04.633 INFO:tasks.workunit.client.1.smithi044.stdout:6/237: unlink d5/dd/d18/f2a 0 2024-04-22T23:46:04.641 INFO:tasks.workunit.client.1.smithi044.stdout:7/325: read dd/d1a/f1d [620312,63217] 0 2024-04-22T23:46:04.642 INFO:tasks.workunit.client.1.smithi044.stdout:7/326: stat dd/d1f/d24 0 2024-04-22T23:46:04.643 INFO:tasks.workunit.client.1.smithi044.stdout:5/291: dread d15/d16/f28 [0,4194304] 0 2024-04-22T23:46:04.650 INFO:tasks.workunit.client.1.smithi044.stdout:9/180: rmdir d2/d6 39 2024-04-22T23:46:04.652 INFO:tasks.workunit.client.1.smithi044.stdout:7/327: symlink dd/d55/l72 0 2024-04-22T23:46:04.654 INFO:tasks.workunit.client.1.smithi044.stdout:6/238: dread d5/d3f/f22 [0,4194304] 0 2024-04-22T23:46:04.658 INFO:tasks.workunit.client.1.smithi044.stdout:1/314: rename d1/d8/d20/d47/l6d to d1/d8/d20/d45/l76 0 2024-04-22T23:46:04.665 INFO:tasks.workunit.client.1.smithi044.stdout:0/303: link d0/d2e/f32 d0/d2/dd/d3e/d40/f73 0 2024-04-22T23:46:04.666 INFO:tasks.workunit.client.1.smithi044.stdout:9/181: stat d2/d6/l21 0 2024-04-22T23:46:04.674 INFO:tasks.workunit.client.1.smithi044.stdout:7/328: dwrite dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:04.675 INFO:tasks.workunit.client.1.smithi044.stdout:5/292: mknod d15/d16/c5c 0 2024-04-22T23:46:04.679 INFO:tasks.workunit.client.1.smithi044.stdout:4/332: link d5/d31/d44/d20/d22/l45 d5/l6b 0 2024-04-22T23:46:04.679 INFO:tasks.workunit.client.1.smithi044.stdout:6/239: chown d5/d30/c45 1710 1 2024-04-22T23:46:04.680 INFO:tasks.workunit.client.1.smithi044.stdout:5/293: dread - d15/d16/d2c/d38/f50 zero size 2024-04-22T23:46:04.685 INFO:tasks.workunit.client.1.smithi044.stdout:9/182: mkdir d2/d6/d42 0 2024-04-22T23:46:04.686 INFO:tasks.workunit.client.1.smithi044.stdout:9/183: readlink d2/d6/l29 0 2024-04-22T23:46:04.688 INFO:tasks.workunit.client.1.smithi044.stdout:0/304: truncate d0/d2/f6 628437 0 2024-04-22T23:46:04.693 INFO:tasks.workunit.client.1.smithi044.stdout:9/184: dwrite d2/d6/f33 [0,4194304] 0 2024-04-22T23:46:04.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:04 smithi031 ceph-mon[26721]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:04.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:04 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3580776039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:04.696 INFO:tasks.workunit.client.1.smithi044.stdout:1/315: mkdir d1/d8/d20/d45/d75/d77 0 2024-04-22T23:46:04.703 INFO:tasks.workunit.client.1.smithi044.stdout:6/240: creat d5/d3f/d4c/d23/d3d/f51 x:0 0 0 2024-04-22T23:46:04.709 INFO:tasks.workunit.client.1.smithi044.stdout:2/264: dwrite f6 [0,4194304] 0 2024-04-22T23:46:04.709 INFO:tasks.workunit.client.1.smithi044.stdout:1/316: dwrite d1/d8/d1c/f5b [0,4194304] 0 2024-04-22T23:46:04.711 INFO:tasks.workunit.client.1.smithi044.stdout:1/317: dread - d1/d8/d24/d4b/f5a zero size 2024-04-22T23:46:04.715 INFO:tasks.workunit.client.1.smithi044.stdout:9/185: creat d2/d6/f43 x:0 0 0 2024-04-22T23:46:04.716 INFO:tasks.workunit.client.1.smithi044.stdout:7/329: creat dd/d55/d5e/d60/d68/f73 x:0 0 0 2024-04-22T23:46:04.723 INFO:tasks.workunit.client.1.smithi044.stdout:1/318: dwrite d1/d8/d34/f43 [0,4194304] 0 2024-04-22T23:46:04.724 INFO:tasks.workunit.client.1.smithi044.stdout:1/319: truncate d1/d8/d34/f43 4722364 0 2024-04-22T23:46:04.726 INFO:tasks.workunit.client.1.smithi044.stdout:3/304: write d9/db/d15/d2e/f32 [561860,97857] 0 2024-04-22T23:46:04.726 INFO:tasks.workunit.client.1.smithi044.stdout:6/241: fdatasync d5/dd/f16 0 2024-04-22T23:46:04.730 INFO:tasks.workunit.client.1.smithi044.stdout:3/305: truncate d9/db/d15/d25/d5b/d52/f59 1437597 0 2024-04-22T23:46:04.733 INFO:tasks.workunit.client.1.smithi044.stdout:1/320: dread d1/d8/d1c/d23/d3e/d4d/f66 [4194304,4194304] 0 2024-04-22T23:46:04.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/330: creat dd/d1a/d3c/f74 x:0 0 0 2024-04-22T23:46:04.742 INFO:tasks.workunit.client.1.smithi044.stdout:1/321: dread d1/d8/d20/d45/f3f [0,4194304] 0 2024-04-22T23:46:04.742 INFO:tasks.workunit.client.1.smithi044.stdout:1/322: readlink d1/l6 0 2024-04-22T23:46:04.743 INFO:tasks.workunit.client.1.smithi044.stdout:1/323: write d1/d8/d34/f40 [68551,84664] 0 2024-04-22T23:46:04.744 INFO:tasks.workunit.client.1.smithi044.stdout:2/265: link d8/f41 d8/d27/f56 0 2024-04-22T23:46:04.745 INFO:tasks.workunit.client.1.smithi044.stdout:9/186: mknod d2/d9/c44 0 2024-04-22T23:46:04.750 INFO:tasks.workunit.client.1.smithi044.stdout:5/294: dwrite d15/d16/d1d/f49 [0,4194304] 0 2024-04-22T23:46:04.751 INFO:tasks.workunit.client.1.smithi044.stdout:5/295: write d15/f30 [5186234,8668] 0 2024-04-22T23:46:04.753 INFO:tasks.workunit.client.1.smithi044.stdout:7/331: mkdir dd/d55/d5e/d60/d75 0 2024-04-22T23:46:04.754 INFO:tasks.workunit.client.1.smithi044.stdout:4/333: getdents d5/d31/d24/d38 0 2024-04-22T23:46:04.757 INFO:tasks.workunit.client.1.smithi044.stdout:7/332: read dd/d1f/d24/f19 [2774297,62358] 0 2024-04-22T23:46:04.762 INFO:tasks.workunit.client.1.smithi044.stdout:1/324: rmdir d1/d8/d24/d4b 39 2024-04-22T23:46:04.764 INFO:tasks.workunit.client.1.smithi044.stdout:2/266: unlink d8/d19/l22 0 2024-04-22T23:46:04.771 INFO:tasks.workunit.client.1.smithi044.stdout:7/333: mknod dd/d55/d5e/c76 0 2024-04-22T23:46:04.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/267: dread d8/d19/d36/f3e [0,4194304] 0 2024-04-22T23:46:04.772 INFO:tasks.workunit.client.1.smithi044.stdout:6/242: rename d5/d3f/l49 to d5/d3f/d4c/l52 0 2024-04-22T23:46:04.773 INFO:tasks.workunit.client.1.smithi044.stdout:6/243: chown d5/f4d 660518880 1 2024-04-22T23:46:04.774 INFO:tasks.workunit.client.1.smithi044.stdout:2/268: dread d8/d19/d36/f3e [0,4194304] 0 2024-04-22T23:46:04.776 INFO:tasks.workunit.client.1.smithi044.stdout:1/325: creat d1/d8/d34/f78 x:0 0 0 2024-04-22T23:46:04.776 INFO:tasks.workunit.client.1.smithi044.stdout:5/296: dread d15/d16/d2c/f54 [0,4194304] 0 2024-04-22T23:46:04.777 INFO:tasks.workunit.client.1.smithi044.stdout:1/326: dread - d1/d14/f36 zero size 2024-04-22T23:46:04.778 INFO:tasks.workunit.client.1.smithi044.stdout:5/297: chown d15/d16/d2c/d38/f50 197 1 2024-04-22T23:46:04.780 INFO:tasks.workunit.client.1.smithi044.stdout:2/269: dread d8/d19/f1f [0,4194304] 0 2024-04-22T23:46:04.781 INFO:tasks.workunit.client.1.smithi044.stdout:4/334: creat d5/d31/d24/d69/f6c x:0 0 0 2024-04-22T23:46:04.784 INFO:tasks.workunit.client.1.smithi044.stdout:6/244: mkdir d5/d3f/d4c/d40/d53 0 2024-04-22T23:46:04.785 INFO:tasks.workunit.client.1.smithi044.stdout:1/327: dwrite d1/d8/d20/d45/f29 [0,4194304] 0 2024-04-22T23:46:04.789 INFO:tasks.workunit.client.1.smithi044.stdout:3/306: truncate d9/db/d15/d25/f3b 1972659 0 2024-04-22T23:46:04.790 INFO:tasks.workunit.client.1.smithi044.stdout:3/307: fdatasync f5 0 2024-04-22T23:46:04.791 INFO:tasks.workunit.client.1.smithi044.stdout:3/308: truncate f8 5066649 0 2024-04-22T23:46:04.794 INFO:tasks.workunit.client.1.smithi044.stdout:2/270: truncate d8/d27/f2c 337003 0 2024-04-22T23:46:04.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/335: chown d5/d31/d24/l4f 20918 1 2024-04-22T23:46:04.798 INFO:tasks.workunit.client.1.smithi044.stdout:6/245: creat d5/d3f/f54 x:0 0 0 2024-04-22T23:46:04.801 INFO:tasks.workunit.client.1.smithi044.stdout:6/246: truncate d5/dd/d28/f2c 385841 0 2024-04-22T23:46:04.802 INFO:tasks.workunit.client.1.smithi044.stdout:2/271: dwrite d8/d27/f2a [0,4194304] 0 2024-04-22T23:46:04.803 INFO:tasks.workunit.client.1.smithi044.stdout:3/309: chown d9/f2a 0 1 2024-04-22T23:46:04.807 INFO:tasks.workunit.client.1.smithi044.stdout:7/334: creat dd/d1f/f77 x:0 0 0 2024-04-22T23:46:04.807 INFO:tasks.workunit.client.1.smithi044.stdout:7/335: chown dd/d1f 38474325 1 2024-04-22T23:46:04.808 INFO:tasks.workunit.client.1.smithi044.stdout:7/336: write dd/d1a/d3c/f49 [2906588,71161] 0 2024-04-22T23:46:04.813 INFO:tasks.workunit.client.1.smithi044.stdout:5/298: rename d15/d16/d1d/d22/d2a/d3a/d43/d55 to d15/d5d 0 2024-04-22T23:46:04.814 INFO:tasks.workunit.client.1.smithi044.stdout:5/299: readlink d15/d16/d1d/l5b 0 2024-04-22T23:46:04.815 INFO:tasks.workunit.client.1.smithi044.stdout:7/337: dwrite f5 [0,4194304] 0 2024-04-22T23:46:04.821 INFO:tasks.workunit.client.1.smithi044.stdout:3/310: unlink d9/f17 0 2024-04-22T23:46:04.821 INFO:tasks.workunit.client.1.smithi044.stdout:7/338: dread dd/d1f/f34 [0,4194304] 0 2024-04-22T23:46:04.823 INFO:tasks.workunit.client.1.smithi044.stdout:3/311: truncate d9/db/d22/f24 1871372 0 2024-04-22T23:46:04.824 INFO:tasks.workunit.client.1.smithi044.stdout:3/312: dread - d9/db/d15/d2e/f3f zero size 2024-04-22T23:46:04.824 INFO:tasks.workunit.client.1.smithi044.stdout:6/247: symlink d5/d3f/d4c/d23/l55 0 2024-04-22T23:46:04.828 INFO:tasks.workunit.client.1.smithi044.stdout:3/313: dread f7 [0,4194304] 0 2024-04-22T23:46:04.828 INFO:tasks.workunit.client.1.smithi044.stdout:1/328: creat d1/d8/d1c/f79 x:0 0 0 2024-04-22T23:46:04.835 INFO:tasks.workunit.client.1.smithi044.stdout:1/329: dwrite d1/d8/f19 [0,4194304] 0 2024-04-22T23:46:04.838 INFO:tasks.workunit.client.1.smithi044.stdout:5/300: rename d15/d16/f17 to d15/d5d/f5e 0 2024-04-22T23:46:04.839 INFO:tasks.workunit.client.1.smithi044.stdout:5/301: write d15/d16/f36 [941247,43587] 0 2024-04-22T23:46:04.839 INFO:tasks.workunit.client.1.smithi044.stdout:2/272: creat d8/d19/d36/d46/d54/f57 x:0 0 0 2024-04-22T23:46:04.842 INFO:tasks.workunit.client.1.smithi044.stdout:7/339: mknod dd/d55/d5e/d60/d68/c78 0 2024-04-22T23:46:04.843 INFO:tasks.workunit.client.1.smithi044.stdout:7/340: readlink dd/d55/l72 0 2024-04-22T23:46:04.845 INFO:tasks.workunit.client.1.smithi044.stdout:6/248: write d5/d3f/d4c/d23/f29 [1240919,29978] 0 2024-04-22T23:46:04.862 INFO:tasks.workunit.client.1.smithi044.stdout:7/341: unlink dd/l4e 0 2024-04-22T23:46:04.870 INFO:tasks.workunit.client.1.smithi044.stdout:1/330: unlink d1/d8/d24/d4b/f6b 0 2024-04-22T23:46:04.871 INFO:tasks.workunit.client.1.smithi044.stdout:1/331: write d1/d8/d1c/d23/d3e/d4d/f65 [183969,16286] 0 2024-04-22T23:46:04.872 INFO:tasks.workunit.client.1.smithi044.stdout:1/332: write d1/d8/d1c/d23/d3e/f58 [4041886,57997] 0 2024-04-22T23:46:04.876 INFO:tasks.workunit.client.1.smithi044.stdout:2/273: symlink d8/d19/d53/l58 0 2024-04-22T23:46:04.877 INFO:tasks.workunit.client.1.smithi044.stdout:4/336: link d5/d31/d44/d20/d22/c2c d5/db/c6d 0 2024-04-22T23:46:04.877 INFO:tasks.workunit.client.1.smithi044.stdout:1/333: dread d1/d8/d20/d45/f29 [0,4194304] 0 2024-04-22T23:46:04.879 INFO:tasks.workunit.client.1.smithi044.stdout:4/337: fsync d5/d31/d44/f30 0 2024-04-22T23:46:04.883 INFO:tasks.workunit.client.1.smithi044.stdout:2/274: dwrite f3 [0,4194304] 0 2024-04-22T23:46:04.885 INFO:tasks.workunit.client.1.smithi044.stdout:2/275: write d8/d19/f2b [919374,76329] 0 2024-04-22T23:46:04.885 INFO:tasks.workunit.client.1.smithi044.stdout:3/314: creat d9/db/d22/f68 x:0 0 0 2024-04-22T23:46:04.886 INFO:tasks.workunit.client.1.smithi044.stdout:6/249: mkdir d5/d3f/d4c/d40/d53/d56 0 2024-04-22T23:46:04.887 INFO:tasks.workunit.client.1.smithi044.stdout:3/315: chown d9/db/d33/l3d 1334616 1 2024-04-22T23:46:04.901 INFO:tasks.workunit.client.1.smithi044.stdout:7/342: truncate dd/f4a 686245 0 2024-04-22T23:46:04.901 INFO:tasks.workunit.client.1.smithi044.stdout:7/343: dread - dd/d27/f62 zero size 2024-04-22T23:46:04.907 INFO:tasks.workunit.client.1.smithi044.stdout:2/276: truncate d8/d19/f1f 1389987 0 2024-04-22T23:46:04.907 INFO:tasks.workunit.client.1.smithi044.stdout:5/302: link l9 d15/d16/d2c/d38/l5f 0 2024-04-22T23:46:04.908 INFO:tasks.workunit.client.1.smithi044.stdout:2/277: write d8/d19/f29 [1022457,42463] 0 2024-04-22T23:46:04.918 INFO:tasks.workunit.client.1.smithi044.stdout:5/303: fsync d15/d16/f28 0 2024-04-22T23:46:04.921 INFO:tasks.workunit.client.1.smithi044.stdout:4/338: write d5/d31/d24/f3d [729810,8490] 0 2024-04-22T23:46:04.922 INFO:tasks.workunit.client.1.smithi044.stdout:4/339: read - d5/d31/d24/d38/f40 zero size 2024-04-22T23:46:04.923 INFO:tasks.workunit.client.1.smithi044.stdout:8/327: sync 2024-04-22T23:46:04.924 INFO:tasks.workunit.client.1.smithi044.stdout:4/340: fsync d5/d31/d44/f30 0 2024-04-22T23:46:04.925 INFO:tasks.workunit.client.1.smithi044.stdout:7/344: mkdir dd/d79 0 2024-04-22T23:46:04.927 INFO:tasks.workunit.client.1.smithi044.stdout:6/250: creat d5/d3f/d4c/f57 x:0 0 0 2024-04-22T23:46:04.932 INFO:tasks.workunit.client.1.smithi044.stdout:2/278: symlink d8/d19/l59 0 2024-04-22T23:46:04.933 INFO:tasks.workunit.client.1.smithi044.stdout:2/279: fdatasync d8/d19/f51 0 2024-04-22T23:46:04.933 INFO:tasks.workunit.client.1.smithi044.stdout:5/304: creat d15/d16/d1d/d22/d2a/f60 x:0 0 0 2024-04-22T23:46:04.934 INFO:tasks.workunit.client.1.smithi044.stdout:3/316: link d9/db/d22/d38/c3e d9/db/d33/d5d/d64/c69 0 2024-04-22T23:46:04.935 INFO:tasks.workunit.client.1.smithi044.stdout:3/317: readlink d9/l51 0 2024-04-22T23:46:04.935 INFO:tasks.workunit.client.1.smithi044.stdout:1/334: getdents d1/d8/d1c/d23/d3e/d4d 0 2024-04-22T23:46:04.936 INFO:tasks.workunit.client.1.smithi044.stdout:5/305: write d15/d16/d1d/d22/d2a/d3a/f4b [1681990,58174] 0 2024-04-22T23:46:04.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/251: dread d5/d3f/f22 [0,4194304] 0 2024-04-22T23:46:04.938 INFO:tasks.workunit.client.1.smithi044.stdout:8/328: rmdir db/d6c 39 2024-04-22T23:46:04.939 INFO:tasks.workunit.client.1.smithi044.stdout:1/335: write d1/d8/d1c/f5b [446814,108228] 0 2024-04-22T23:46:04.940 INFO:tasks.workunit.client.1.smithi044.stdout:3/318: write d9/db/d15/d25/d5b/d52/f59 [146075,57150] 0 2024-04-22T23:46:04.955 INFO:tasks.workunit.client.1.smithi044.stdout:4/341: dwrite d5/d31/d44/d20/d22/f65 [0,4194304] 0 2024-04-22T23:46:04.956 INFO:tasks.workunit.client.1.smithi044.stdout:9/187: sync 2024-04-22T23:46:04.956 INFO:tasks.workunit.client.1.smithi044.stdout:0/305: sync 2024-04-22T23:46:04.958 INFO:tasks.workunit.client.1.smithi044.stdout:8/329: dread db/f27 [0,4194304] 0 2024-04-22T23:46:04.958 INFO:tasks.workunit.client.1.smithi044.stdout:9/188: readlink d2/d6/l35 0 2024-04-22T23:46:04.959 INFO:tasks.workunit.client.1.smithi044.stdout:8/330: truncate db/f6a 483667 0 2024-04-22T23:46:04.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:04 smithi044 ceph-mon[34463]: from='client.24485 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:04.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:04 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3580776039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:04.961 INFO:tasks.workunit.client.1.smithi044.stdout:7/345: dwrite dd/f3a [0,4194304] 0 2024-04-22T23:46:04.962 INFO:tasks.workunit.client.1.smithi044.stdout:7/346: truncate dd/d55/f58 431213 0 2024-04-22T23:46:04.964 INFO:tasks.workunit.client.1.smithi044.stdout:6/252: unlink d5/d30/c45 0 2024-04-22T23:46:04.965 INFO:tasks.workunit.client.1.smithi044.stdout:6/253: stat d5/dd/d18/c43 0 2024-04-22T23:46:04.966 INFO:tasks.workunit.client.1.smithi044.stdout:5/306: fdatasync d15/d16/d1d/d22/f23 0 2024-04-22T23:46:04.968 INFO:tasks.workunit.client.1.smithi044.stdout:3/319: write d9/db/d22/f56 [8343734,94976] 0 2024-04-22T23:46:04.972 INFO:tasks.workunit.client.1.smithi044.stdout:4/342: fsync d5/d31/f1b 0 2024-04-22T23:46:04.973 INFO:tasks.workunit.client.1.smithi044.stdout:0/306: creat d0/d2/d4b/f74 x:0 0 0 2024-04-22T23:46:04.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/347: mknod dd/d1f/d24/c7a 0 2024-04-22T23:46:04.975 INFO:tasks.workunit.client.1.smithi044.stdout:7/348: fsync dd/f54 0 2024-04-22T23:46:04.976 INFO:tasks.workunit.client.1.smithi044.stdout:2/280: symlink d8/d19/l5a 0 2024-04-22T23:46:04.977 INFO:tasks.workunit.client.1.smithi044.stdout:1/336: symlink d1/d8/d1c/d74/l7a 0 2024-04-22T23:46:04.977 INFO:tasks.workunit.client.1.smithi044.stdout:2/281: chown d8/d27/f4f 49 1 2024-04-22T23:46:04.979 INFO:tasks.workunit.client.1.smithi044.stdout:1/337: truncate d1/d8/d1c/d23/d3e/d4d/f65 1240124 0 2024-04-22T23:46:04.980 INFO:tasks.workunit.client.1.smithi044.stdout:1/338: dread - d1/d8/d1c/d23/f5f zero size 2024-04-22T23:46:04.981 INFO:tasks.workunit.client.1.smithi044.stdout:1/339: write d1/d8/d1c/d23/f5f [292521,95610] 0 2024-04-22T23:46:04.982 INFO:tasks.workunit.client.1.smithi044.stdout:3/320: rename d9/f23 to d9/db/d15/d25/d4c/f6a 0 2024-04-22T23:46:04.986 INFO:tasks.workunit.client.1.smithi044.stdout:0/307: creat d0/d4c/f75 x:0 0 0 2024-04-22T23:46:04.989 INFO:tasks.workunit.client.1.smithi044.stdout:8/331: creat db/d14/d72/f7b x:0 0 0 2024-04-22T23:46:04.995 INFO:tasks.workunit.client.1.smithi044.stdout:2/282: read d8/ff [1561359,66498] 0 2024-04-22T23:46:05.000 INFO:tasks.workunit.client.1.smithi044.stdout:5/307: getdents d15/d16/d1d/d22/d40 0 2024-04-22T23:46:05.004 INFO:tasks.workunit.client.1.smithi044.stdout:4/343: symlink d5/l6e 0 2024-04-22T23:46:05.008 INFO:tasks.workunit.client.1.smithi044.stdout:0/308: readlink d0/d2e/l3c 0 2024-04-22T23:46:05.014 INFO:tasks.workunit.client.1.smithi044.stdout:0/309: dwrite d0/f22 [0,4194304] 0 2024-04-22T23:46:05.018 INFO:tasks.workunit.client.1.smithi044.stdout:5/308: read f12 [1674798,18301] 0 2024-04-22T23:46:05.023 INFO:tasks.workunit.client.1.smithi044.stdout:7/349: dwrite dd/d1f/f34 [0,4194304] 0 2024-04-22T23:46:05.025 INFO:tasks.workunit.client.1.smithi044.stdout:9/189: getdents d2 0 2024-04-22T23:46:05.035 INFO:tasks.workunit.client.1.smithi044.stdout:5/309: dread f12 [0,4194304] 0 2024-04-22T23:46:05.036 INFO:tasks.workunit.client.1.smithi044.stdout:0/310: unlink d0/f22 0 2024-04-22T23:46:05.036 INFO:tasks.workunit.client.1.smithi044.stdout:3/321: truncate d9/db/d15/f21 118052 0 2024-04-22T23:46:05.038 INFO:tasks.workunit.client.1.smithi044.stdout:6/254: getdents d5/dd/d28 0 2024-04-22T23:46:05.040 INFO:tasks.workunit.client.1.smithi044.stdout:1/340: rename d1/d8/d1c/d23/d3e/d4d to d1/d8/d1c/d7b 0 2024-04-22T23:46:05.045 INFO:tasks.workunit.client.1.smithi044.stdout:7/350: mkdir dd/d55/d5e/d60/d7b 0 2024-04-22T23:46:05.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/341: dwrite d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:05.047 INFO:tasks.workunit.client.1.smithi044.stdout:7/351: chown dd/d1a/l1b 570895 1 2024-04-22T23:46:05.047 INFO:tasks.workunit.client.1.smithi044.stdout:7/352: fdatasync dd/d27/f46 0 2024-04-22T23:46:05.048 INFO:tasks.workunit.client.1.smithi044.stdout:7/353: chown dd/d1f/f34 90600 1 2024-04-22T23:46:05.049 INFO:tasks.workunit.client.1.smithi044.stdout:7/354: dread - dd/d27/f6d zero size 2024-04-22T23:46:05.059 INFO:tasks.workunit.client.1.smithi044.stdout:5/310: chown f11 153 1 2024-04-22T23:46:05.061 INFO:tasks.workunit.client.1.smithi044.stdout:3/322: unlink d9/db/d33/l3d 0 2024-04-22T23:46:05.069 INFO:tasks.workunit.client.1.smithi044.stdout:6/255: rename d5/d3f/f35 to d5/d3f/d4c/d40/d53/d56/f58 0 2024-04-22T23:46:05.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/256: dread - d5/dd/d28/f2e zero size 2024-04-22T23:46:05.071 INFO:tasks.workunit.client.1.smithi044.stdout:6/257: fsync d5/d3f/d4c/d23/f29 0 2024-04-22T23:46:05.220 INFO:tasks.workunit.client.1.smithi044.stdout:4/344: rmdir d5/db/d34/d64 0 2024-04-22T23:46:05.221 INFO:tasks.workunit.client.1.smithi044.stdout:4/345: write d5/db/d34/f59 [13895,20358] 0 2024-04-22T23:46:05.225 INFO:tasks.workunit.client.1.smithi044.stdout:1/342: chown d1/d8/d34/l35 387165992 1 2024-04-22T23:46:05.226 INFO:tasks.workunit.client.1.smithi044.stdout:1/343: write d1/d8/d24/d4b/f5a [566849,62928] 0 2024-04-22T23:46:05.230 INFO:tasks.workunit.client.1.smithi044.stdout:9/190: creat d2/dc/d1d/d3f/f45 x:0 0 0 2024-04-22T23:46:05.234 INFO:tasks.workunit.client.1.smithi044.stdout:7/355: symlink dd/d1a/d3c/l7c 0 2024-04-22T23:46:05.234 INFO:tasks.workunit.client.1.smithi044.stdout:1/344: dwrite d1/d8/f19 [0,4194304] 0 2024-04-22T23:46:05.236 INFO:tasks.workunit.client.1.smithi044.stdout:2/283: getdents d8/d19/d36/d44 0 2024-04-22T23:46:05.262 INFO:tasks.workunit.client.1.smithi044.stdout:5/311: rename d15/d16/f28 to d15/d16/d1d/d22/d2a/d3a/d43/f61 0 2024-04-22T23:46:05.272 INFO:tasks.workunit.client.1.smithi044.stdout:4/346: creat d5/d31/d44/d20/d22/f6f x:0 0 0 2024-04-22T23:46:05.273 INFO:tasks.workunit.client.1.smithi044.stdout:8/332: sync 2024-04-22T23:46:05.274 INFO:tasks.workunit.client.1.smithi044.stdout:8/333: dread - db/d52/f5e zero size 2024-04-22T23:46:05.278 INFO:tasks.workunit.client.1.smithi044.stdout:9/191: mknod d2/dc/d1d/d34/d38/c46 0 2024-04-22T23:46:05.284 INFO:tasks.workunit.client.1.smithi044.stdout:2/284: mkdir d8/d27/d5b 0 2024-04-22T23:46:05.286 INFO:tasks.workunit.client.1.smithi044.stdout:3/323: mkdir d9/db/d15/d25/d67/d6b 0 2024-04-22T23:46:05.289 INFO:tasks.workunit.client.1.smithi044.stdout:6/258: creat d5/d3f/d4c/d4f/f59 x:0 0 0 2024-04-22T23:46:05.296 INFO:tasks.workunit.client.1.smithi044.stdout:3/324: truncate d9/db/f13 962319 0 2024-04-22T23:46:05.297 INFO:tasks.workunit.client.1.smithi044.stdout:3/325: write d9/db/d15/d25/d5b/f40 [486626,127065] 0 2024-04-22T23:46:05.313 INFO:tasks.workunit.client.1.smithi044.stdout:9/192: link d2/d5/ld d2/dc/d1d/d3f/l47 0 2024-04-22T23:46:05.315 INFO:tasks.workunit.client.1.smithi044.stdout:1/345: getdents d1/d8/d24 0 2024-04-22T23:46:05.317 INFO:tasks.workunit.client.1.smithi044.stdout:3/326: creat d9/db/d15/d54/d3c/f6c x:0 0 0 2024-04-22T23:46:05.319 INFO:tasks.workunit.client.1.smithi044.stdout:9/193: creat d2/dc/d1d/d3f/f48 x:0 0 0 2024-04-22T23:46:05.319 INFO:tasks.workunit.client.1.smithi044.stdout:1/346: dread d1/d8/d1c/d7b/f66 [4194304,4194304] 0 2024-04-22T23:46:05.320 INFO:tasks.workunit.client.1.smithi044.stdout:2/285: getdents d8 0 2024-04-22T23:46:05.321 INFO:tasks.workunit.client.1.smithi044.stdout:1/347: stat d1/d8/d20/d45/d5d/f61 0 2024-04-22T23:46:05.323 INFO:tasks.workunit.client.1.smithi044.stdout:3/327: unlink d9/d14/l49 0 2024-04-22T23:46:05.324 INFO:tasks.workunit.client.1.smithi044.stdout:3/328: stat d9/d14/c1e 0 2024-04-22T23:46:05.325 INFO:tasks.workunit.client.1.smithi044.stdout:9/194: rmdir d2/dc/d1d/d34/d38 39 2024-04-22T23:46:05.326 INFO:tasks.workunit.client.1.smithi044.stdout:1/348: dread d1/d8/d34/f43 [0,4194304] 0 2024-04-22T23:46:05.326 INFO:tasks.workunit.client.1.smithi044.stdout:9/195: dread - d2/dc/d1d/d3f/f48 zero size 2024-04-22T23:46:05.330 INFO:tasks.workunit.client.1.smithi044.stdout:3/329: creat d9/db/d15/d25/f6d x:0 0 0 2024-04-22T23:46:05.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/349: unlink d1/d8/d20/f44 0 2024-04-22T23:46:05.337 INFO:tasks.workunit.client.1.smithi044.stdout:1/350: creat d1/d8/d1c/d23/f7c x:0 0 0 2024-04-22T23:46:05.338 INFO:tasks.workunit.client.1.smithi044.stdout:1/351: write d1/d8/d1c/d23/d3e/f58 [1214113,49933] 0 2024-04-22T23:46:05.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/196: dread d2/dc/f18 [0,4194304] 0 2024-04-22T23:46:05.344 INFO:tasks.workunit.client.1.smithi044.stdout:1/352: fsync d1/d8/d1c/d7b/f66 0 2024-04-22T23:46:05.345 INFO:tasks.workunit.client.1.smithi044.stdout:3/330: link d9/db/d15/d25/c4f d9/db/d15/d54/d3c/c6e 0 2024-04-22T23:46:05.350 INFO:tasks.workunit.client.1.smithi044.stdout:1/353: creat d1/d8/d20/f7d x:0 0 0 2024-04-22T23:46:05.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/331: creat d9/db/d15/d25/d5b/d52/f6f x:0 0 0 2024-04-22T23:46:05.355 INFO:tasks.workunit.client.1.smithi044.stdout:9/197: creat d2/dc/d1d/d34/f49 x:0 0 0 2024-04-22T23:46:05.355 INFO:tasks.workunit.client.1.smithi044.stdout:7/356: rmdir dd/d1f/d24 39 2024-04-22T23:46:05.356 INFO:tasks.workunit.client.1.smithi044.stdout:9/198: dread - d2/dc/d1d/d3f/f48 zero size 2024-04-22T23:46:05.357 INFO:tasks.workunit.client.1.smithi044.stdout:8/334: rename db/d68/l6b to db/d14/l7c 0 2024-04-22T23:46:05.362 INFO:tasks.workunit.client.1.smithi044.stdout:3/332: unlink d9/db/d15/d54/c46 0 2024-04-22T23:46:05.362 INFO:tasks.workunit.client.1.smithi044.stdout:3/333: stat d9/db/d15/d25/d5b/d52/f59 0 2024-04-22T23:46:05.362 INFO:tasks.workunit.client.1.smithi044.stdout:7/357: creat dd/d1a/f7d x:0 0 0 2024-04-22T23:46:05.368 INFO:tasks.workunit.client.1.smithi044.stdout:3/334: mkdir d9/db/d33/d5d/d70 0 2024-04-22T23:46:05.369 INFO:tasks.workunit.client.1.smithi044.stdout:3/335: write d9/db/d22/f56 [7508436,9316] 0 2024-04-22T23:46:05.373 INFO:tasks.workunit.client.1.smithi044.stdout:8/335: symlink db/d6c/l7d 0 2024-04-22T23:46:05.373 INFO:tasks.workunit.client.1.smithi044.stdout:8/336: chown db/d14/d24/f75 1562776 1 2024-04-22T23:46:05.378 INFO:tasks.workunit.client.1.smithi044.stdout:7/358: link dd/f3a dd/d1a/f7e 0 2024-04-22T23:46:05.382 INFO:tasks.workunit.client.1.smithi044.stdout:7/359: unlink dd/d1a/f65 0 2024-04-22T23:46:05.385 INFO:tasks.workunit.client.1.smithi044.stdout:8/337: link c3 db/d14/d72/c7e 0 2024-04-22T23:46:05.387 INFO:tasks.workunit.client.1.smithi044.stdout:8/338: truncate db/d14/d24/f58 986070 0 2024-04-22T23:46:05.393 INFO:tasks.workunit.client.1.smithi044.stdout:0/311: truncate d0/d2/d3d/f5c 556 0 2024-04-22T23:46:05.403 INFO:tasks.workunit.client.1.smithi044.stdout:5/312: dread d15/d16/d1d/d22/d2a/d3a/d43/f61 [0,4194304] 0 2024-04-22T23:46:05.405 INFO:tasks.workunit.client.1.smithi044.stdout:5/313: fsync d15/d16/d2c/f2e 0 2024-04-22T23:46:05.406 INFO:tasks.workunit.client.1.smithi044.stdout:5/314: truncate d15/d16/f1a 2017079 0 2024-04-22T23:46:05.409 INFO:tasks.workunit.client.1.smithi044.stdout:5/315: symlink d15/d16/d1d/l62 0 2024-04-22T23:46:05.416 INFO:tasks.workunit.client.1.smithi044.stdout:5/316: dwrite d15/d16/f36 [0,4194304] 0 2024-04-22T23:46:05.417 INFO:tasks.workunit.client.1.smithi044.stdout:5/317: chown d15/f1e 7256 1 2024-04-22T23:46:05.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/318: write f14 [2525483,13740] 0 2024-04-22T23:46:05.421 INFO:tasks.workunit.client.1.smithi044.stdout:5/319: truncate d15/d16/d2c/d38/f50 231778 0 2024-04-22T23:46:05.422 INFO:tasks.workunit.client.1.smithi044.stdout:5/320: chown d15/d16/l19 35304371 1 2024-04-22T23:46:05.439 INFO:tasks.workunit.client.1.smithi044.stdout:2/286: rmdir d8 39 2024-04-22T23:46:05.446 INFO:tasks.workunit.client.1.smithi044.stdout:8/339: mknod db/c7f 0 2024-04-22T23:46:05.447 INFO:tasks.workunit.client.1.smithi044.stdout:2/287: dwrite d8/d19/f30 [0,4194304] 0 2024-04-22T23:46:05.447 INFO:tasks.workunit.client.1.smithi044.stdout:8/340: chown db/d14/d24/f35 24889 1 2024-04-22T23:46:05.452 INFO:tasks.workunit.client.1.smithi044.stdout:8/341: symlink db/d6c/l80 0 2024-04-22T23:46:05.457 INFO:tasks.workunit.client.1.smithi044.stdout:6/259: rename d5/d3f/d4c/f46 to d5/d3f/d50/f5a 0 2024-04-22T23:46:05.458 INFO:tasks.workunit.client.1.smithi044.stdout:8/342: creat db/d14/d72/f81 x:0 0 0 2024-04-22T23:46:05.459 INFO:tasks.workunit.client.1.smithi044.stdout:2/288: getdents d8/d27/d5b 0 2024-04-22T23:46:05.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/343: chown db/d6c/c6e 1351 1 2024-04-22T23:46:05.460 INFO:tasks.workunit.client.1.smithi044.stdout:9/199: write d2/dc/f15 [3999886,91756] 0 2024-04-22T23:46:05.461 INFO:tasks.workunit.client.1.smithi044.stdout:6/260: truncate d5/d3f/fa 4563686 0 2024-04-22T23:46:05.463 INFO:tasks.workunit.client.1.smithi044.stdout:6/261: write d5/dd/f16 [2055857,77344] 0 2024-04-22T23:46:05.465 INFO:tasks.workunit.client.1.smithi044.stdout:8/344: unlink db/d14/d24/f49 0 2024-04-22T23:46:05.465 INFO:tasks.workunit.client.1.smithi044.stdout:8/345: dread - db/f4f zero size 2024-04-22T23:46:05.468 INFO:tasks.workunit.client.1.smithi044.stdout:4/347: symlink d5/d31/d44/d20/l70 0 2024-04-22T23:46:05.472 INFO:tasks.workunit.client.1.smithi044.stdout:6/262: creat d5/d3f/d4c/d40/f5b x:0 0 0 2024-04-22T23:46:05.475 INFO:tasks.workunit.client.1.smithi044.stdout:9/200: mkdir d2/dc/d1d/d34/d38/d4a 0 2024-04-22T23:46:05.476 INFO:tasks.workunit.client.1.smithi044.stdout:9/201: fdatasync d2/dc/f32 0 2024-04-22T23:46:05.476 INFO:tasks.workunit.client.1.smithi044.stdout:2/289: creat d8/d19/f5c x:0 0 0 2024-04-22T23:46:05.477 INFO:tasks.workunit.client.1.smithi044.stdout:2/290: write d8/fe [3090227,7580] 0 2024-04-22T23:46:05.478 INFO:tasks.workunit.client.1.smithi044.stdout:3/336: rename d9/d12/f1f to d9/db/f71 0 2024-04-22T23:46:05.482 INFO:tasks.workunit.client.1.smithi044.stdout:8/346: mknod db/c82 0 2024-04-22T23:46:05.490 INFO:tasks.workunit.client.1.smithi044.stdout:3/337: dread d9/db/d15/f21 [0,4194304] 0 2024-04-22T23:46:05.493 INFO:tasks.workunit.client.1.smithi044.stdout:8/347: unlink db/d14/f37 0 2024-04-22T23:46:05.493 INFO:tasks.workunit.client.1.smithi044.stdout:9/202: mkdir d2/d6/d42/d4b 0 2024-04-22T23:46:05.495 INFO:tasks.workunit.client.1.smithi044.stdout:2/291: mkdir d8/d19/d5d 0 2024-04-22T23:46:05.495 INFO:tasks.workunit.client.1.smithi044.stdout:2/292: dread - d8/d19/f4c zero size 2024-04-22T23:46:05.497 INFO:tasks.workunit.client.1.smithi044.stdout:2/293: write d8/d19/f32 [404474,40204] 0 2024-04-22T23:46:05.498 INFO:tasks.workunit.client.1.smithi044.stdout:2/294: write f6 [3777114,127591] 0 2024-04-22T23:46:05.500 INFO:tasks.workunit.client.1.smithi044.stdout:3/338: creat d9/db/d15/d2e/d48/f72 x:0 0 0 2024-04-22T23:46:05.502 INFO:tasks.workunit.client.1.smithi044.stdout:9/203: write d2/d5/f1e [4330448,50190] 0 2024-04-22T23:46:05.503 INFO:tasks.workunit.client.1.smithi044.stdout:4/348: getdents d5/d31/d24/d69 0 2024-04-22T23:46:05.507 INFO:tasks.workunit.client.1.smithi044.stdout:3/339: unlink d9/db/d15/d25/d5b/d52/f59 0 2024-04-22T23:46:05.509 INFO:tasks.workunit.client.1.smithi044.stdout:3/340: stat d9/db/d15/d54/d3c 0 2024-04-22T23:46:05.509 INFO:tasks.workunit.client.1.smithi044.stdout:8/348: link db/d52/f71 db/d52/f83 0 2024-04-22T23:46:05.510 INFO:tasks.workunit.client.1.smithi044.stdout:9/204: dwrite d2/dc/f27 [0,4194304] 0 2024-04-22T23:46:05.516 INFO:tasks.workunit.client.1.smithi044.stdout:9/205: dread d2/f3d [0,4194304] 0 2024-04-22T23:46:05.517 INFO:tasks.workunit.client.1.smithi044.stdout:5/321: write d15/d16/d1d/f26 [9372925,31027] 0 2024-04-22T23:46:05.522 INFO:tasks.workunit.client.1.smithi044.stdout:3/341: creat d9/db/d15/d2e/d48/f73 x:0 0 0 2024-04-22T23:46:05.525 INFO:tasks.workunit.client.1.smithi044.stdout:5/322: rmdir d15/d16 39 2024-04-22T23:46:05.527 INFO:tasks.workunit.client.1.smithi044.stdout:8/349: mkdir db/d14/d84 0 2024-04-22T23:46:05.529 INFO:tasks.workunit.client.1.smithi044.stdout:9/206: rename d2/f1c to d2/dc/d1d/d3f/f4c 0 2024-04-22T23:46:05.533 INFO:tasks.workunit.client.1.smithi044.stdout:3/342: mkdir d9/db/d22/d38/d74 0 2024-04-22T23:46:05.533 INFO:tasks.workunit.client.1.smithi044.stdout:2/295: getdents d8/d19/d53 0 2024-04-22T23:46:05.535 INFO:tasks.workunit.client.1.smithi044.stdout:2/296: write d8/f26 [748532,20314] 0 2024-04-22T23:46:05.535 INFO:tasks.workunit.client.1.smithi044.stdout:2/297: stat d8/d27/f4d 0 2024-04-22T23:46:05.538 INFO:tasks.workunit.client.1.smithi044.stdout:5/323: readlink d15/d16/d1d/l4d 0 2024-04-22T23:46:05.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/343: creat d9/db/d15/d25/d4c/f75 x:0 0 0 2024-04-22T23:46:05.552 INFO:tasks.workunit.client.1.smithi044.stdout:6/263: truncate d5/d30/f39 1232440 0 2024-04-22T23:46:05.553 INFO:tasks.workunit.client.1.smithi044.stdout:8/350: mknod db/d14/c85 0 2024-04-22T23:46:05.555 INFO:tasks.workunit.client.1.smithi044.stdout:8/351: fsync db/f44 0 2024-04-22T23:46:05.556 INFO:tasks.workunit.client.1.smithi044.stdout:2/298: creat d8/d19/d5d/f5e x:0 0 0 2024-04-22T23:46:05.556 INFO:tasks.workunit.client.1.smithi044.stdout:3/344: dwrite d9/db/d22/f68 [0,4194304] 0 2024-04-22T23:46:05.557 INFO:tasks.workunit.client.1.smithi044.stdout:5/324: symlink d15/l63 0 2024-04-22T23:46:05.557 INFO:tasks.workunit.client.1.smithi044.stdout:8/352: chown db/d14/l2d 56092955 1 2024-04-22T23:46:05.558 INFO:tasks.workunit.client.1.smithi044.stdout:8/353: read - db/d14/f41 zero size 2024-04-22T23:46:05.560 INFO:tasks.workunit.client.1.smithi044.stdout:8/354: dread - db/d14/d72/f81 zero size 2024-04-22T23:46:05.561 INFO:tasks.workunit.client.1.smithi044.stdout:6/264: mknod d5/d3f/d50/c5c 0 2024-04-22T23:46:05.561 INFO:tasks.workunit.client.1.smithi044.stdout:8/355: dread - db/d52/f79 zero size 2024-04-22T23:46:05.563 INFO:tasks.workunit.client.1.smithi044.stdout:2/299: rmdir d8/d27 39 2024-04-22T23:46:05.564 INFO:tasks.workunit.client.1.smithi044.stdout:8/356: write f6 [3874847,22265] 0 2024-04-22T23:46:05.566 INFO:tasks.workunit.client.1.smithi044.stdout:8/357: stat db/d14/l2d 0 2024-04-22T23:46:05.570 INFO:tasks.workunit.client.1.smithi044.stdout:3/345: dwrite d9/db/d15/d2e/f3f [0,4194304] 0 2024-04-22T23:46:05.570 INFO:tasks.workunit.client.1.smithi044.stdout:3/346: fdatasync d9/db/d15/d2e/d48/f73 0 2024-04-22T23:46:05.572 INFO:tasks.workunit.client.1.smithi044.stdout:3/347: truncate d9/db/d15/d54/f58 296164 0 2024-04-22T23:46:05.577 INFO:tasks.workunit.client.1.smithi044.stdout:5/325: write f5 [6304901,125915] 0 2024-04-22T23:46:05.579 INFO:tasks.workunit.client.1.smithi044.stdout:6/265: unlink d5/dd/c3c 0 2024-04-22T23:46:05.583 INFO:tasks.workunit.client.1.smithi044.stdout:2/300: creat d8/d19/d5d/f5f x:0 0 0 2024-04-22T23:46:05.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/301: dread - d8/f41 zero size 2024-04-22T23:46:05.585 INFO:tasks.workunit.client.1.smithi044.stdout:2/302: stat d8/f41 0 2024-04-22T23:46:05.587 INFO:tasks.workunit.client.1.smithi044.stdout:2/303: write d8/f11 [3374596,54008] 0 2024-04-22T23:46:05.587 INFO:tasks.workunit.client.1.smithi044.stdout:3/348: symlink d9/d12/l76 0 2024-04-22T23:46:05.588 INFO:tasks.workunit.client.1.smithi044.stdout:4/349: dwrite d5/db/f21 [0,4194304] 0 2024-04-22T23:46:05.590 INFO:tasks.workunit.client.1.smithi044.stdout:4/350: fdatasync d5/db/d34/f59 0 2024-04-22T23:46:05.591 INFO:tasks.workunit.client.1.smithi044.stdout:3/349: truncate d9/db/d15/d54/d3c/f6c 253308 0 2024-04-22T23:46:05.594 INFO:tasks.workunit.client.1.smithi044.stdout:6/266: creat d5/dd/d18/f5d x:0 0 0 2024-04-22T23:46:05.594 INFO:tasks.workunit.client.1.smithi044.stdout:4/351: fdatasync d5/d31/d44/d20/d22/f4c 0 2024-04-22T23:46:05.594 INFO:tasks.workunit.client.1.smithi044.stdout:3/350: dread d9/d14/f2b [0,4194304] 0 2024-04-22T23:46:05.618 INFO:tasks.workunit.client.1.smithi044.stdout:3/351: symlink d9/db/d33/l77 0 2024-04-22T23:46:05.620 INFO:tasks.workunit.client.1.smithi044.stdout:9/207: read - d2/dc/d1d/d3f/f45 zero size 2024-04-22T23:46:05.623 INFO:tasks.workunit.client.1.smithi044.stdout:8/358: rename db/d14/l32 to db/d14/l86 0 2024-04-22T23:46:05.625 INFO:tasks.workunit.client.1.smithi044.stdout:2/304: mkdir d8/d60 0 2024-04-22T23:46:05.704 INFO:tasks.workunit.client.1.smithi044.stdout:2/305: symlink d8/d19/d36/l61 0 2024-04-22T23:46:05.706 INFO:tasks.workunit.client.1.smithi044.stdout:4/352: creat d5/d31/d44/d20/f71 x:0 0 0 2024-04-22T23:46:05.708 INFO:tasks.workunit.client.1.smithi044.stdout:9/208: mkdir d2/d6/d4d 0 2024-04-22T23:46:05.709 INFO:tasks.workunit.client.1.smithi044.stdout:8/359: mknod db/d14/c87 0 2024-04-22T23:46:05.709 INFO:tasks.workunit.client.1.smithi044.stdout:8/360: stat db/d14/d24/f35 0 2024-04-22T23:46:05.710 INFO:tasks.workunit.client.1.smithi044.stdout:6/267: getdents d5/d3f 0 2024-04-22T23:46:05.711 INFO:tasks.workunit.client.1.smithi044.stdout:6/268: write f1 [2536791,74536] 0 2024-04-22T23:46:05.711 INFO:tasks.workunit.client.1.smithi044.stdout:2/306: mknod d8/d19/d5d/c62 0 2024-04-22T23:46:05.713 INFO:tasks.workunit.client.1.smithi044.stdout:4/353: creat d5/d31/d24/f72 x:0 0 0 2024-04-22T23:46:05.715 INFO:tasks.workunit.client.1.smithi044.stdout:9/209: symlink d2/d6/d42/l4e 0 2024-04-22T23:46:05.715 INFO:tasks.workunit.client.1.smithi044.stdout:4/354: dread - d5/d31/d24/d38/f63 zero size 2024-04-22T23:46:05.715 INFO:tasks.workunit.client.1.smithi044.stdout:9/210: chown d2 249920 1 2024-04-22T23:46:05.717 INFO:tasks.workunit.client.1.smithi044.stdout:4/355: fsync d5/d31/d24/d69/f6c 0 2024-04-22T23:46:05.719 INFO:tasks.workunit.client.1.smithi044.stdout:6/269: unlink d5/d3f/d4c/d23/l55 0 2024-04-22T23:46:05.720 INFO:tasks.workunit.client.1.smithi044.stdout:6/270: write d5/dd/f1b [3393454,64571] 0 2024-04-22T23:46:05.721 INFO:tasks.workunit.client.1.smithi044.stdout:5/326: dwrite d15/d16/f39 [0,4194304] 0 2024-04-22T23:46:05.722 INFO:tasks.workunit.client.1.smithi044.stdout:4/356: mkdir d5/d42/d73 0 2024-04-22T23:46:05.725 INFO:tasks.workunit.client.1.smithi044.stdout:4/357: write d5/d31/f1b [2253394,107768] 0 2024-04-22T23:46:05.728 INFO:tasks.workunit.client.1.smithi044.stdout:8/361: rename db/f21 to db/d14/f88 0 2024-04-22T23:46:05.729 INFO:tasks.workunit.client.1.smithi044.stdout:9/211: mknod d2/dc/d1d/d34/d38/d4a/c4f 0 2024-04-22T23:46:05.730 INFO:tasks.workunit.client.1.smithi044.stdout:8/362: write f6 [7051893,97958] 0 2024-04-22T23:46:05.731 INFO:tasks.workunit.client.1.smithi044.stdout:6/271: rmdir d5/d3f/d4c/d40/d53/d56 39 2024-04-22T23:46:05.731 INFO:tasks.workunit.client.1.smithi044.stdout:5/327: dwrite d15/d16/d1d/f51 [0,4194304] 0 2024-04-22T23:46:05.740 INFO:tasks.workunit.client.1.smithi044.stdout:9/212: dwrite d2/dc/d1d/d34/f49 [0,4194304] 0 2024-04-22T23:46:05.743 INFO:tasks.workunit.client.1.smithi044.stdout:5/328: mkdir d15/d5d/d64 0 2024-04-22T23:46:05.745 INFO:tasks.workunit.client.1.smithi044.stdout:4/358: getdents d5/d31/d24/d69 0 2024-04-22T23:46:05.753 INFO:tasks.workunit.client.1.smithi044.stdout:9/213: dwrite d2/dc/f3a [0,4194304] 0 2024-04-22T23:46:05.753 INFO:tasks.workunit.client.1.smithi044.stdout:8/363: mknod db/d68/d74/c89 0 2024-04-22T23:46:05.756 INFO:tasks.workunit.client.1.smithi044.stdout:8/364: readlink db/d52/l57 0 2024-04-22T23:46:05.759 INFO:tasks.workunit.client.1.smithi044.stdout:8/365: chown db/f44 109742 1 2024-04-22T23:46:05.760 INFO:tasks.workunit.client.1.smithi044.stdout:5/329: dwrite d15/d16/d1d/f42 [0,4194304] 0 2024-04-22T23:46:05.763 INFO:tasks.workunit.client.1.smithi044.stdout:4/359: dwrite d5/d31/d44/d20/d22/f4c [0,4194304] 0 2024-04-22T23:46:05.773 INFO:tasks.workunit.client.1.smithi044.stdout:5/330: mkdir d15/d3c/d65 0 2024-04-22T23:46:05.775 INFO:tasks.workunit.client.1.smithi044.stdout:5/331: dread - d15/d16/d1d/d22/d2a/d3a/d43/f58 zero size 2024-04-22T23:46:05.777 INFO:tasks.workunit.client.1.smithi044.stdout:4/360: symlink d5/d42/l74 0 2024-04-22T23:46:05.778 INFO:tasks.workunit.client.1.smithi044.stdout:4/361: fdatasync d5/d31/f11 0 2024-04-22T23:46:05.780 INFO:tasks.workunit.client.1.smithi044.stdout:8/366: unlink c3 0 2024-04-22T23:46:05.787 INFO:tasks.workunit.client.1.smithi044.stdout:5/332: unlink d15/d16/f36 0 2024-04-22T23:46:05.788 INFO:tasks.workunit.client.1.smithi044.stdout:5/333: stat d15/d16/d1d/d22/d4f/f59 0 2024-04-22T23:46:05.793 INFO:tasks.workunit.client.1.smithi044.stdout:4/362: mkdir d5/d31/d24/d38/d6a/d75 0 2024-04-22T23:46:05.794 INFO:tasks.workunit.client.1.smithi044.stdout:5/334: unlink d15/d16/c2f 0 2024-04-22T23:46:05.797 INFO:tasks.workunit.client.1.smithi044.stdout:8/367: mknod db/c8a 0 2024-04-22T23:46:05.802 INFO:tasks.workunit.client.1.smithi044.stdout:8/368: fsync f5 0 2024-04-22T23:46:05.803 INFO:tasks.workunit.client.1.smithi044.stdout:8/369: chown db/d52/l70 28 1 2024-04-22T23:46:05.808 INFO:tasks.workunit.client.1.smithi044.stdout:4/363: rename d5/d31/d44/d20/d22/f65 to d5/d31/f76 0 2024-04-22T23:46:05.810 INFO:tasks.workunit.client.1.smithi044.stdout:8/370: rmdir db/d14/d72 39 2024-04-22T23:46:05.810 INFO:tasks.workunit.client.1.smithi044.stdout:4/364: write d5/d31/d44/f30 [2499941,86106] 0 2024-04-22T23:46:05.813 INFO:tasks.workunit.client.1.smithi044.stdout:5/335: rename d15/d16/d1d/d22/d2a/f60 to d15/d16/d2c/f66 0 2024-04-22T23:46:05.814 INFO:tasks.workunit.client.1.smithi044.stdout:8/371: mkdir db/d68/d8b 0 2024-04-22T23:46:05.815 INFO:tasks.workunit.client.1.smithi044.stdout:8/372: write db/f50 [1482140,74904] 0 2024-04-22T23:46:05.816 INFO:tasks.workunit.client.1.smithi044.stdout:8/373: chown db/l4d 12 1 2024-04-22T23:46:05.816 INFO:tasks.workunit.client.1.smithi044.stdout:5/336: symlink d15/d16/d1d/l67 0 2024-04-22T23:46:05.817 INFO:tasks.workunit.client.1.smithi044.stdout:8/374: write db/d14/d24/f26 [5018050,82732] 0 2024-04-22T23:46:05.818 INFO:tasks.workunit.client.1.smithi044.stdout:5/337: chown d15/d16/d2c/f5a 1613558326 1 2024-04-22T23:46:05.819 INFO:tasks.workunit.client.1.smithi044.stdout:5/338: chown d15/d16/f1a 7821 1 2024-04-22T23:46:05.821 INFO:tasks.workunit.client.1.smithi044.stdout:8/375: mknod db/d52/c8c 0 2024-04-22T23:46:05.822 INFO:tasks.workunit.client.1.smithi044.stdout:8/376: fdatasync db/d14/f63 0 2024-04-22T23:46:05.833 INFO:tasks.workunit.client.1.smithi044.stdout:9/214: truncate d2/d5/f1f 2005296 0 2024-04-22T23:46:05.838 INFO:tasks.workunit.client.1.smithi044.stdout:9/215: creat d2/d5/f50 x:0 0 0 2024-04-22T23:46:05.841 INFO:tasks.workunit.client.1.smithi044.stdout:9/216: mkdir d2/dc/d51 0 2024-04-22T23:46:05.845 INFO:tasks.workunit.client.1.smithi044.stdout:9/217: unlink d2/d5/f1e 0 2024-04-22T23:46:05.851 INFO:tasks.workunit.client.1.smithi044.stdout:9/218: mknod d2/d6/c52 0 2024-04-22T23:46:05.855 INFO:tasks.workunit.client.1.smithi044.stdout:9/219: creat d2/d6/f53 x:0 0 0 2024-04-22T23:46:05.859 INFO:tasks.workunit.client.1.smithi044.stdout:9/220: symlink d2/d6/d42/l54 0 2024-04-22T23:46:05.860 INFO:tasks.workunit.client.1.smithi044.stdout:9/221: write d2/d5/f3b [112653,20651] 0 2024-04-22T23:46:05.860 INFO:tasks.workunit.client.1.smithi044.stdout:9/222: chown d2/dc/f15 61821731 1 2024-04-22T23:46:05.862 INFO:tasks.workunit.client.1.smithi044.stdout:9/223: creat d2/d5/f55 x:0 0 0 2024-04-22T23:46:05.863 INFO:tasks.workunit.client.1.smithi044.stdout:4/365: truncate d5/d31/d44/d20/f23 950376 0 2024-04-22T23:46:05.868 INFO:tasks.workunit.client.1.smithi044.stdout:6/272: write d5/d30/f39 [1574782,53372] 0 2024-04-22T23:46:05.870 INFO:tasks.workunit.client.1.smithi044.stdout:6/273: fdatasync d5/d3f/d4c/f57 0 2024-04-22T23:46:05.872 INFO:tasks.workunit.client.1.smithi044.stdout:6/274: dread - d5/d3f/f54 zero size 2024-04-22T23:46:05.873 INFO:tasks.workunit.client.1.smithi044.stdout:4/366: creat d5/d42/d73/f77 x:0 0 0 2024-04-22T23:46:05.873 INFO:tasks.workunit.client.1.smithi044.stdout:5/339: dwrite d15/f18 [0,4194304] 0 2024-04-22T23:46:05.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/275: dread - d5/d3f/f54 zero size 2024-04-22T23:46:05.875 INFO:tasks.workunit.client.1.smithi044.stdout:6/276: fsync d5/dd/d18/f20 0 2024-04-22T23:46:05.876 INFO:tasks.workunit.client.1.smithi044.stdout:6/277: stat d5/d3f/d4c/f57 0 2024-04-22T23:46:05.876 INFO:tasks.workunit.client.1.smithi044.stdout:6/278: chown d5/d30 18302612 1 2024-04-22T23:46:05.876 INFO:tasks.workunit.client.1.smithi044.stdout:9/224: link d2/d6/l35 d2/dc/d1d/d34/d38/l56 0 2024-04-22T23:46:05.878 INFO:tasks.workunit.client.1.smithi044.stdout:6/279: write d5/d3f/d4c/d40/f41 [255029,119141] 0 2024-04-22T23:46:05.886 INFO:tasks.workunit.client.1.smithi044.stdout:4/367: unlink d5/db/f2e 0 2024-04-22T23:46:05.888 INFO:tasks.workunit.client.1.smithi044.stdout:9/225: dwrite d2/d6/f43 [0,4194304] 0 2024-04-22T23:46:05.889 INFO:tasks.workunit.client.1.smithi044.stdout:6/280: dwrite d5/dd/f1b [4194304,4194304] 0 2024-04-22T23:46:05.893 INFO:tasks.workunit.client.1.smithi044.stdout:5/340: mkdir d15/d16/d1d/d22/d4f/d68 0 2024-04-22T23:46:05.896 INFO:tasks.workunit.client.1.smithi044.stdout:5/341: write d15/d16/d1d/d22/d4f/f52 [618064,119427] 0 2024-04-22T23:46:05.899 INFO:tasks.workunit.client.1.smithi044.stdout:4/368: dread d5/d31/d24/f3d [0,4194304] 0 2024-04-22T23:46:05.906 INFO:tasks.workunit.client.1.smithi044.stdout:6/281: symlink d5/d30/l5e 0 2024-04-22T23:46:05.907 INFO:tasks.workunit.client.1.smithi044.stdout:5/342: fsync d15/d16/d2c/f54 0 2024-04-22T23:46:05.908 INFO:tasks.workunit.client.1.smithi044.stdout:5/343: write f5 [4032000,21036] 0 2024-04-22T23:46:05.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/369: unlink d5/d42/d73/f77 0 2024-04-22T23:46:05.912 INFO:tasks.workunit.client.1.smithi044.stdout:9/226: symlink d2/dc/d51/l57 0 2024-04-22T23:46:05.916 INFO:tasks.workunit.client.1.smithi044.stdout:5/344: dwrite d15/d16/d1d/f51 [0,4194304] 0 2024-04-22T23:46:05.916 INFO:tasks.workunit.client.1.smithi044.stdout:6/282: write d5/f21 [1049969,128487] 0 2024-04-22T23:46:05.918 INFO:tasks.workunit.client.1.smithi044.stdout:5/345: read d15/d16/d1d/f51 [4187028,58592] 0 2024-04-22T23:46:05.926 INFO:tasks.workunit.client.1.smithi044.stdout:5/346: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:05.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/370: creat d5/db/d34/f78 x:0 0 0 2024-04-22T23:46:05.929 INFO:tasks.workunit.client.1.smithi044.stdout:4/371: dread - d5/d31/d44/d20/d22/f6f zero size 2024-04-22T23:46:05.931 INFO:tasks.workunit.client.1.smithi044.stdout:4/372: chown d5/l6b 2067674543 1 2024-04-22T23:46:05.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/283: mknod d5/dd/d28/c5f 0 2024-04-22T23:46:05.937 INFO:tasks.workunit.client.1.smithi044.stdout:4/373: dread d5/d42/f61 [0,4194304] 0 2024-04-22T23:46:05.941 INFO:tasks.workunit.client.1.smithi044.stdout:9/227: dwrite d2/f19 [0,4194304] 0 2024-04-22T23:46:05.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:05 smithi031 ceph-mon[26721]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:05.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:05 smithi031 ceph-mon[26721]: pgmap v6: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 22 MiB/s rd, 108 MiB/s wr, 136 op/s 2024-04-22T23:46:05.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:05 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:05.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:05 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:05.950 INFO:tasks.workunit.client.1.smithi044.stdout:5/347: write d15/d16/d1d/f3f [2017052,108801] 0 2024-04-22T23:46:05.955 INFO:tasks.workunit.client.1.smithi044.stdout:5/348: read d15/d16/d1d/d22/f2d [3257250,114532] 0 2024-04-22T23:46:05.956 INFO:tasks.workunit.client.1.smithi044.stdout:9/228: fdatasync d2/f19 0 2024-04-22T23:46:05.956 INFO:tasks.workunit.client.1.smithi044.stdout:4/374: unlink d5/d31/d44/f30 0 2024-04-22T23:46:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:05 smithi044 ceph-mon[34463]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:05 smithi044 ceph-mon[34463]: pgmap v6: 65 pgs: 65 active+clean; 2.5 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 22 MiB/s rd, 108 MiB/s wr, 136 op/s 2024-04-22T23:46:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:05 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:05 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:05.961 INFO:tasks.workunit.client.1.smithi044.stdout:5/349: creat d15/d16/d1d/f69 x:0 0 0 2024-04-22T23:46:05.962 INFO:tasks.workunit.client.1.smithi044.stdout:9/229: symlink d2/dc/d1d/l58 0 2024-04-22T23:46:05.962 INFO:tasks.workunit.client.1.smithi044.stdout:5/350: readlink d15/d16/d1d/d22/l34 0 2024-04-22T23:46:05.966 INFO:tasks.workunit.client.1.smithi044.stdout:9/230: dread d2/dc/f27 [0,4194304] 0 2024-04-22T23:46:05.967 INFO:tasks.workunit.client.1.smithi044.stdout:6/284: getdents d5/d3f/d4c/d40 0 2024-04-22T23:46:05.970 INFO:tasks.workunit.client.1.smithi044.stdout:4/375: rename d5/db/d3c/f53 to d5/db/f79 0 2024-04-22T23:46:05.974 INFO:tasks.workunit.client.1.smithi044.stdout:9/231: read d2/dc/f18 [5174183,62805] 0 2024-04-22T23:46:05.975 INFO:tasks.workunit.client.1.smithi044.stdout:6/285: write d5/dd/f3e [108585,105604] 0 2024-04-22T23:46:05.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/376: dwrite d5/db/d34/f51 [0,4194304] 0 2024-04-22T23:46:05.977 INFO:tasks.workunit.client.1.smithi044.stdout:1/354: sync 2024-04-22T23:46:05.978 INFO:tasks.workunit.client.1.smithi044.stdout:7/360: sync 2024-04-22T23:46:05.978 INFO:tasks.workunit.client.1.smithi044.stdout:0/312: sync 2024-04-22T23:46:05.980 INFO:tasks.workunit.client.1.smithi044.stdout:9/232: dwrite d2/f19 [0,4194304] 0 2024-04-22T23:46:05.983 INFO:tasks.workunit.client.1.smithi044.stdout:6/286: creat d5/d3f/d50/f60 x:0 0 0 2024-04-22T23:46:05.984 INFO:tasks.workunit.client.1.smithi044.stdout:4/377: write d5/d31/f54 [193662,71080] 0 2024-04-22T23:46:05.988 INFO:tasks.workunit.client.1.smithi044.stdout:9/233: write d2/d5/f30 [1706356,125308] 0 2024-04-22T23:46:05.992 INFO:tasks.workunit.client.1.smithi044.stdout:1/355: dwrite d1/d8/d24/d46/f68 [0,4194304] 0 2024-04-22T23:46:05.998 INFO:tasks.workunit.client.1.smithi044.stdout:4/378: write d5/d31/f17 [4295127,71483] 0 2024-04-22T23:46:05.999 INFO:tasks.workunit.client.1.smithi044.stdout:4/379: truncate d5/d31/d24/d38/f40 397466 0 2024-04-22T23:46:06.009 INFO:tasks.workunit.client.1.smithi044.stdout:7/361: creat dd/d55/d5e/d60/d7b/f7f x:0 0 0 2024-04-22T23:46:06.013 INFO:tasks.workunit.client.1.smithi044.stdout:1/356: creat d1/d8/d1c/d23/d3e/f7e x:0 0 0 2024-04-22T23:46:06.013 INFO:tasks.workunit.client.1.smithi044.stdout:7/362: dread f5 [0,4194304] 0 2024-04-22T23:46:06.014 INFO:tasks.workunit.client.1.smithi044.stdout:4/380: write d5/db/f79 [1310224,86638] 0 2024-04-22T23:46:06.015 INFO:tasks.workunit.client.1.smithi044.stdout:7/363: dread - dd/d55/d5e/d60/d68/f73 zero size 2024-04-22T23:46:06.021 INFO:tasks.workunit.client.1.smithi044.stdout:1/357: dwrite d1/f7 [0,4194304] 0 2024-04-22T23:46:06.024 INFO:tasks.workunit.client.1.smithi044.stdout:0/313: getdents d0/d4c/d59 0 2024-04-22T23:46:06.027 INFO:tasks.workunit.client.1.smithi044.stdout:4/381: dwrite d5/d31/d44/d20/d22/f62 [0,4194304] 0 2024-04-22T23:46:06.031 INFO:tasks.workunit.client.1.smithi044.stdout:6/287: truncate d5/d3f/d4c/d40/f41 367165 0 2024-04-22T23:46:06.034 INFO:tasks.workunit.client.1.smithi044.stdout:1/358: symlink d1/d8/d1c/d23/l7f 0 2024-04-22T23:46:06.035 INFO:tasks.workunit.client.1.smithi044.stdout:7/364: creat dd/d79/f80 x:0 0 0 2024-04-22T23:46:06.036 INFO:tasks.workunit.client.1.smithi044.stdout:7/365: chown dd/d27/f48 1856 1 2024-04-22T23:46:06.039 INFO:tasks.workunit.client.1.smithi044.stdout:4/382: symlink d5/d31/d24/d38/d6a/l7a 0 2024-04-22T23:46:06.042 INFO:tasks.workunit.client.1.smithi044.stdout:9/234: getdents d2/dc/d1d/d34 0 2024-04-22T23:46:06.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/359: rmdir d1/d8/d20/d45/d75 39 2024-04-22T23:46:06.047 INFO:tasks.workunit.client.1.smithi044.stdout:1/360: write d1/f67 [627352,93443] 0 2024-04-22T23:46:06.054 INFO:tasks.workunit.client.1.smithi044.stdout:0/314: rename d0/d2e/d2f/f3b to d0/d4c/d59/f76 0 2024-04-22T23:46:06.055 INFO:tasks.workunit.client.1.smithi044.stdout:4/383: symlink d5/d31/d24/d38/d6a/l7b 0 2024-04-22T23:46:06.056 INFO:tasks.workunit.client.1.smithi044.stdout:4/384: chown d5/db 48127954 1 2024-04-22T23:46:06.056 INFO:tasks.workunit.client.1.smithi044.stdout:4/385: stat d5/l6e 0 2024-04-22T23:46:06.057 INFO:tasks.workunit.client.1.smithi044.stdout:4/386: chown d5/d31/d24/d38 36147 1 2024-04-22T23:46:06.060 INFO:tasks.workunit.client.1.smithi044.stdout:6/288: symlink d5/l61 0 2024-04-22T23:46:06.063 INFO:tasks.workunit.client.1.smithi044.stdout:0/315: dread d0/d2/d3d/f26 [0,4194304] 0 2024-04-22T23:46:06.070 INFO:tasks.workunit.client.1.smithi044.stdout:4/387: dwrite d5/db/f79 [0,4194304] 0 2024-04-22T23:46:06.071 INFO:tasks.workunit.client.1.smithi044.stdout:9/235: dwrite d2/dc/f27 [0,4194304] 0 2024-04-22T23:46:06.081 INFO:tasks.workunit.client.1.smithi044.stdout:7/366: dwrite f8 [0,4194304] 0 2024-04-22T23:46:06.081 INFO:tasks.workunit.client.1.smithi044.stdout:6/289: truncate d5/d3f/f15 1051241 0 2024-04-22T23:46:06.081 INFO:tasks.workunit.client.1.smithi044.stdout:1/361: dread d1/d8/d20/d45/f29 [0,4194304] 0 2024-04-22T23:46:06.084 INFO:tasks.workunit.client.1.smithi044.stdout:7/367: fsync dd/d27/f48 0 2024-04-22T23:46:06.084 INFO:tasks.workunit.client.1.smithi044.stdout:6/290: chown d5 1 1 2024-04-22T23:46:06.085 INFO:tasks.workunit.client.1.smithi044.stdout:7/368: write dd/f59 [462754,37051] 0 2024-04-22T23:46:06.089 INFO:tasks.workunit.client.1.smithi044.stdout:9/236: mknod d2/d9/c59 0 2024-04-22T23:46:06.090 INFO:tasks.workunit.client.1.smithi044.stdout:6/291: rmdir d5/d3f/d4c/d40 39 2024-04-22T23:46:06.092 INFO:tasks.workunit.client.1.smithi044.stdout:7/369: mkdir dd/d55/d5e/d81 0 2024-04-22T23:46:06.093 INFO:tasks.workunit.client.1.smithi044.stdout:4/388: symlink d5/d31/d44/d20/l7c 0 2024-04-22T23:46:06.094 INFO:tasks.workunit.client.1.smithi044.stdout:0/316: dread d0/f16 [0,4194304] 0 2024-04-22T23:46:06.095 INFO:tasks.workunit.client.1.smithi044.stdout:4/389: write d5/d31/d24/d38/f63 [969304,80801] 0 2024-04-22T23:46:06.096 INFO:tasks.workunit.client.1.smithi044.stdout:9/237: creat d2/d9/f5a x:0 0 0 2024-04-22T23:46:06.101 INFO:tasks.workunit.client.1.smithi044.stdout:7/370: symlink dd/d55/d5e/d60/d68/l82 0 2024-04-22T23:46:06.102 INFO:tasks.workunit.client.1.smithi044.stdout:4/390: dread d5/d31/d44/d20/d22/f4c [0,4194304] 0 2024-04-22T23:46:06.110 INFO:tasks.workunit.client.1.smithi044.stdout:6/292: rename d5/d3f/d4c/d40/d53 to d5/d3f/d4c/d40/d53/d56/d62 22 2024-04-22T23:46:06.111 INFO:tasks.workunit.client.1.smithi044.stdout:6/293: write d5/d3f/f54 [412128,3450] 0 2024-04-22T23:46:06.111 INFO:tasks.workunit.client.1.smithi044.stdout:1/362: getdents d1/d8/d20 0 2024-04-22T23:46:06.113 INFO:tasks.workunit.client.1.smithi044.stdout:1/363: dread - d1/d8/d1c/d23/f7c zero size 2024-04-22T23:46:06.114 INFO:tasks.workunit.client.1.smithi044.stdout:1/364: truncate d1/d8/d34/f40 1099964 0 2024-04-22T23:46:06.114 INFO:tasks.workunit.client.1.smithi044.stdout:4/391: truncate d5/d31/f32 16792 0 2024-04-22T23:46:06.116 INFO:tasks.workunit.client.1.smithi044.stdout:4/392: dread - d5/d31/d44/d20/f71 zero size 2024-04-22T23:46:06.118 INFO:tasks.workunit.client.1.smithi044.stdout:9/238: write d2/d5/f7 [2969082,43450] 0 2024-04-22T23:46:06.121 INFO:tasks.workunit.client.1.smithi044.stdout:7/371: mknod dd/d55/d5e/d81/c83 0 2024-04-22T23:46:06.122 INFO:tasks.workunit.client.1.smithi044.stdout:7/372: write dd/d1a/d3c/f74 [449195,75721] 0 2024-04-22T23:46:06.126 INFO:tasks.workunit.client.1.smithi044.stdout:1/365: rename d1/d8/d24 to d1/d8/d1c/d7b/d80 0 2024-04-22T23:46:06.128 INFO:tasks.workunit.client.1.smithi044.stdout:1/366: chown d1/d8/d34/l35 737224 1 2024-04-22T23:46:06.129 INFO:tasks.workunit.client.1.smithi044.stdout:7/373: dwrite dd/f20 [0,4194304] 0 2024-04-22T23:46:06.134 INFO:tasks.workunit.client.1.smithi044.stdout:6/294: unlink d5/d30/f31 0 2024-04-22T23:46:06.141 INFO:tasks.workunit.client.1.smithi044.stdout:0/317: getdents d0/d2/dd/d3e 0 2024-04-22T23:46:06.147 INFO:tasks.workunit.client.1.smithi044.stdout:4/393: unlink d5/db/c26 0 2024-04-22T23:46:06.148 INFO:tasks.workunit.client.1.smithi044.stdout:1/367: mkdir d1/d8/d1c/d7b/d80/d4b/d81 0 2024-04-22T23:46:06.155 INFO:tasks.workunit.client.1.smithi044.stdout:6/295: creat d5/d3f/f63 x:0 0 0 2024-04-22T23:46:06.157 INFO:tasks.workunit.client.1.smithi044.stdout:0/318: mkdir d0/d2e/d2f/d77 0 2024-04-22T23:46:06.158 INFO:tasks.workunit.client.1.smithi044.stdout:0/319: truncate d0/d2/dd/d3e/d40/d43/f66 38000 0 2024-04-22T23:46:06.163 INFO:tasks.workunit.client.1.smithi044.stdout:4/394: creat d5/d31/d44/f7d x:0 0 0 2024-04-22T23:46:06.164 INFO:tasks.workunit.client.1.smithi044.stdout:4/395: fdatasync d5/d31/f54 0 2024-04-22T23:46:06.166 INFO:tasks.workunit.client.1.smithi044.stdout:1/368: rename d1/d8/c2e to d1/d8/d1c/d23/c82 0 2024-04-22T23:46:06.168 INFO:tasks.workunit.client.1.smithi044.stdout:7/374: truncate dd/d1a/f7e 3690694 0 2024-04-22T23:46:06.173 INFO:tasks.workunit.client.1.smithi044.stdout:9/239: truncate d2/d5/f30 3919223 0 2024-04-22T23:46:06.173 INFO:tasks.workunit.client.1.smithi044.stdout:9/240: stat d2/d6/f33 0 2024-04-22T23:46:06.177 INFO:tasks.workunit.client.1.smithi044.stdout:0/320: dwrite d0/f16 [0,4194304] 0 2024-04-22T23:46:06.179 INFO:tasks.workunit.client.1.smithi044.stdout:1/369: rename d1/f67 to d1/d14/f83 0 2024-04-22T23:46:06.180 INFO:tasks.workunit.client.1.smithi044.stdout:1/370: chown d1/d8/d1c/l57 3678252 1 2024-04-22T23:46:06.182 INFO:tasks.workunit.client.1.smithi044.stdout:2/307: sync 2024-04-22T23:46:06.182 INFO:tasks.workunit.client.1.smithi044.stdout:3/352: sync 2024-04-22T23:46:06.184 INFO:tasks.workunit.client.1.smithi044.stdout:2/308: read - d8/d19/f51 zero size 2024-04-22T23:46:06.184 INFO:tasks.workunit.client.1.smithi044.stdout:0/321: dread d0/d2/dd/f2a [0,4194304] 0 2024-04-22T23:46:06.193 INFO:tasks.workunit.client.1.smithi044.stdout:3/353: dwrite d9/db/d15/f63 [0,4194304] 0 2024-04-22T23:46:06.196 INFO:tasks.workunit.client.1.smithi044.stdout:6/296: dwrite d5/d3f/d50/f5a [0,4194304] 0 2024-04-22T23:46:06.196 INFO:tasks.workunit.client.1.smithi044.stdout:4/396: fsync d5/d31/f32 0 2024-04-22T23:46:06.203 INFO:tasks.workunit.client.1.smithi044.stdout:1/371: unlink d1/d8/d1c/c21 0 2024-04-22T23:46:06.205 INFO:tasks.workunit.client.1.smithi044.stdout:1/372: read d1/d8/d20/d45/f29 [1992766,61601] 0 2024-04-22T23:46:06.206 INFO:tasks.workunit.client.1.smithi044.stdout:1/373: write d1/d8/f18 [4277932,45521] 0 2024-04-22T23:46:06.207 INFO:tasks.workunit.client.1.smithi044.stdout:1/374: fdatasync d1/d8/d1c/f5b 0 2024-04-22T23:46:06.208 INFO:tasks.workunit.client.1.smithi044.stdout:1/375: chown d1/d8/f18 97 1 2024-04-22T23:46:06.210 INFO:tasks.workunit.client.1.smithi044.stdout:0/322: rmdir d0/d2e 39 2024-04-22T23:46:06.211 INFO:tasks.workunit.client.1.smithi044.stdout:2/309: mkdir d8/d19/d36/d46/d63 0 2024-04-22T23:46:06.212 INFO:tasks.workunit.client.1.smithi044.stdout:2/310: write d8/d19/f4c [748823,54607] 0 2024-04-22T23:46:06.215 INFO:tasks.workunit.client.1.smithi044.stdout:9/241: unlink d2/d9/cb 0 2024-04-22T23:46:06.218 INFO:tasks.workunit.client.1.smithi044.stdout:4/397: mkdir d5/d31/d44/d7e 0 2024-04-22T23:46:06.219 INFO:tasks.workunit.client.1.smithi044.stdout:6/297: rmdir d5/d3f/d4c/d23/d3d 39 2024-04-22T23:46:06.225 INFO:tasks.workunit.client.1.smithi044.stdout:3/354: dread d9/d12/f2d [0,4194304] 0 2024-04-22T23:46:06.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/375: truncate dd/d1f/f34 2993698 0 2024-04-22T23:46:06.228 INFO:tasks.workunit.client.1.smithi044.stdout:6/298: dwrite d5/d3f/f54 [0,4194304] 0 2024-04-22T23:46:06.231 INFO:tasks.workunit.client.1.smithi044.stdout:2/311: mknod d8/d19/d36/d46/d54/c64 0 2024-04-22T23:46:06.233 INFO:tasks.workunit.client.1.smithi044.stdout:9/242: creat d2/d5/f5b x:0 0 0 2024-04-22T23:46:06.242 INFO:tasks.workunit.client.1.smithi044.stdout:1/376: rename d1/d8/d1c/f79 to d1/d8/d20/d45/d75/f84 0 2024-04-22T23:46:06.243 INFO:tasks.workunit.client.1.smithi044.stdout:0/323: creat d0/d2e/d2f/d61/f78 x:0 0 0 2024-04-22T23:46:06.247 INFO:tasks.workunit.client.1.smithi044.stdout:7/376: chown dd/d1a/f7e 3199458 1 2024-04-22T23:46:06.248 INFO:tasks.workunit.client.1.smithi044.stdout:1/377: dwrite d1/d8/f1a [0,4194304] 0 2024-04-22T23:46:06.249 INFO:tasks.workunit.client.1.smithi044.stdout:7/377: fsync dd/d27/f5b 0 2024-04-22T23:46:06.250 INFO:tasks.workunit.client.1.smithi044.stdout:1/378: write d1/d8/f19 [1153311,15719] 0 2024-04-22T23:46:06.252 INFO:tasks.workunit.client.1.smithi044.stdout:6/299: write d5/dd/f1b [1142292,110690] 0 2024-04-22T23:46:06.255 INFO:tasks.workunit.client.1.smithi044.stdout:2/312: rmdir d8/d19/d36 39 2024-04-22T23:46:06.257 INFO:tasks.workunit.client.1.smithi044.stdout:4/398: creat d5/d31/d24/d38/d6a/d75/f7f x:0 0 0 2024-04-22T23:46:06.258 INFO:tasks.workunit.client.1.smithi044.stdout:7/378: dread dd/d27/f5b [0,4194304] 0 2024-04-22T23:46:06.275 INFO:tasks.workunit.client.1.smithi044.stdout:1/379: creat d1/d8/d20/d47/f85 x:0 0 0 2024-04-22T23:46:06.276 INFO:tasks.workunit.client.1.smithi044.stdout:1/380: write d1/d8/d1c/d23/f5f [508778,25475] 0 2024-04-22T23:46:06.276 INFO:tasks.workunit.client.1.smithi044.stdout:1/381: stat d1/c62 0 2024-04-22T23:46:06.278 INFO:tasks.workunit.client.1.smithi044.stdout:1/382: read d1/d8/d20/d47/d4a/f54 [359032,32487] 0 2024-04-22T23:46:06.282 INFO:tasks.workunit.client.1.smithi044.stdout:3/355: truncate d9/db/d15/d2e/f39 656099 0 2024-04-22T23:46:06.282 INFO:tasks.workunit.client.1.smithi044.stdout:3/356: stat d9/db/d15/d25/d5b 0 2024-04-22T23:46:06.285 INFO:tasks.workunit.client.1.smithi044.stdout:4/399: rename d5/d31/d44/d20/f5f to d5/d31/d44/d20/d22/f80 0 2024-04-22T23:46:06.286 INFO:tasks.workunit.client.1.smithi044.stdout:0/324: getdents d0/d2e/d33/d5d 0 2024-04-22T23:46:06.288 INFO:tasks.workunit.client.1.smithi044.stdout:6/300: mkdir d5/d3f/d4c/d23/d3d/d64 0 2024-04-22T23:46:06.289 INFO:tasks.workunit.client.1.smithi044.stdout:6/301: fsync d5/dd/d28/f2e 0 2024-04-22T23:46:06.291 INFO:tasks.workunit.client.1.smithi044.stdout:6/302: write d5/dd/d18/f20 [1539152,65598] 0 2024-04-22T23:46:06.293 INFO:tasks.workunit.client.1.smithi044.stdout:3/357: creat d9/d12/f78 x:0 0 0 2024-04-22T23:46:06.293 INFO:tasks.workunit.client.1.smithi044.stdout:1/383: dread d1/d8/d20/d45/f2a [0,4194304] 0 2024-04-22T23:46:06.298 INFO:tasks.workunit.client.1.smithi044.stdout:0/325: unlink d0/d2/dd/f1d 0 2024-04-22T23:46:06.302 INFO:tasks.workunit.client.1.smithi044.stdout:1/384: dwrite d1/d8/d1c/d7b/f65 [0,4194304] 0 2024-04-22T23:46:06.307 INFO:tasks.workunit.client.1.smithi044.stdout:6/303: mknod d5/d3f/d4c/d40/c65 0 2024-04-22T23:46:06.309 INFO:tasks.workunit.client.1.smithi044.stdout:3/358: rmdir d9/db/d22/d38 39 2024-04-22T23:46:06.313 INFO:tasks.workunit.client.1.smithi044.stdout:7/379: dwrite dd/f4a [0,4194304] 0 2024-04-22T23:46:06.315 INFO:tasks.workunit.client.1.smithi044.stdout:0/326: mkdir d0/d2/dd/d3e/d40/d43/d79 0 2024-04-22T23:46:06.318 INFO:tasks.workunit.client.1.smithi044.stdout:4/400: dwrite d5/d31/d24/f3d [0,4194304] 0 2024-04-22T23:46:06.319 INFO:tasks.workunit.client.1.smithi044.stdout:0/327: read d0/d2/dd/d3e/d40/d43/f66 [32399,88333] 0 2024-04-22T23:46:06.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/304: rename d5/l61 to d5/d3f/d4c/d4f/l66 0 2024-04-22T23:46:06.328 INFO:tasks.workunit.client.1.smithi044.stdout:1/385: dread d1/d8/d1c/f5b [0,4194304] 0 2024-04-22T23:46:06.329 INFO:tasks.workunit.client.1.smithi044.stdout:7/380: symlink dd/d55/d5e/d60/l84 0 2024-04-22T23:46:06.332 INFO:tasks.workunit.client.1.smithi044.stdout:2/313: getdents d8/d19/d36 0 2024-04-22T23:46:06.333 INFO:tasks.workunit.client.1.smithi044.stdout:7/381: dread dd/f3a [0,4194304] 0 2024-04-22T23:46:06.335 INFO:tasks.workunit.client.1.smithi044.stdout:4/401: chown d5/d31/d44/c1a 447058196 1 2024-04-22T23:46:06.335 INFO:tasks.workunit.client.1.smithi044.stdout:6/305: rmdir d5/d3f/d4c 39 2024-04-22T23:46:06.340 INFO:tasks.workunit.client.1.smithi044.stdout:1/386: creat d1/d8/d20/d45/d5d/f86 x:0 0 0 2024-04-22T23:46:06.346 INFO:tasks.workunit.client.1.smithi044.stdout:1/387: dwrite d1/d14/f36 [0,4194304] 0 2024-04-22T23:46:06.379 INFO:tasks.workunit.client.1.smithi044.stdout:8/377: sync 2024-04-22T23:46:06.379 INFO:tasks.workunit.client.1.smithi044.stdout:5/351: sync 2024-04-22T23:46:06.379 INFO:tasks.workunit.client.1.smithi044.stdout:6/306: rmdir d5/d3f 39 2024-04-22T23:46:06.381 INFO:tasks.workunit.client.1.smithi044.stdout:2/314: mkdir d8/d19/d36/d46/d63/d65 0 2024-04-22T23:46:06.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/378: rmdir db 39 2024-04-22T23:46:06.383 INFO:tasks.workunit.client.1.smithi044.stdout:1/388: dread d1/d14/f83 [0,4194304] 0 2024-04-22T23:46:06.384 INFO:tasks.workunit.client.1.smithi044.stdout:0/328: rmdir d0/d2e/d33/d5d 0 2024-04-22T23:46:06.385 INFO:tasks.workunit.client.1.smithi044.stdout:3/359: getdents d9/db/d15 0 2024-04-22T23:46:06.386 INFO:tasks.workunit.client.1.smithi044.stdout:0/329: chown d0/d2/dd/d3e/d40/l62 1518485 1 2024-04-22T23:46:06.386 INFO:tasks.workunit.client.1.smithi044.stdout:1/389: truncate d1/d8/d20/d47/f85 764764 0 2024-04-22T23:46:06.390 INFO:tasks.workunit.client.1.smithi044.stdout:1/390: dread d1/d8/d20/d47/f85 [0,4194304] 0 2024-04-22T23:46:06.391 INFO:tasks.workunit.client.1.smithi044.stdout:1/391: fsync d1/d14/f36 0 2024-04-22T23:46:06.392 INFO:tasks.workunit.client.1.smithi044.stdout:1/392: stat d1/d8/d34/f78 0 2024-04-22T23:46:06.395 INFO:tasks.workunit.client.1.smithi044.stdout:1/393: fsync d1/d8/d20/d45/d5d/f61 0 2024-04-22T23:46:06.395 INFO:tasks.workunit.client.1.smithi044.stdout:2/315: symlink d8/d19/d36/d46/l66 0 2024-04-22T23:46:06.401 INFO:tasks.workunit.client.1.smithi044.stdout:5/352: creat d15/d5d/d64/f6a x:0 0 0 2024-04-22T23:46:06.402 INFO:tasks.workunit.client.1.smithi044.stdout:7/382: link dd/d1f/d24/c2a dd/d55/d5e/d60/d7b/c85 0 2024-04-22T23:46:06.403 INFO:tasks.workunit.client.1.smithi044.stdout:1/394: dread d1/d8/d1c/d23/f5f [0,4194304] 0 2024-04-22T23:46:06.404 INFO:tasks.workunit.client.1.smithi044.stdout:7/383: chown dd/d55/d5e/d60 43498694 1 2024-04-22T23:46:06.405 INFO:tasks.workunit.client.1.smithi044.stdout:3/360: dread d9/db/f71 [0,4194304] 0 2024-04-22T23:46:06.407 INFO:tasks.workunit.client.1.smithi044.stdout:1/395: truncate d1/d8/d20/d45/f70 679972 0 2024-04-22T23:46:06.407 INFO:tasks.workunit.client.1.smithi044.stdout:0/330: dread d0/d2e/d2f/f4d [0,4194304] 0 2024-04-22T23:46:06.415 INFO:tasks.workunit.client.1.smithi044.stdout:6/307: rename d5/dd/d18 to d5/dd/d67 0 2024-04-22T23:46:06.418 INFO:tasks.workunit.client.1.smithi044.stdout:5/353: rename d15/d16/d1d/l67 to d15/d3c/d65/l6b 0 2024-04-22T23:46:06.419 INFO:tasks.workunit.client.1.smithi044.stdout:5/354: write d15/d16/d1d/f26 [7651820,50927] 0 2024-04-22T23:46:06.420 INFO:tasks.workunit.client.1.smithi044.stdout:7/384: fsync dd/d1f/d24/f28 0 2024-04-22T23:46:06.424 INFO:tasks.workunit.client.1.smithi044.stdout:0/331: fdatasync d0/f17 0 2024-04-22T23:46:06.428 INFO:tasks.workunit.client.1.smithi044.stdout:8/379: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:06.431 INFO:tasks.workunit.client.1.smithi044.stdout:6/308: write d5/d3f/f63 [1039278,30624] 0 2024-04-22T23:46:06.441 INFO:tasks.workunit.client.1.smithi044.stdout:7/385: rename dd/d1a/f3b to dd/d55/d71/f86 0 2024-04-22T23:46:06.441 INFO:tasks.workunit.client.1.smithi044.stdout:3/361: symlink d9/db/d33/d5d/d70/l79 0 2024-04-22T23:46:06.444 INFO:tasks.workunit.client.1.smithi044.stdout:2/316: link d8/d27/f2a d8/d19/d36/d46/f67 0 2024-04-22T23:46:06.448 INFO:tasks.workunit.client.1.smithi044.stdout:5/355: creat d15/d16/d1d/d22/d4f/d68/f6c x:0 0 0 2024-04-22T23:46:06.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/356: chown d15/d16/d1d/d22/d4f 1022699 1 2024-04-22T23:46:06.451 INFO:tasks.workunit.client.1.smithi044.stdout:7/386: stat dd/d1a/l69 0 2024-04-22T23:46:06.457 INFO:tasks.workunit.client.1.smithi044.stdout:2/317: symlink d8/d19/d36/d46/d54/l68 0 2024-04-22T23:46:06.458 INFO:tasks.workunit.client.1.smithi044.stdout:8/380: creat db/d14/d24/f8d x:0 0 0 2024-04-22T23:46:06.460 INFO:tasks.workunit.client.1.smithi044.stdout:7/387: truncate dd/d1a/f1d 3396436 0 2024-04-22T23:46:06.461 INFO:tasks.workunit.client.1.smithi044.stdout:7/388: readlink dd/l5c 0 2024-04-22T23:46:06.461 INFO:tasks.workunit.client.1.smithi044.stdout:3/362: mknod d9/db/d22/d38/c7a 0 2024-04-22T23:46:06.463 INFO:tasks.workunit.client.1.smithi044.stdout:0/332: creat d0/d2e/f7a x:0 0 0 2024-04-22T23:46:06.466 INFO:tasks.workunit.client.1.smithi044.stdout:6/309: creat d5/dd/f68 x:0 0 0 2024-04-22T23:46:06.467 INFO:tasks.workunit.client.1.smithi044.stdout:5/357: mknod d15/c6d 0 2024-04-22T23:46:06.472 INFO:tasks.workunit.client.1.smithi044.stdout:2/318: creat d8/d60/f69 x:0 0 0 2024-04-22T23:46:06.473 INFO:tasks.workunit.client.1.smithi044.stdout:2/319: chown f3 10 1 2024-04-22T23:46:06.473 INFO:tasks.workunit.client.1.smithi044.stdout:2/320: dread - d8/d27/f56 zero size 2024-04-22T23:46:06.478 INFO:tasks.workunit.client.1.smithi044.stdout:2/321: symlink d8/d19/d5d/l6a 0 2024-04-22T23:46:06.482 INFO:tasks.workunit.client.1.smithi044.stdout:5/358: unlink ff 0 2024-04-22T23:46:06.485 INFO:tasks.workunit.client.1.smithi044.stdout:3/363: rename d9/db/d15/d2e/d48/l50 to d9/l7b 0 2024-04-22T23:46:06.486 INFO:tasks.workunit.client.1.smithi044.stdout:6/310: mknod d5/d3f/d4c/d40/d53/c69 0 2024-04-22T23:46:06.487 INFO:tasks.workunit.client.1.smithi044.stdout:3/364: truncate d9/db/d22/d38/f4a 4967034 0 2024-04-22T23:46:06.487 INFO:tasks.workunit.client.1.smithi044.stdout:6/311: dread - d5/d3f/d4c/d23/d3d/f51 zero size 2024-04-22T23:46:06.490 INFO:tasks.workunit.client.1.smithi044.stdout:8/381: write db/d14/f15 [4242569,72239] 0 2024-04-22T23:46:06.491 INFO:tasks.workunit.client.1.smithi044.stdout:8/382: write f6 [336663,41006] 0 2024-04-22T23:46:06.491 INFO:tasks.workunit.client.1.smithi044.stdout:0/333: rename d0/d2/d3d to d0/d2/dd/d3e/d40/d43/d7b 0 2024-04-22T23:46:06.494 INFO:tasks.workunit.client.1.smithi044.stdout:8/383: dread - db/f4f zero size 2024-04-22T23:46:06.497 INFO:tasks.workunit.client.1.smithi044.stdout:5/359: dwrite d15/d5d/f5e [0,4194304] 0 2024-04-22T23:46:06.500 INFO:tasks.workunit.client.1.smithi044.stdout:2/322: rename d8/d19/d36/d46/d54/c64 to d8/d27/d5b/c6b 0 2024-04-22T23:46:06.501 INFO:tasks.workunit.client.1.smithi044.stdout:7/389: dwrite dd/d1f/d24/f28 [0,4194304] 0 2024-04-22T23:46:06.506 INFO:tasks.workunit.client.1.smithi044.stdout:3/365: link d9/db/d15/d25/d4c/f4d d9/db/d33/f7c 0 2024-04-22T23:46:06.509 INFO:tasks.workunit.client.1.smithi044.stdout:6/312: creat d5/dd/f6a x:0 0 0 2024-04-22T23:46:06.509 INFO:tasks.workunit.client.1.smithi044.stdout:6/313: readlink d5/d3f/lb 0 2024-04-22T23:46:06.514 INFO:tasks.workunit.client.1.smithi044.stdout:7/390: mkdir dd/d55/d5e/d87 0 2024-04-22T23:46:06.514 INFO:tasks.workunit.client.1.smithi044.stdout:0/334: mkdir d0/d2/dd/d3e/d40/d67/d7c 0 2024-04-22T23:46:06.516 INFO:tasks.workunit.client.1.smithi044.stdout:8/384: write db/d14/d72/f81 [796148,102538] 0 2024-04-22T23:46:06.518 INFO:tasks.workunit.client.1.smithi044.stdout:6/314: rmdir d5/d3f/d4c/d23 39 2024-04-22T23:46:06.522 INFO:tasks.workunit.client.1.smithi044.stdout:0/335: creat d0/d2e/d33/f7d x:0 0 0 2024-04-22T23:46:06.529 INFO:tasks.workunit.client.1.smithi044.stdout:7/391: rename c6 to dd/d55/d5e/d60/d7b/c88 0 2024-04-22T23:46:06.529 INFO:tasks.workunit.client.1.smithi044.stdout:7/392: truncate dd/d27/f46 767165 0 2024-04-22T23:46:06.534 INFO:tasks.workunit.client.1.smithi044.stdout:8/385: dread db/d14/f42 [0,4194304] 0 2024-04-22T23:46:06.534 INFO:tasks.workunit.client.1.smithi044.stdout:0/336: symlink d0/d2/dd/d3e/l7e 0 2024-04-22T23:46:06.538 INFO:tasks.workunit.client.1.smithi044.stdout:3/366: getdents d9/db/d15/d25/d5b/d52 0 2024-04-22T23:46:06.540 INFO:tasks.workunit.client.1.smithi044.stdout:3/367: chown d9/db/d15/d2e/d48/f73 5 1 2024-04-22T23:46:06.543 INFO:tasks.workunit.client.1.smithi044.stdout:0/337: symlink d0/d4c/d59/d5a/l7f 0 2024-04-22T23:46:06.543 INFO:tasks.workunit.client.1.smithi044.stdout:2/323: write d8/d27/f28 [143164,63227] 0 2024-04-22T23:46:06.544 INFO:tasks.workunit.client.1.smithi044.stdout:6/315: creat d5/f6b x:0 0 0 2024-04-22T23:46:06.546 INFO:tasks.workunit.client.1.smithi044.stdout:6/316: chown d5/d3f/d50/c5c 86503 1 2024-04-22T23:46:06.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/368: mkdir d9/d14/d7d 0 2024-04-22T23:46:06.548 INFO:tasks.workunit.client.1.smithi044.stdout:3/369: chown f7 50755 1 2024-04-22T23:46:06.551 INFO:tasks.workunit.client.1.smithi044.stdout:2/324: unlink d8/d19/d5d/c62 0 2024-04-22T23:46:06.552 INFO:tasks.workunit.client.1.smithi044.stdout:5/360: truncate d15/d16/d1d/d22/d4f/f52 500670 0 2024-04-22T23:46:06.553 INFO:tasks.workunit.client.1.smithi044.stdout:6/317: dwrite d5/dd/f3e [0,4194304] 0 2024-04-22T23:46:06.555 INFO:tasks.workunit.client.1.smithi044.stdout:8/386: rename db/f10 to db/d14/d72/f8e 0 2024-04-22T23:46:06.558 INFO:tasks.workunit.client.1.smithi044.stdout:3/370: truncate d9/d12/f2d 1154304 0 2024-04-22T23:46:06.563 INFO:tasks.workunit.client.1.smithi044.stdout:5/361: symlink d15/d5d/d64/l6e 0 2024-04-22T23:46:06.571 INFO:tasks.workunit.client.1.smithi044.stdout:5/362: dwrite d15/d16/d1d/d22/f23 [0,4194304] 0 2024-04-22T23:46:06.572 INFO:tasks.workunit.client.1.smithi044.stdout:5/363: chown d15/d16/d2c/f5a 829 1 2024-04-22T23:46:06.574 INFO:tasks.workunit.client.1.smithi044.stdout:3/371: dread d9/db/d15/d2e/f32 [0,4194304] 0 2024-04-22T23:46:06.574 INFO:tasks.workunit.client.1.smithi044.stdout:0/338: write d0/d2/dd/d3e/d40/d43/f4a [2548753,83157] 0 2024-04-22T23:46:06.576 INFO:tasks.workunit.client.1.smithi044.stdout:0/339: chown d0/d2e/f32 105 1 2024-04-22T23:46:06.587 INFO:tasks.workunit.client.1.smithi044.stdout:5/364: rmdir d15/d5d 39 2024-04-22T23:46:06.587 INFO:tasks.workunit.client.1.smithi044.stdout:2/325: write d8/d27/f2c [303251,77324] 0 2024-04-22T23:46:06.588 INFO:tasks.workunit.client.1.smithi044.stdout:2/326: chown d8/d27/f2c 112471 1 2024-04-22T23:46:06.596 INFO:tasks.workunit.client.1.smithi044.stdout:6/318: creat d5/d3f/d4c/d40/d53/f6c x:0 0 0 2024-04-22T23:46:06.596 INFO:tasks.workunit.client.1.smithi044.stdout:8/387: creat db/d14/d72/f8f x:0 0 0 2024-04-22T23:46:06.597 INFO:tasks.workunit.client.1.smithi044.stdout:5/365: fsync f11 0 2024-04-22T23:46:06.598 INFO:tasks.workunit.client.1.smithi044.stdout:5/366: chown d15/d16/d1d/d22/d40 3642653 1 2024-04-22T23:46:06.600 INFO:tasks.workunit.client.1.smithi044.stdout:0/340: getdents d0/d2/dd/d3e/d40/d43/d79 0 2024-04-22T23:46:06.601 INFO:tasks.workunit.client.1.smithi044.stdout:0/341: readlink d0/d2e/d33/l4e 0 2024-04-22T23:46:06.602 INFO:tasks.workunit.client.1.smithi044.stdout:6/319: creat d5/d3f/d50/f6d x:0 0 0 2024-04-22T23:46:06.604 INFO:tasks.workunit.client.1.smithi044.stdout:8/388: chown db/d14/l6d 453847 1 2024-04-22T23:46:06.605 INFO:tasks.workunit.client.1.smithi044.stdout:8/389: dread - db/f3f zero size 2024-04-22T23:46:06.606 INFO:tasks.workunit.client.1.smithi044.stdout:2/327: mkdir d8/d19/d36/d46/d63/d65/d6c 0 2024-04-22T23:46:06.609 INFO:tasks.workunit.client.1.smithi044.stdout:0/342: dwrite d0/d2e/d2f/f5e [0,4194304] 0 2024-04-22T23:46:06.610 INFO:tasks.workunit.client.1.smithi044.stdout:5/367: write d15/d5d/d64/f6a [209317,77411] 0 2024-04-22T23:46:06.610 INFO:tasks.workunit.client.1.smithi044.stdout:0/343: dread - d0/d4c/f75 zero size 2024-04-22T23:46:06.614 INFO:tasks.workunit.client.1.smithi044.stdout:6/320: truncate d5/d3f/fa 1844745 0 2024-04-22T23:46:06.615 INFO:tasks.workunit.client.1.smithi044.stdout:6/321: fsync d5/f38 0 2024-04-22T23:46:06.619 INFO:tasks.workunit.client.1.smithi044.stdout:2/328: symlink d8/d19/d1c/l6d 0 2024-04-22T23:46:06.620 INFO:tasks.workunit.client.1.smithi044.stdout:2/329: readlink d8/d19/d36/l61 0 2024-04-22T23:46:06.621 INFO:tasks.workunit.client.1.smithi044.stdout:3/372: unlink d9/d12/f2d 0 2024-04-22T23:46:06.622 INFO:tasks.workunit.client.1.smithi044.stdout:5/368: mkdir d15/d5d/d64/d6f 0 2024-04-22T23:46:06.624 INFO:tasks.workunit.client.1.smithi044.stdout:0/344: creat d0/d2/d4b/f80 x:0 0 0 2024-04-22T23:46:06.627 INFO:tasks.workunit.client.1.smithi044.stdout:8/390: mknod db/d68/d8b/c90 0 2024-04-22T23:46:06.627 INFO:tasks.workunit.client.1.smithi044.stdout:6/322: dread d5/d3f/d4c/d40/f41 [0,4194304] 0 2024-04-22T23:46:06.628 INFO:tasks.workunit.client.1.smithi044.stdout:6/323: readlink d5/d3f/d4c/l33 0 2024-04-22T23:46:06.629 INFO:tasks.workunit.client.1.smithi044.stdout:8/391: write db/d14/d24/f8d [206725,18171] 0 2024-04-22T23:46:06.631 INFO:tasks.workunit.client.1.smithi044.stdout:3/373: chown d9/db/d15/d54/d3c/c6e 3825625 1 2024-04-22T23:46:06.631 INFO:tasks.workunit.client.1.smithi044.stdout:3/374: stat d9/db/d33/f7c 0 2024-04-22T23:46:06.632 INFO:tasks.workunit.client.1.smithi044.stdout:5/369: rename d15/d16/d1d/d22/d4f/l57 to d15/d5d/d64/d6f/l70 0 2024-04-22T23:46:06.637 INFO:tasks.workunit.client.1.smithi044.stdout:2/330: mknod d8/d19/c6e 0 2024-04-22T23:46:06.639 INFO:tasks.workunit.client.1.smithi044.stdout:6/324: creat d5/d3f/d4c/d40/d53/d56/f6e x:0 0 0 2024-04-22T23:46:06.641 INFO:tasks.workunit.client.1.smithi044.stdout:0/345: dwrite d0/d2e/d2f/f4d [4194304,4194304] 0 2024-04-22T23:46:06.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/346: dwrite d0/d2e/d2f/f4d [0,4194304] 0 2024-04-22T23:46:06.651 INFO:tasks.workunit.client.1.smithi044.stdout:3/375: creat d9/db/d15/d25/d5b/f7e x:0 0 0 2024-04-22T23:46:06.653 INFO:tasks.workunit.client.1.smithi044.stdout:2/331: rename d8/d19/d36/d46/d54/f57 to d8/d27/f6f 0 2024-04-22T23:46:06.657 INFO:tasks.workunit.client.1.smithi044.stdout:8/392: link db/d14/d24/f26 db/d6c/f91 0 2024-04-22T23:46:06.663 INFO:tasks.workunit.client.1.smithi044.stdout:2/332: dwrite d8/d27/f4b [0,4194304] 0 2024-04-22T23:46:06.663 INFO:tasks.workunit.client.1.smithi044.stdout:8/393: creat db/d68/d74/f92 x:0 0 0 2024-04-22T23:46:06.664 INFO:tasks.workunit.client.1.smithi044.stdout:8/394: readlink db/d14/l2d 0 2024-04-22T23:46:06.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/395: dread - db/d14/d24/f30 zero size 2024-04-22T23:46:06.666 INFO:tasks.workunit.client.1.smithi044.stdout:0/347: symlink d0/l81 0 2024-04-22T23:46:06.668 INFO:tasks.workunit.client.1.smithi044.stdout:0/348: write d0/d4c/f75 [931119,2902] 0 2024-04-22T23:46:06.668 INFO:tasks.workunit.client.1.smithi044.stdout:2/333: unlink d8/d19/d36/l3c 0 2024-04-22T23:46:06.672 INFO:tasks.workunit.client.1.smithi044.stdout:8/396: mkdir db/d6c/d93 0 2024-04-22T23:46:06.677 INFO:tasks.workunit.client.1.smithi044.stdout:3/376: rename c3 to d9/db/d22/c7f 0 2024-04-22T23:46:06.679 INFO:tasks.workunit.client.1.smithi044.stdout:0/349: dwrite d0/d4c/f75 [0,4194304] 0 2024-04-22T23:46:06.684 INFO:tasks.workunit.client.1.smithi044.stdout:2/334: creat d8/d19/d36/d44/f70 x:0 0 0 2024-04-22T23:46:06.685 INFO:tasks.workunit.client.1.smithi044.stdout:2/335: dread - d8/d19/f5c zero size 2024-04-22T23:46:06.689 INFO:tasks.workunit.client.1.smithi044.stdout:3/377: mknod d9/db/d15/d25/c80 0 2024-04-22T23:46:06.691 INFO:tasks.workunit.client.1.smithi044.stdout:0/350: creat d0/d2/d4b/d6b/f82 x:0 0 0 2024-04-22T23:46:06.693 INFO:tasks.workunit.client.1.smithi044.stdout:2/336: creat d8/d19/d36/d46/d63/f71 x:0 0 0 2024-04-22T23:46:06.697 INFO:tasks.workunit.client.1.smithi044.stdout:8/397: mknod db/d14/c94 0 2024-04-22T23:46:06.699 INFO:tasks.workunit.client.1.smithi044.stdout:5/370: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f61 [0,4194304] 0 2024-04-22T23:46:06.699 INFO:tasks.workunit.client.1.smithi044.stdout:8/398: readlink db/d14/l2e 0 2024-04-22T23:46:06.700 INFO:tasks.workunit.client.1.smithi044.stdout:5/371: write d15/f30 [3260381,75083] 0 2024-04-22T23:46:06.702 INFO:tasks.workunit.client.1.smithi044.stdout:5/372: dread - d15/d16/d1d/d22/d4f/d68/f6c zero size 2024-04-22T23:46:06.702 INFO:tasks.workunit.client.1.smithi044.stdout:2/337: write d8/d19/f37 [3508517,101218] 0 2024-04-22T23:46:06.706 INFO:tasks.workunit.client.1.smithi044.stdout:8/399: rmdir db/d52 39 2024-04-22T23:46:06.709 INFO:tasks.workunit.client.1.smithi044.stdout:5/373: symlink d15/d16/d1d/d22/l71 0 2024-04-22T23:46:06.712 INFO:tasks.workunit.client.1.smithi044.stdout:3/378: getdents d9/db/d15/d25/d67 0 2024-04-22T23:46:06.712 INFO:tasks.workunit.client.1.smithi044.stdout:3/379: readlink l4 0 2024-04-22T23:46:06.713 INFO:tasks.workunit.client.1.smithi044.stdout:0/351: getdents d0/d2/dd/d3e/d40 0 2024-04-22T23:46:06.714 INFO:tasks.workunit.client.1.smithi044.stdout:3/380: fdatasync d9/db/d15/d2e/f3f 0 2024-04-22T23:46:06.720 INFO:tasks.workunit.client.1.smithi044.stdout:3/381: creat d9/db/d15/d54/d3c/f81 x:0 0 0 2024-04-22T23:46:06.721 INFO:tasks.workunit.client.1.smithi044.stdout:5/374: dread d15/d16/d2c/f54 [0,4194304] 0 2024-04-22T23:46:06.722 INFO:tasks.workunit.client.1.smithi044.stdout:5/375: readlink d15/d16/d1d/l62 0 2024-04-22T23:46:06.725 INFO:tasks.workunit.client.1.smithi044.stdout:3/382: rmdir d9/db/d15/d2e/d48 39 2024-04-22T23:46:06.726 INFO:tasks.workunit.client.1.smithi044.stdout:3/383: stat d9/f2a 0 2024-04-22T23:46:06.730 INFO:tasks.workunit.client.1.smithi044.stdout:8/400: dread f6 [4194304,4194304] 0 2024-04-22T23:46:06.733 INFO:tasks.workunit.client.1.smithi044.stdout:3/384: symlink d9/db/d15/d25/d4c/l82 0 2024-04-22T23:46:06.734 INFO:tasks.workunit.client.1.smithi044.stdout:8/401: dread db/f44 [0,4194304] 0 2024-04-22T23:46:06.735 INFO:tasks.workunit.client.1.smithi044.stdout:8/402: chown db/fd 103197 1 2024-04-22T23:46:06.740 INFO:tasks.workunit.client.1.smithi044.stdout:5/376: dread d15/d16/d1d/f26 [0,4194304] 0 2024-04-22T23:46:06.742 INFO:tasks.workunit.client.1.smithi044.stdout:3/385: rename d9/c20 to d9/db/d15/d25/d67/d6b/c83 0 2024-04-22T23:46:06.751 INFO:tasks.workunit.client.1.smithi044.stdout:5/377: truncate d15/d16/d2c/f2e 713229 0 2024-04-22T23:46:06.751 INFO:tasks.workunit.client.1.smithi044.stdout:0/352: dwrite d0/f17 [0,4194304] 0 2024-04-22T23:46:06.752 INFO:tasks.workunit.client.1.smithi044.stdout:0/353: chown d0/d2e/d33/f7d 274 1 2024-04-22T23:46:06.753 INFO:tasks.workunit.client.1.smithi044.stdout:8/403: link db/f3d db/d14/d72/f95 0 2024-04-22T23:46:06.755 INFO:tasks.workunit.client.1.smithi044.stdout:8/404: read - db/d14/d24/f59 zero size 2024-04-22T23:46:06.760 INFO:tasks.workunit.client.1.smithi044.stdout:5/378: symlink d15/d16/d2c/l72 0 2024-04-22T23:46:06.767 INFO:tasks.workunit.client.1.smithi044.stdout:8/405: write db/d14/f42 [681899,12569] 0 2024-04-22T23:46:06.768 INFO:tasks.workunit.client.1.smithi044.stdout:5/379: dwrite d15/d16/d1d/d22/d4f/d68/f6c [0,4194304] 0 2024-04-22T23:46:06.771 INFO:tasks.workunit.client.1.smithi044.stdout:3/386: getdents d9/db/d15/d54 0 2024-04-22T23:46:06.771 INFO:tasks.workunit.client.1.smithi044.stdout:5/380: write d15/d16/d1d/f69 [417716,56159] 0 2024-04-22T23:46:06.774 INFO:tasks.workunit.client.1.smithi044.stdout:5/381: fdatasync d15/d16/d1d/d22/d2a/d3a/d43/f4c 0 2024-04-22T23:46:06.775 INFO:tasks.workunit.client.1.smithi044.stdout:8/406: rename db/d14/d24/l40 to db/d68/d8b/l96 0 2024-04-22T23:46:06.776 INFO:tasks.workunit.client.1.smithi044.stdout:8/407: chown db/d68/d74 172704 1 2024-04-22T23:46:06.777 INFO:tasks.workunit.client.1.smithi044.stdout:3/387: read - d9/db/d15/d2e/d48/f73 zero size 2024-04-22T23:46:06.779 INFO:tasks.workunit.client.1.smithi044.stdout:9/243: sync 2024-04-22T23:46:06.780 INFO:tasks.workunit.client.1.smithi044.stdout:4/402: sync 2024-04-22T23:46:06.780 INFO:tasks.workunit.client.1.smithi044.stdout:0/354: link d0/d2/l29 d0/d2/dd/d3e/d40/d43/d7b/l83 0 2024-04-22T23:46:06.783 INFO:tasks.workunit.client.1.smithi044.stdout:5/382: dwrite d15/d16/d1d/d22/d4f/f59 [0,4194304] 0 2024-04-22T23:46:06.787 INFO:tasks.workunit.client.1.smithi044.stdout:3/388: chown d9/l7b 1 1 2024-04-22T23:46:06.789 INFO:tasks.workunit.client.1.smithi044.stdout:3/389: fsync f8 0 2024-04-22T23:46:06.793 INFO:tasks.workunit.client.1.smithi044.stdout:0/355: dwrite d0/d2/f1e [0,4194304] 0 2024-04-22T23:46:06.796 INFO:tasks.workunit.client.1.smithi044.stdout:5/383: dwrite d15/d5d/f5e [0,4194304] 0 2024-04-22T23:46:06.801 INFO:tasks.workunit.client.1.smithi044.stdout:4/403: dwrite d5/d31/f76 [0,4194304] 0 2024-04-22T23:46:06.802 INFO:tasks.workunit.client.1.smithi044.stdout:4/404: dread - d5/d31/d24/f72 zero size 2024-04-22T23:46:06.806 INFO:tasks.workunit.client.1.smithi044.stdout:9/244: mkdir d2/dc/d1d/d34/d38/d5c 0 2024-04-22T23:46:06.809 INFO:tasks.workunit.client.1.smithi044.stdout:9/245: write d2/dc/f32 [921894,21432] 0 2024-04-22T23:46:06.809 INFO:tasks.workunit.client.1.smithi044.stdout:9/246: readlink d2/d6/d42/l4e 0 2024-04-22T23:46:06.817 INFO:tasks.workunit.client.1.smithi044.stdout:4/405: creat d5/d31/d24/d69/f81 x:0 0 0 2024-04-22T23:46:06.819 INFO:tasks.workunit.client.1.smithi044.stdout:4/406: fdatasync d5/d31/d24/f72 0 2024-04-22T23:46:06.825 INFO:tasks.workunit.client.1.smithi044.stdout:5/384: dread f12 [0,4194304] 0 2024-04-22T23:46:06.826 INFO:tasks.workunit.client.1.smithi044.stdout:5/385: truncate d15/d16/d1d/f3f 4876234 0 2024-04-22T23:46:06.888 INFO:tasks.workunit.client.1.smithi044.stdout:8/408: creat db/d14/f97 x:0 0 0 2024-04-22T23:46:06.889 INFO:tasks.workunit.client.1.smithi044.stdout:0/356: truncate d0/d2/dd/d3e/d40/f73 6613375 0 2024-04-22T23:46:06.891 INFO:tasks.workunit.client.1.smithi044.stdout:4/407: write d5/d31/d44/d20/f55 [1540906,83715] 0 2024-04-22T23:46:06.891 INFO:tasks.workunit.client.1.smithi044.stdout:5/386: mknod d15/d16/d1d/d22/d2a/d3a/d43/c73 0 2024-04-22T23:46:06.897 INFO:tasks.workunit.client.1.smithi044.stdout:8/409: symlink db/d14/d84/l98 0 2024-04-22T23:46:06.899 INFO:tasks.workunit.client.1.smithi044.stdout:8/410: write db/d14/f5a [865478,128481] 0 2024-04-22T23:46:06.899 INFO:tasks.workunit.client.1.smithi044.stdout:4/408: creat d5/db/d3c/f82 x:0 0 0 2024-04-22T23:46:06.902 INFO:tasks.workunit.client.1.smithi044.stdout:5/387: creat d15/d5d/d64/d6f/f74 x:0 0 0 2024-04-22T23:46:06.904 INFO:tasks.workunit.client.1.smithi044.stdout:9/247: getdents d2/d9 0 2024-04-22T23:46:06.904 INFO:tasks.workunit.client.1.smithi044.stdout:0/357: dwrite d0/d2/dd/d3e/d40/d43/d7b/f26 [0,4194304] 0 2024-04-22T23:46:06.908 INFO:tasks.workunit.client.1.smithi044.stdout:4/409: dread d5/d31/d24/d38/f63 [0,4194304] 0 2024-04-22T23:46:06.913 INFO:tasks.workunit.client.1.smithi044.stdout:5/388: unlink d15/f1e 0 2024-04-22T23:46:06.914 INFO:tasks.workunit.client.1.smithi044.stdout:5/389: fsync d15/d16/d1d/f49 0 2024-04-22T23:46:06.917 INFO:tasks.workunit.client.1.smithi044.stdout:8/411: rename db/d68/d74/f92 to db/d14/d84/f99 0 2024-04-22T23:46:06.918 INFO:tasks.workunit.client.1.smithi044.stdout:8/412: stat db/f48 0 2024-04-22T23:46:06.918 INFO:tasks.workunit.client.1.smithi044.stdout:8/413: stat db/c7f 0 2024-04-22T23:46:06.920 INFO:tasks.workunit.client.1.smithi044.stdout:9/248: mknod d2/dc/d1d/c5d 0 2024-04-22T23:46:06.921 INFO:tasks.workunit.client.1.smithi044.stdout:9/249: write d2/dc/d1d/d3f/f48 [935491,66191] 0 2024-04-22T23:46:06.922 INFO:tasks.workunit.client.1.smithi044.stdout:0/358: creat d0/d2/dd/d3e/d40/f84 x:0 0 0 2024-04-22T23:46:06.926 INFO:tasks.workunit.client.1.smithi044.stdout:4/410: creat d5/d31/d24/d38/d6a/f83 x:0 0 0 2024-04-22T23:46:06.931 INFO:tasks.workunit.client.1.smithi044.stdout:4/411: dread d5/d31/d44/d20/d22/f62 [0,4194304] 0 2024-04-22T23:46:06.932 INFO:tasks.workunit.client.1.smithi044.stdout:4/412: stat d5/d31/d44/d20/l7c 0 2024-04-22T23:46:06.940 INFO:tasks.workunit.client.1.smithi044.stdout:8/414: rmdir db/d14/d24 39 2024-04-22T23:46:06.942 INFO:tasks.workunit.client.1.smithi044.stdout:8/415: write db/f11 [1196684,88986] 0 2024-04-22T23:46:06.949 INFO:tasks.workunit.client.1.smithi044.stdout:8/416: dwrite db/d14/f15 [4194304,4194304] 0 2024-04-22T23:46:06.950 INFO:tasks.workunit.client.1.smithi044.stdout:4/413: rmdir d5/db/d16 39 2024-04-22T23:46:06.956 INFO:tasks.workunit.client.1.smithi044.stdout:1/396: sync 2024-04-22T23:46:06.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:06 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:06.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:06 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:06.963 INFO:tasks.workunit.client.1.smithi044.stdout:9/250: getdents d2/dc/d1d/d34/d41 0 2024-04-22T23:46:06.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/359: fsync d0/d2/f25 0 2024-04-22T23:46:06.967 INFO:tasks.workunit.client.1.smithi044.stdout:8/417: symlink db/d68/d74/l9a 0 2024-04-22T23:46:06.970 INFO:tasks.workunit.client.1.smithi044.stdout:4/414: mkdir d5/d31/d24/d38/d84 0 2024-04-22T23:46:06.971 INFO:tasks.workunit.client.1.smithi044.stdout:4/415: dread - d5/db/d3c/f82 zero size 2024-04-22T23:46:06.972 INFO:tasks.workunit.client.1.smithi044.stdout:5/390: creat d15/d16/f75 x:0 0 0 2024-04-22T23:46:06.976 INFO:tasks.workunit.client.1.smithi044.stdout:0/360: creat d0/d2e/d33/f85 x:0 0 0 2024-04-22T23:46:06.977 INFO:tasks.workunit.client.1.smithi044.stdout:8/418: readlink db/d68/d8b/l96 0 2024-04-22T23:46:06.979 INFO:tasks.workunit.client.1.smithi044.stdout:4/416: symlink d5/d31/d24/d38/d6a/l85 0 2024-04-22T23:46:06.979 INFO:tasks.workunit.client.1.smithi044.stdout:5/391: rmdir d15/d3c/d65 39 2024-04-22T23:46:06.980 INFO:tasks.workunit.client.1.smithi044.stdout:4/417: read - d5/d31/d24/d38/d6a/f83 zero size 2024-04-22T23:46:06.982 INFO:tasks.workunit.client.1.smithi044.stdout:5/392: truncate d15/d16/d1d/d22/d2a/d3a/d43/f58 282654 0 2024-04-22T23:46:06.984 INFO:tasks.workunit.client.1.smithi044.stdout:0/361: write d0/d2/f25 [2217432,119821] 0 2024-04-22T23:46:06.986 INFO:tasks.workunit.client.1.smithi044.stdout:9/251: creat d2/d6/d4d/f5e x:0 0 0 2024-04-22T23:46:06.990 INFO:tasks.workunit.client.1.smithi044.stdout:8/419: dread - db/d14/d24/f75 zero size 2024-04-22T23:46:06.990 INFO:tasks.workunit.client.1.smithi044.stdout:9/252: dread d2/f3d [0,4194304] 0 2024-04-22T23:46:06.991 INFO:tasks.workunit.client.1.smithi044.stdout:1/397: truncate d1/d8/d1c/d23/d3e/f58 3700334 0 2024-04-22T23:46:06.993 INFO:tasks.workunit.client.1.smithi044.stdout:5/393: mkdir d15/d16/d1d/d22/d4f/d68/d76 0 2024-04-22T23:46:07.002 INFO:tasks.workunit.client.1.smithi044.stdout:8/420: write db/f69 [560612,83907] 0 2024-04-22T23:46:07.002 INFO:tasks.workunit.client.1.smithi044.stdout:1/398: symlink d1/d8/d20/d45/d5d/l87 0 2024-04-22T23:46:07.002 INFO:tasks.workunit.client.1.smithi044.stdout:0/362: dwrite d0/d2e/f70 [4194304,4194304] 0 2024-04-22T23:46:07.005 INFO:tasks.workunit.client.1.smithi044.stdout:4/418: creat d5/db/f86 x:0 0 0 2024-04-22T23:46:07.005 INFO:tasks.workunit.client.1.smithi044.stdout:4/419: read - d5/d31/d24/d69/f81 zero size 2024-04-22T23:46:07.008 INFO:tasks.workunit.client.1.smithi044.stdout:4/420: truncate d5/db/d3c/f82 324972 0 2024-04-22T23:46:07.009 INFO:tasks.workunit.client.1.smithi044.stdout:4/421: dread - d5/d31/d24/d38/d6a/d75/f7f zero size 2024-04-22T23:46:07.010 INFO:tasks.workunit.client.1.smithi044.stdout:1/399: readlink d1/d14/l30 0 2024-04-22T23:46:07.012 INFO:tasks.workunit.client.1.smithi044.stdout:0/363: creat d0/d2/d4b/f86 x:0 0 0 2024-04-22T23:46:07.014 INFO:tasks.workunit.client.1.smithi044.stdout:0/364: write d0/d2e/d2f/f4d [1298153,66444] 0 2024-04-22T23:46:07.015 INFO:tasks.workunit.client.1.smithi044.stdout:8/421: symlink db/d52/l9b 0 2024-04-22T23:46:07.018 INFO:tasks.workunit.client.1.smithi044.stdout:5/394: creat d15/d3c/f77 x:0 0 0 2024-04-22T23:46:07.018 INFO:tasks.workunit.client.1.smithi044.stdout:9/253: getdents d2/dc/d1d/d34 0 2024-04-22T23:46:07.019 INFO:tasks.workunit.client.1.smithi044.stdout:9/254: write d2/d6/f53 [109492,2343] 0 2024-04-22T23:46:07.021 INFO:tasks.workunit.client.1.smithi044.stdout:4/422: symlink d5/db/d34/l87 0 2024-04-22T23:46:07.022 INFO:tasks.workunit.client.1.smithi044.stdout:4/423: chown d5/d31/d44/d20/d22/c2c 1242485 1 2024-04-22T23:46:07.023 INFO:tasks.workunit.client.1.smithi044.stdout:1/400: creat d1/d8/d1c/d7b/d80/d4b/f88 x:0 0 0 2024-04-22T23:46:07.026 INFO:tasks.workunit.client.1.smithi044.stdout:1/401: truncate d1/d8/d1c/d7b/d80/d4b/f5a 1102796 0 2024-04-22T23:46:07.026 INFO:tasks.workunit.client.1.smithi044.stdout:7/393: sync 2024-04-22T23:46:07.027 INFO:tasks.workunit.client.1.smithi044.stdout:1/402: write d1/d8/d20/d45/d5d/f86 [893843,91252] 0 2024-04-22T23:46:07.030 INFO:tasks.workunit.client.1.smithi044.stdout:8/422: mkdir db/d52/d9c 0 2024-04-22T23:46:07.032 INFO:tasks.workunit.client.1.smithi044.stdout:4/424: creat d5/d42/d73/f88 x:0 0 0 2024-04-22T23:46:07.037 INFO:tasks.workunit.client.1.smithi044.stdout:9/255: rename d2/le to d2/dc/l5f 0 2024-04-22T23:46:07.039 INFO:tasks.workunit.client.1.smithi044.stdout:1/403: creat d1/d8/d20/d45/d75/d77/f89 x:0 0 0 2024-04-22T23:46:07.040 INFO:tasks.workunit.client.1.smithi044.stdout:4/425: creat d5/d31/d24/d38/d6a/f89 x:0 0 0 2024-04-22T23:46:07.040 INFO:tasks.workunit.client.1.smithi044.stdout:0/365: dread d0/d2/f6 [0,4194304] 0 2024-04-22T23:46:07.043 INFO:tasks.workunit.client.1.smithi044.stdout:8/423: symlink db/d14/l9d 0 2024-04-22T23:46:07.044 INFO:tasks.workunit.client.1.smithi044.stdout:8/424: write db/f22 [3637730,19611] 0 2024-04-22T23:46:07.045 INFO:tasks.workunit.client.1.smithi044.stdout:8/425: fsync db/f48 0 2024-04-22T23:46:07.047 INFO:tasks.workunit.client.1.smithi044.stdout:8/426: dread - db/d52/f5e zero size 2024-04-22T23:46:07.048 INFO:tasks.workunit.client.1.smithi044.stdout:1/404: dwrite d1/d8/f18 [4194304,4194304] 0 2024-04-22T23:46:07.054 INFO:tasks.workunit.client.1.smithi044.stdout:0/366: rename d0/d2e/d33/l4e to d0/d2e/d2f/l87 0 2024-04-22T23:46:07.056 INFO:tasks.workunit.client.1.smithi044.stdout:7/394: write dd/d1f/f34 [2891479,114648] 0 2024-04-22T23:46:07.059 INFO:tasks.workunit.client.1.smithi044.stdout:5/395: getdents d15/d16/d1d/d22/d4f 0 2024-04-22T23:46:07.064 INFO:tasks.workunit.client.1.smithi044.stdout:8/427: rmdir db/d6c 39 2024-04-22T23:46:07.064 INFO:tasks.workunit.client.1.smithi044.stdout:8/428: chown db/d14/d72 488020062 1 2024-04-22T23:46:07.066 INFO:tasks.workunit.client.1.smithi044.stdout:1/405: truncate d1/d8/d20/f2f 280840 0 2024-04-22T23:46:07.070 INFO:tasks.workunit.client.1.smithi044.stdout:2/338: sync 2024-04-22T23:46:07.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/325: sync 2024-04-22T23:46:07.071 INFO:tasks.workunit.client.1.smithi044.stdout:3/390: sync 2024-04-22T23:46:07.071 INFO:tasks.workunit.client.1.smithi044.stdout:2/339: fdatasync d8/d19/f37 0 2024-04-22T23:46:07.073 INFO:tasks.workunit.client.1.smithi044.stdout:7/395: mkdir dd/d55/d5e/d89 0 2024-04-22T23:46:07.074 INFO:tasks.workunit.client.1.smithi044.stdout:5/396: write d15/d16/d1d/f26 [8787144,93273] 0 2024-04-22T23:46:07.078 INFO:tasks.workunit.client.1.smithi044.stdout:4/426: creat d5/db/d16/d4d/f8a x:0 0 0 2024-04-22T23:46:07.081 INFO:tasks.workunit.client.1.smithi044.stdout:1/406: creat d1/d8/d20/d47/f8a x:0 0 0 2024-04-22T23:46:07.081 INFO:tasks.workunit.client.1.smithi044.stdout:4/427: dread d5/d31/d44/d20/d22/f37 [0,4194304] 0 2024-04-22T23:46:07.081 INFO:tasks.workunit.client.1.smithi044.stdout:9/256: fdatasync d2/d6/fa 0 2024-04-22T23:46:07.083 INFO:tasks.workunit.client.1.smithi044.stdout:4/428: dread - d5/d42/d73/f88 zero size 2024-04-22T23:46:07.084 INFO:tasks.workunit.client.1.smithi044.stdout:0/367: mknod d0/d2/dd/d3e/d40/d67/d7c/c88 0 2024-04-22T23:46:07.086 INFO:tasks.workunit.client.1.smithi044.stdout:6/326: unlink d5/d30/l5e 0 2024-04-22T23:46:07.088 INFO:tasks.workunit.client.1.smithi044.stdout:6/327: write d5/dd/f16 [4757023,64281] 0 2024-04-22T23:46:07.090 INFO:tasks.workunit.client.1.smithi044.stdout:9/257: dwrite d2/d6/f43 [4194304,4194304] 0 2024-04-22T23:46:07.093 INFO:tasks.workunit.client.1.smithi044.stdout:2/340: mkdir d8/d19/d36/d46/d63/d65/d72 0 2024-04-22T23:46:07.095 INFO:tasks.workunit.client.1.smithi044.stdout:2/341: write d8/d19/d36/d44/f52 [927977,18142] 0 2024-04-22T23:46:07.096 INFO:tasks.workunit.client.1.smithi044.stdout:7/396: unlink dd/d1f/f31 0 2024-04-22T23:46:07.099 INFO:tasks.workunit.client.1.smithi044.stdout:8/429: mkdir db/d14/d9e 0 2024-04-22T23:46:07.110 INFO:tasks.workunit.client.1.smithi044.stdout:0/368: rmdir d0/d2e/d33 39 2024-04-22T23:46:07.112 INFO:tasks.workunit.client.1.smithi044.stdout:4/429: rename d5/d31/d24/d38/d6a/l7b to d5/d31/d44/d20/d22/l8b 0 2024-04-22T23:46:07.116 INFO:tasks.workunit.client.1.smithi044.stdout:4/430: write d5/d31/d24/d69/f6c [64972,81292] 0 2024-04-22T23:46:07.117 INFO:tasks.workunit.client.1.smithi044.stdout:1/407: dwrite d1/d8/d20/d45/f3f [0,4194304] 0 2024-04-22T23:46:07.120 INFO:tasks.workunit.client.1.smithi044.stdout:3/391: mknod d9/d14/d7d/c84 0 2024-04-22T23:46:07.122 INFO:tasks.workunit.client.1.smithi044.stdout:3/392: fsync d9/db/d15/d25/d5b/f40 0 2024-04-22T23:46:07.126 INFO:tasks.workunit.client.1.smithi044.stdout:2/342: creat d8/d19/d36/d46/d54/f73 x:0 0 0 2024-04-22T23:46:07.127 INFO:tasks.workunit.client.1.smithi044.stdout:5/397: mknod d15/d16/c78 0 2024-04-22T23:46:07.128 INFO:tasks.workunit.client.1.smithi044.stdout:2/343: chown d8/d19/d1c/f2f 51 1 2024-04-22T23:46:07.129 INFO:tasks.workunit.client.1.smithi044.stdout:0/369: rename d0/d2/d4b/f72 to d0/d2/dd/d3e/d40/d67/f89 0 2024-04-22T23:46:07.130 INFO:tasks.workunit.client.1.smithi044.stdout:9/258: dwrite d2/f3 [8388608,4194304] 0 2024-04-22T23:46:07.132 INFO:tasks.workunit.client.1.smithi044.stdout:6/328: symlink d5/d3f/d4c/d40/d53/l6f 0 2024-04-22T23:46:07.133 INFO:tasks.workunit.client.1.smithi044.stdout:5/398: write d15/d16/d1d/f3f [1439590,113269] 0 2024-04-22T23:46:07.137 INFO:tasks.workunit.client.1.smithi044.stdout:3/393: creat d9/db/d15/f85 x:0 0 0 2024-04-22T23:46:07.138 INFO:tasks.workunit.client.1.smithi044.stdout:9/259: truncate d2/d6/f53 1149550 0 2024-04-22T23:46:07.139 INFO:tasks.workunit.client.1.smithi044.stdout:7/397: symlink dd/d55/d5e/d89/l8a 0 2024-04-22T23:46:07.140 INFO:tasks.workunit.client.1.smithi044.stdout:3/394: fsync d9/db/d15/d54/f4b 0 2024-04-22T23:46:07.144 INFO:tasks.workunit.client.1.smithi044.stdout:2/344: rmdir d8/d19/d36/d46/d63/d65 39 2024-04-22T23:46:07.148 INFO:tasks.workunit.client.1.smithi044.stdout:2/345: dread d8/d19/f32 [0,4194304] 0 2024-04-22T23:46:07.150 INFO:tasks.workunit.client.1.smithi044.stdout:2/346: dread d8/d27/f28 [0,4194304] 0 2024-04-22T23:46:07.151 INFO:tasks.workunit.client.1.smithi044.stdout:2/347: chown d8/d19/f32 3 1 2024-04-22T23:46:07.158 INFO:tasks.workunit.client.1.smithi044.stdout:6/329: unlink d5/d3f/d4c/d40/f41 0 2024-04-22T23:46:07.166 INFO:tasks.workunit.client.1.smithi044.stdout:5/399: rename d15/f18 to d15/d16/d1d/d22/d2a/f79 0 2024-04-22T23:46:07.167 INFO:tasks.workunit.client.1.smithi044.stdout:5/400: chown d15/d16/d1d/d22/d2a/f79 1493840936 1 2024-04-22T23:46:07.168 INFO:tasks.workunit.client.1.smithi044.stdout:8/430: creat db/d14/d72/f9f x:0 0 0 2024-04-22T23:46:07.169 INFO:tasks.workunit.client.1.smithi044.stdout:8/431: write db/f44 [1054341,30051] 0 2024-04-22T23:46:07.171 INFO:tasks.workunit.client.1.smithi044.stdout:7/398: symlink dd/d1a/l8b 0 2024-04-22T23:46:07.171 INFO:tasks.workunit.client.1.smithi044.stdout:4/431: truncate d5/d42/f48 3990359 0 2024-04-22T23:46:07.172 INFO:tasks.workunit.client.1.smithi044.stdout:8/432: read - db/d14/d24/f59 zero size 2024-04-22T23:46:07.172 INFO:tasks.workunit.client.1.smithi044.stdout:4/432: chown d5/d31/d44 1660445967 1 2024-04-22T23:46:07.177 INFO:tasks.workunit.client.1.smithi044.stdout:3/395: mkdir d9/db/d15/d25/d4c/d86 0 2024-04-22T23:46:07.178 INFO:tasks.workunit.client.1.smithi044.stdout:0/370: mknod d0/d2/c8a 0 2024-04-22T23:46:07.187 INFO:tasks.workunit.client.1.smithi044.stdout:2/348: mkdir d8/d27/d5b/d74 0 2024-04-22T23:46:07.188 INFO:tasks.workunit.client.1.smithi044.stdout:6/330: readlink d5/l7 0 2024-04-22T23:46:07.190 INFO:tasks.workunit.client.1.smithi044.stdout:9/260: symlink d2/d9/l60 0 2024-04-22T23:46:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:06 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:06 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:07.195 INFO:tasks.workunit.client.1.smithi044.stdout:5/401: rename d15/d5d/f5e to d15/d16/d1d/d22/d2a/d3a/f7a 0 2024-04-22T23:46:07.197 INFO:tasks.workunit.client.1.smithi044.stdout:4/433: symlink d5/d52/l8c 0 2024-04-22T23:46:07.199 INFO:tasks.workunit.client.1.smithi044.stdout:4/434: truncate d5/d31/d24/d38/f40 397775 0 2024-04-22T23:46:07.200 INFO:tasks.workunit.client.1.smithi044.stdout:0/371: mkdir d0/d4c/d8b 0 2024-04-22T23:46:07.200 INFO:tasks.workunit.client.1.smithi044.stdout:5/402: dwrite d15/d16/f39 [0,4194304] 0 2024-04-22T23:46:07.204 INFO:tasks.workunit.client.1.smithi044.stdout:1/408: getdents d1/d8/d20/d47 0 2024-04-22T23:46:07.205 INFO:tasks.workunit.client.1.smithi044.stdout:2/349: mknod d8/d19/d1c/c75 0 2024-04-22T23:46:07.209 INFO:tasks.workunit.client.1.smithi044.stdout:6/331: symlink d5/dd/d28/l70 0 2024-04-22T23:46:07.212 INFO:tasks.workunit.client.1.smithi044.stdout:8/433: creat db/d6c/fa0 x:0 0 0 2024-04-22T23:46:07.213 INFO:tasks.workunit.client.1.smithi044.stdout:8/434: readlink db/d52/l9b 0 2024-04-22T23:46:07.213 INFO:tasks.workunit.client.1.smithi044.stdout:6/332: dwrite f0 [0,4194304] 0 2024-04-22T23:46:07.214 INFO:tasks.workunit.client.1.smithi044.stdout:4/435: mkdir d5/d31/d44/d20/d22/d8d 0 2024-04-22T23:46:07.217 INFO:tasks.workunit.client.1.smithi044.stdout:6/333: truncate d5/d3f/d4c/d40/d53/d56/f58 373126 0 2024-04-22T23:46:07.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/409: creat d1/d8/d1c/d7b/f8b x:0 0 0 2024-04-22T23:46:07.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/350: mknod d8/d19/d1c/c76 0 2024-04-22T23:46:07.224 INFO:tasks.workunit.client.1.smithi044.stdout:1/410: dread d1/d8/d1c/f59 [0,4194304] 0 2024-04-22T23:46:07.227 INFO:tasks.workunit.client.1.smithi044.stdout:8/435: creat db/d68/d8b/fa1 x:0 0 0 2024-04-22T23:46:07.229 INFO:tasks.workunit.client.1.smithi044.stdout:6/334: creat d5/d3f/d4c/d40/f71 x:0 0 0 2024-04-22T23:46:07.233 INFO:tasks.workunit.client.1.smithi044.stdout:6/335: dread f0 [0,4194304] 0 2024-04-22T23:46:07.234 INFO:tasks.workunit.client.1.smithi044.stdout:2/351: rename d8/l45 to d8/d19/d36/l77 0 2024-04-22T23:46:07.235 INFO:tasks.workunit.client.1.smithi044.stdout:3/396: getdents d9/db/d15/d2e 0 2024-04-22T23:46:07.235 INFO:tasks.workunit.client.1.smithi044.stdout:2/352: truncate d8/d19/d36/d46/f67 4732193 0 2024-04-22T23:46:07.236 INFO:tasks.workunit.client.1.smithi044.stdout:1/411: fsync d1/d8/d1c/d7b/f66 0 2024-04-22T23:46:07.239 INFO:tasks.workunit.client.1.smithi044.stdout:5/403: creat d15/d16/f7b x:0 0 0 2024-04-22T23:46:07.240 INFO:tasks.workunit.client.1.smithi044.stdout:8/436: creat db/d14/d24/fa2 x:0 0 0 2024-04-22T23:46:07.244 INFO:tasks.workunit.client.1.smithi044.stdout:3/397: symlink d9/db/d15/d25/d67/l87 0 2024-04-22T23:46:07.252 INFO:tasks.workunit.client.1.smithi044.stdout:0/372: dwrite d0/d2/dd/d3e/d40/d43/d7b/f55 [0,4194304] 0 2024-04-22T23:46:07.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/436: creat d5/db/d16/f8e x:0 0 0 2024-04-22T23:46:07.256 INFO:tasks.workunit.client.1.smithi044.stdout:9/261: dwrite d2/dc/f15 [0,4194304] 0 2024-04-22T23:46:07.261 INFO:tasks.workunit.client.1.smithi044.stdout:8/437: read db/d14/f2b [224029,97942] 0 2024-04-22T23:46:07.262 INFO:tasks.workunit.client.1.smithi044.stdout:8/438: chown db/f12 30 1 2024-04-22T23:46:07.272 INFO:tasks.workunit.client.1.smithi044.stdout:8/439: mknod db/d68/ca3 0 2024-04-22T23:46:07.273 INFO:tasks.workunit.client.1.smithi044.stdout:3/398: unlink d9/f34 0 2024-04-22T23:46:07.275 INFO:tasks.workunit.client.1.smithi044.stdout:2/353: link d8/d19/d1c/f2f d8/d19/d36/d46/d63/d65/d6c/f78 0 2024-04-22T23:46:07.276 INFO:tasks.workunit.client.1.smithi044.stdout:1/412: creat d1/d8/d1c/f8c x:0 0 0 2024-04-22T23:46:07.276 INFO:tasks.workunit.client.1.smithi044.stdout:2/354: chown d8/d19/d36/d46/d54/l68 34122562 1 2024-04-22T23:46:07.277 INFO:tasks.workunit.client.1.smithi044.stdout:0/373: symlink d0/l8c 0 2024-04-22T23:46:07.277 INFO:tasks.workunit.client.1.smithi044.stdout:5/404: link d15/d16/d1d/d22/d2a/d3a/d43/c44 d15/d16/d1d/d22/d4f/c7c 0 2024-04-22T23:46:07.279 INFO:tasks.workunit.client.1.smithi044.stdout:6/336: getdents d5/d3f/d4c/d4f 0 2024-04-22T23:46:07.281 INFO:tasks.workunit.client.1.smithi044.stdout:3/399: rmdir d9/d14 39 2024-04-22T23:46:07.282 INFO:tasks.workunit.client.1.smithi044.stdout:3/400: chown d9/d12 22 1 2024-04-22T23:46:07.283 INFO:tasks.workunit.client.1.smithi044.stdout:4/437: creat d5/db/f8f x:0 0 0 2024-04-22T23:46:07.287 INFO:tasks.workunit.client.1.smithi044.stdout:2/355: dwrite d8/d27/f4b [0,4194304] 0 2024-04-22T23:46:07.288 INFO:tasks.workunit.client.1.smithi044.stdout:9/262: symlink d2/d5/l61 0 2024-04-22T23:46:07.290 INFO:tasks.workunit.client.1.smithi044.stdout:5/405: symlink d15/d5d/d64/l7d 0 2024-04-22T23:46:07.293 INFO:tasks.workunit.client.1.smithi044.stdout:5/406: write d15/d16/f1a [823253,46391] 0 2024-04-22T23:46:07.296 INFO:tasks.workunit.client.1.smithi044.stdout:4/438: dwrite d5/db/d34/f59 [0,4194304] 0 2024-04-22T23:46:07.308 INFO:tasks.workunit.client.1.smithi044.stdout:8/440: link db/d14/d24/l64 db/d52/d9c/la4 0 2024-04-22T23:46:07.317 INFO:tasks.workunit.client.1.smithi044.stdout:9/263: dread d2/f3 [4194304,4194304] 0 2024-04-22T23:46:07.319 INFO:tasks.workunit.client.1.smithi044.stdout:4/439: rename d5/d42/d73 to d5/d31/d24/d69/d90 0 2024-04-22T23:46:07.322 INFO:tasks.workunit.client.1.smithi044.stdout:2/356: creat d8/d27/d5b/d74/f79 x:0 0 0 2024-04-22T23:46:07.322 INFO:tasks.workunit.client.1.smithi044.stdout:0/374: write d0/d2/dd/d3e/f3f [642004,24762] 0 2024-04-22T23:46:07.325 INFO:tasks.workunit.client.1.smithi044.stdout:2/357: chown d8/d19/d1c/f24 0 1 2024-04-22T23:46:07.326 INFO:tasks.workunit.client.1.smithi044.stdout:6/337: creat d5/d3f/d4c/f72 x:0 0 0 2024-04-22T23:46:07.328 INFO:tasks.workunit.client.1.smithi044.stdout:5/407: write fe [686190,48731] 0 2024-04-22T23:46:07.328 INFO:tasks.workunit.client.1.smithi044.stdout:1/413: dwrite d1/d8/d20/f2f [0,4194304] 0 2024-04-22T23:46:07.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/414: chown d1/d8/d1c/d23/d3e/c63 9 1 2024-04-22T23:46:07.331 INFO:tasks.workunit.client.1.smithi044.stdout:9/264: symlink d2/dc/d1d/d3f/l62 0 2024-04-22T23:46:07.332 INFO:tasks.workunit.client.1.smithi044.stdout:1/415: dread - d1/d8/d20/d47/f8a zero size 2024-04-22T23:46:07.336 INFO:tasks.workunit.client.1.smithi044.stdout:2/358: dread f4 [0,4194304] 0 2024-04-22T23:46:07.339 INFO:tasks.workunit.client.1.smithi044.stdout:2/359: chown d8/d19/d1c/l3a 22261713 1 2024-04-22T23:46:07.342 INFO:tasks.workunit.client.1.smithi044.stdout:8/441: dwrite db/d14/f46 [0,4194304] 0 2024-04-22T23:46:07.343 INFO:tasks.workunit.client.1.smithi044.stdout:4/440: dread d5/d31/d44/d20/f4b [0,4194304] 0 2024-04-22T23:46:07.344 INFO:tasks.workunit.client.1.smithi044.stdout:6/338: unlink d5/dd/d28/c5f 0 2024-04-22T23:46:07.344 INFO:tasks.workunit.client.1.smithi044.stdout:4/441: readlink d5/d31/d44/d20/d22/l57 0 2024-04-22T23:46:07.346 INFO:tasks.workunit.client.1.smithi044.stdout:3/401: getdents d9/db/d22/d38 0 2024-04-22T23:46:07.346 INFO:tasks.workunit.client.1.smithi044.stdout:3/402: stat d9/l7b 0 2024-04-22T23:46:07.348 INFO:tasks.workunit.client.1.smithi044.stdout:3/403: write d9/db/d15/d54/f4b [216512,34595] 0 2024-04-22T23:46:07.351 INFO:tasks.workunit.client.1.smithi044.stdout:9/265: creat d2/dc/d1d/d3f/f63 x:0 0 0 2024-04-22T23:46:07.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/375: rmdir d0/d2e/d33 39 2024-04-22T23:46:07.364 INFO:tasks.workunit.client.1.smithi044.stdout:6/339: rmdir d5/dd 39 2024-04-22T23:46:07.366 INFO:tasks.workunit.client.1.smithi044.stdout:9/266: rmdir d2/dc/d51 39 2024-04-22T23:46:07.367 INFO:tasks.workunit.client.1.smithi044.stdout:3/404: dread d9/db/d22/f56 [0,4194304] 0 2024-04-22T23:46:07.369 INFO:tasks.workunit.client.1.smithi044.stdout:0/376: creat d0/d2/d4b/f8d x:0 0 0 2024-04-22T23:46:07.375 INFO:tasks.workunit.client.1.smithi044.stdout:5/408: truncate d15/d16/f1a 362272 0 2024-04-22T23:46:07.377 INFO:tasks.workunit.client.1.smithi044.stdout:6/340: mkdir d5/d30/d73 0 2024-04-22T23:46:07.378 INFO:tasks.workunit.client.1.smithi044.stdout:6/341: stat d5/d3f/d4c/d40/f5b 0 2024-04-22T23:46:07.383 INFO:tasks.workunit.client.1.smithi044.stdout:1/416: link d1/d8/c32 d1/d8/d1c/d7b/d80/c8d 0 2024-04-22T23:46:07.388 INFO:tasks.workunit.client.1.smithi044.stdout:3/405: dread d9/db/d33/f7c [0,4194304] 0 2024-04-22T23:46:07.388 INFO:tasks.workunit.client.1.smithi044.stdout:2/360: dwrite d8/d19/f1f [0,4194304] 0 2024-04-22T23:46:07.390 INFO:tasks.workunit.client.1.smithi044.stdout:3/406: chown d9/db/d15/d25/f6d 11334288 1 2024-04-22T23:46:07.391 INFO:tasks.workunit.client.1.smithi044.stdout:0/377: dread d0/d2e/d2f/f51 [0,4194304] 0 2024-04-22T23:46:07.397 INFO:tasks.workunit.client.1.smithi044.stdout:8/442: getdents db/d14/d72 0 2024-04-22T23:46:07.403 INFO:tasks.workunit.client.1.smithi044.stdout:1/417: creat d1/d8/d1c/d74/f8e x:0 0 0 2024-04-22T23:46:07.403 INFO:tasks.workunit.client.1.smithi044.stdout:8/443: chown db/d52/l57 101452521 1 2024-04-22T23:46:07.405 INFO:tasks.workunit.client.1.smithi044.stdout:2/361: creat d8/d19/d36/d46/d54/f7a x:0 0 0 2024-04-22T23:46:07.405 INFO:tasks.workunit.client.1.smithi044.stdout:6/342: dwrite d5/dd/d67/f24 [0,4194304] 0 2024-04-22T23:46:07.405 INFO:tasks.workunit.client.1.smithi044.stdout:2/362: chown f3 3803313 1 2024-04-22T23:46:07.408 INFO:tasks.workunit.client.1.smithi044.stdout:3/407: dwrite d9/db/d15/f63 [0,4194304] 0 2024-04-22T23:46:07.409 INFO:tasks.workunit.client.1.smithi044.stdout:0/378: dwrite d0/d4c/f75 [0,4194304] 0 2024-04-22T23:46:07.409 INFO:tasks.workunit.client.1.smithi044.stdout:9/267: link d2/dc/d1d/l58 d2/dc/d1d/d34/l64 0 2024-04-22T23:46:07.413 INFO:tasks.workunit.client.1.smithi044.stdout:9/268: dread - d2/d9/f5a zero size 2024-04-22T23:46:07.413 INFO:tasks.workunit.client.1.smithi044.stdout:0/379: write d0/d2/dd/d3e/d40/f84 [405236,119454] 0 2024-04-22T23:46:07.416 INFO:tasks.workunit.client.1.smithi044.stdout:5/409: rmdir d15/d16/d1d/d22/d40 0 2024-04-22T23:46:07.417 INFO:tasks.workunit.client.1.smithi044.stdout:6/343: mkdir d5/d3f/d4c/d40/d74 0 2024-04-22T23:46:07.417 INFO:tasks.workunit.client.1.smithi044.stdout:5/410: stat d15/d16/d2c/d38/f50 0 2024-04-22T23:46:07.421 INFO:tasks.workunit.client.1.smithi044.stdout:9/269: dread d2/dc/d1d/d3f/f48 [0,4194304] 0 2024-04-22T23:46:07.423 INFO:tasks.workunit.client.1.smithi044.stdout:2/363: creat d8/d27/d5b/f7b x:0 0 0 2024-04-22T23:46:07.424 INFO:tasks.workunit.client.1.smithi044.stdout:3/408: symlink d9/db/d33/d5d/d70/l88 0 2024-04-22T23:46:07.429 INFO:tasks.workunit.client.1.smithi044.stdout:3/409: dread d9/db/d15/d25/d4c/f4d [0,4194304] 0 2024-04-22T23:46:07.431 INFO:tasks.workunit.client.1.smithi044.stdout:6/344: mkdir d5/dd/d28/d75 0 2024-04-22T23:46:07.435 INFO:tasks.workunit.client.1.smithi044.stdout:5/411: creat d15/d16/d1d/d22/d2a/d3a/d43/f7e x:0 0 0 2024-04-22T23:46:07.436 INFO:tasks.workunit.client.1.smithi044.stdout:8/444: write db/d14/d24/f3b [277890,28949] 0 2024-04-22T23:46:07.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/364: creat d8/d19/d5d/f7c x:0 0 0 2024-04-22T23:46:07.440 INFO:tasks.workunit.client.1.smithi044.stdout:2/365: chown d8/d19/d1c/l3f 4192528 1 2024-04-22T23:46:07.440 INFO:tasks.workunit.client.1.smithi044.stdout:2/366: chown d8/f26 1590 1 2024-04-22T23:46:07.446 INFO:tasks.workunit.client.1.smithi044.stdout:5/412: mkdir d15/d5d/d64/d6f/d7f 0 2024-04-22T23:46:07.447 INFO:tasks.workunit.client.1.smithi044.stdout:8/445: creat db/d6c/fa5 x:0 0 0 2024-04-22T23:46:07.448 INFO:tasks.workunit.client.1.smithi044.stdout:9/270: dwrite d2/dc/f18 [0,4194304] 0 2024-04-22T23:46:07.448 INFO:tasks.workunit.client.1.smithi044.stdout:5/413: fdatasync d15/d16/d2c/f5a 0 2024-04-22T23:46:07.450 INFO:tasks.workunit.client.1.smithi044.stdout:8/446: write db/d14/d24/f5c [3699808,78401] 0 2024-04-22T23:46:07.450 INFO:tasks.workunit.client.1.smithi044.stdout:1/418: link d1/d8/d20/d45/l64 d1/d8/d20/d47/d4a/d60/l8f 0 2024-04-22T23:46:07.453 INFO:tasks.workunit.client.1.smithi044.stdout:0/380: rename d0/d2/dd/f23 to d0/d2/f8e 0 2024-04-22T23:46:07.454 INFO:tasks.workunit.client.1.smithi044.stdout:0/381: readlink d0/d2e/l6f 0 2024-04-22T23:46:07.456 INFO:tasks.workunit.client.1.smithi044.stdout:3/410: mknod d9/db/d15/d25/d4c/d86/c89 0 2024-04-22T23:46:07.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/447: dwrite db/d14/d24/fa2 [0,4194304] 0 2024-04-22T23:46:07.463 INFO:tasks.workunit.client.1.smithi044.stdout:8/448: dread db/d14/d24/f3b [0,4194304] 0 2024-04-22T23:46:07.466 INFO:tasks.workunit.client.1.smithi044.stdout:8/449: read db/f48 [981094,102060] 0 2024-04-22T23:46:07.468 INFO:tasks.workunit.client.1.smithi044.stdout:8/450: truncate db/d14/f97 440753 0 2024-04-22T23:46:07.470 INFO:tasks.workunit.client.1.smithi044.stdout:0/382: rename d0/d4c/d59/d5a/f63 to d0/d4c/f8f 0 2024-04-22T23:46:07.471 INFO:tasks.workunit.client.1.smithi044.stdout:3/411: symlink d9/db/d15/d25/d4c/d86/l8a 0 2024-04-22T23:46:07.473 INFO:tasks.workunit.client.1.smithi044.stdout:9/271: creat d2/dc/d1d/d34/d38/d5c/f65 x:0 0 0 2024-04-22T23:46:07.474 INFO:tasks.workunit.client.1.smithi044.stdout:3/412: truncate d9/db/d15/d25/d5b/f7e 878493 0 2024-04-22T23:46:07.476 INFO:tasks.workunit.client.1.smithi044.stdout:1/419: unlink d1/d8/d1c/d23/d3e/f58 0 2024-04-22T23:46:07.481 INFO:tasks.workunit.client.1.smithi044.stdout:1/420: dread d1/d8/d20/d45/f3f [0,4194304] 0 2024-04-22T23:46:07.483 INFO:tasks.workunit.client.1.smithi044.stdout:0/383: rename d0/d2e/d2f/d61/d6a to d0/d2/dd/d3e/d40/d43/d7b/d90 0 2024-04-22T23:46:07.483 INFO:tasks.workunit.client.1.smithi044.stdout:0/384: readlink d0/l1c 0 2024-04-22T23:46:07.486 INFO:tasks.workunit.client.1.smithi044.stdout:3/413: creat d9/db/d15/d25/d5b/d52/f8b x:0 0 0 2024-04-22T23:46:07.489 INFO:tasks.workunit.client.1.smithi044.stdout:6/345: getdents d5/d3f/d4c/d23 0 2024-04-22T23:46:07.491 INFO:tasks.workunit.client.1.smithi044.stdout:6/346: dread - d5/d3f/d4c/f72 zero size 2024-04-22T23:46:07.492 INFO:tasks.workunit.client.1.smithi044.stdout:1/421: creat d1/d8/d1c/d23/d3e/f90 x:0 0 0 2024-04-22T23:46:07.530 INFO:tasks.workunit.client.1.smithi044.stdout:7/399: sync 2024-04-22T23:46:07.531 INFO:tasks.workunit.client.1.smithi044.stdout:0/385: rename d0/l1c to d0/d2/d4b/d6b/l91 0 2024-04-22T23:46:07.531 INFO:tasks.workunit.client.1.smithi044.stdout:3/414: dwrite d9/db/d33/d5d/f66 [0,4194304] 0 2024-04-22T23:46:07.536 INFO:tasks.workunit.client.1.smithi044.stdout:9/272: truncate d2/d6/f43 4653320 0 2024-04-22T23:46:07.539 INFO:tasks.workunit.client.1.smithi044.stdout:3/415: dread - d9/db/d15/d25/d5b/d52/f8b zero size 2024-04-22T23:46:07.540 INFO:tasks.workunit.client.1.smithi044.stdout:6/347: mknod d5/d30/d73/c76 0 2024-04-22T23:46:07.540 INFO:tasks.workunit.client.1.smithi044.stdout:7/400: mkdir dd/d79/d8c 0 2024-04-22T23:46:07.544 INFO:tasks.workunit.client.1.smithi044.stdout:3/416: fsync d9/d12/f78 0 2024-04-22T23:46:07.545 INFO:tasks.workunit.client.1.smithi044.stdout:8/451: dwrite db/d14/d24/f26 [4194304,4194304] 0 2024-04-22T23:46:07.549 INFO:tasks.workunit.client.1.smithi044.stdout:1/422: dwrite d1/d8/d1c/d7b/f8b [0,4194304] 0 2024-04-22T23:46:07.549 INFO:tasks.workunit.client.1.smithi044.stdout:7/401: write dd/d1f/d24/f13 [2168683,83211] 0 2024-04-22T23:46:07.554 INFO:tasks.workunit.client.1.smithi044.stdout:9/273: dread d2/dc/f27 [0,4194304] 0 2024-04-22T23:46:07.555 INFO:tasks.workunit.client.1.smithi044.stdout:8/452: dread db/d14/f42 [0,4194304] 0 2024-04-22T23:46:07.559 INFO:tasks.workunit.client.1.smithi044.stdout:3/417: dread d9/db/d33/f7c [0,4194304] 0 2024-04-22T23:46:07.561 INFO:tasks.workunit.client.1.smithi044.stdout:0/386: rename d0/l9 to d0/d2/dd/d3e/d40/d43/d7b/l92 0 2024-04-22T23:46:07.566 INFO:tasks.workunit.client.1.smithi044.stdout:1/423: symlink d1/d8/d20/d45/d5d/l91 0 2024-04-22T23:46:07.566 INFO:tasks.workunit.client.1.smithi044.stdout:9/274: dwrite d2/dc/f18 [4194304,4194304] 0 2024-04-22T23:46:07.567 INFO:tasks.workunit.client.1.smithi044.stdout:8/453: mknod db/d14/d72/ca6 0 2024-04-22T23:46:07.573 INFO:tasks.workunit.client.1.smithi044.stdout:7/402: mknod dd/d79/d8c/c8d 0 2024-04-22T23:46:07.575 INFO:tasks.workunit.client.1.smithi044.stdout:0/387: unlink d0/d2e/d2f/d61/f78 0 2024-04-22T23:46:07.578 INFO:tasks.workunit.client.1.smithi044.stdout:3/418: rename d9/l51 to d9/db/d33/d5d/l8c 0 2024-04-22T23:46:07.584 INFO:tasks.workunit.client.1.smithi044.stdout:1/424: mkdir d1/d8/d20/d47/d92 0 2024-04-22T23:46:07.588 INFO:tasks.workunit.client.1.smithi044.stdout:7/403: creat dd/d1a/f8e x:0 0 0 2024-04-22T23:46:07.589 INFO:tasks.workunit.client.1.smithi044.stdout:6/348: write d5/d3f/f15 [554796,129863] 0 2024-04-22T23:46:07.590 INFO:tasks.workunit.client.1.smithi044.stdout:7/404: fsync dd/d1f/d24/f13 0 2024-04-22T23:46:07.591 INFO:tasks.workunit.client.1.smithi044.stdout:0/388: unlink d0/d4c/f71 0 2024-04-22T23:46:07.595 INFO:tasks.workunit.client.1.smithi044.stdout:1/425: creat d1/d8/d20/d45/d75/f93 x:0 0 0 2024-04-22T23:46:07.602 INFO:tasks.workunit.client.1.smithi044.stdout:1/426: dwrite d1/d8/d1c/f5b [4194304,4194304] 0 2024-04-22T23:46:07.603 INFO:tasks.workunit.client.1.smithi044.stdout:1/427: dread - d1/d8/d1c/d23/d3e/f7e zero size 2024-04-22T23:46:07.605 INFO:tasks.workunit.client.1.smithi044.stdout:7/405: creat dd/d1a/d3c/f8f x:0 0 0 2024-04-22T23:46:07.607 INFO:tasks.workunit.client.1.smithi044.stdout:0/389: chown d0/d2/dd/d3e/d40/d43/d7b/l83 923 1 2024-04-22T23:46:07.609 INFO:tasks.workunit.client.1.smithi044.stdout:9/275: creat d2/dc/f66 x:0 0 0 2024-04-22T23:46:07.610 INFO:tasks.workunit.client.1.smithi044.stdout:9/276: chown d2/dc/d1d/d34/d41 0 1 2024-04-22T23:46:07.613 INFO:tasks.workunit.client.1.smithi044.stdout:6/349: creat d5/d3f/d4c/d23/d3d/d64/f77 x:0 0 0 2024-04-22T23:46:07.618 INFO:tasks.workunit.client.1.smithi044.stdout:1/428: creat d1/d8/d1c/d23/d48/f94 x:0 0 0 2024-04-22T23:46:07.619 INFO:tasks.workunit.client.1.smithi044.stdout:7/406: creat dd/d1a/f90 x:0 0 0 2024-04-22T23:46:07.619 INFO:tasks.workunit.client.1.smithi044.stdout:1/429: write d1/d8/d20/d45/d75/f93 [192735,64000] 0 2024-04-22T23:46:07.619 INFO:tasks.workunit.client.1.smithi044.stdout:1/430: chown d1/d14/l30 65372 1 2024-04-22T23:46:07.620 INFO:tasks.workunit.client.1.smithi044.stdout:1/431: chown d1/d8/d1c/d23 7734407 1 2024-04-22T23:46:07.622 INFO:tasks.workunit.client.1.smithi044.stdout:0/390: write d0/d2/dd/d3e/d40/d43/f5f [981116,44281] 0 2024-04-22T23:46:07.628 INFO:tasks.workunit.client.1.smithi044.stdout:3/419: dwrite d9/db/f10 [4194304,4194304] 0 2024-04-22T23:46:07.630 INFO:tasks.workunit.client.1.smithi044.stdout:1/432: mkdir d1/d14/d95 0 2024-04-22T23:46:07.631 INFO:tasks.workunit.client.1.smithi044.stdout:3/420: truncate d9/db/d15/f18 8485023 0 2024-04-22T23:46:07.642 INFO:tasks.workunit.client.1.smithi044.stdout:7/407: dread dd/d27/f37 [0,4194304] 0 2024-04-22T23:46:07.644 INFO:tasks.workunit.client.1.smithi044.stdout:7/408: write dd/d1a/d3c/f49 [2523743,35304] 0 2024-04-22T23:46:07.645 INFO:tasks.workunit.client.1.smithi044.stdout:9/277: getdents d2/dc/d1d/d34 0 2024-04-22T23:46:07.648 INFO:tasks.workunit.client.1.smithi044.stdout:1/433: creat d1/d14/d95/f96 x:0 0 0 2024-04-22T23:46:07.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/278: truncate d2/f4 2171533 0 2024-04-22T23:46:07.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/279: stat d2/d6/l21 0 2024-04-22T23:46:07.665 INFO:tasks.workunit.client.1.smithi044.stdout:9/280: mkdir d2/dc/d1d/d34/d67 0 2024-04-22T23:46:07.666 INFO:tasks.workunit.client.1.smithi044.stdout:7/409: link dd/l2d dd/d1a/d3c/l91 0 2024-04-22T23:46:07.667 INFO:tasks.workunit.client.1.smithi044.stdout:7/410: truncate dd/d1a/d3c/f8f 816678 0 2024-04-22T23:46:07.668 INFO:tasks.workunit.client.1.smithi044.stdout:1/434: creat d1/d8/d20/d45/f97 x:0 0 0 2024-04-22T23:46:07.670 INFO:tasks.workunit.client.1.smithi044.stdout:6/350: write d5/dd/f1e [511732,112312] 0 2024-04-22T23:46:07.673 INFO:tasks.workunit.client.1.smithi044.stdout:1/435: mknod d1/d14/c98 0 2024-04-22T23:46:07.674 INFO:tasks.workunit.client.1.smithi044.stdout:6/351: creat d5/d3f/d4c/d40/f78 x:0 0 0 2024-04-22T23:46:07.676 INFO:tasks.workunit.client.1.smithi044.stdout:7/411: creat dd/d55/d5e/d60/d75/f92 x:0 0 0 2024-04-22T23:46:07.679 INFO:tasks.workunit.client.1.smithi044.stdout:6/352: mkdir d5/d3f/d50/d79 0 2024-04-22T23:46:07.680 INFO:tasks.workunit.client.1.smithi044.stdout:6/353: chown d5/d3f/d4c/f72 1010021 1 2024-04-22T23:46:07.680 INFO:tasks.workunit.client.1.smithi044.stdout:7/412: symlink dd/d55/d5e/d81/l93 0 2024-04-22T23:46:07.681 INFO:tasks.workunit.client.1.smithi044.stdout:7/413: read - dd/f54 zero size 2024-04-22T23:46:07.682 INFO:tasks.workunit.client.1.smithi044.stdout:6/354: write d5/dd/f1e [581007,70735] 0 2024-04-22T23:46:07.684 INFO:tasks.workunit.client.1.smithi044.stdout:6/355: chown d5/d3f/d4c/d40/d53/d56/f58 583 1 2024-04-22T23:46:07.685 INFO:tasks.workunit.client.1.smithi044.stdout:7/414: fdatasync f5 0 2024-04-22T23:46:07.688 INFO:tasks.workunit.client.1.smithi044.stdout:3/421: write d9/db/d22/d38/f41 [487239,60964] 0 2024-04-22T23:46:07.690 INFO:tasks.workunit.client.1.smithi044.stdout:3/422: read d9/db/f10 [7333707,2335] 0 2024-04-22T23:46:07.693 INFO:tasks.workunit.client.1.smithi044.stdout:0/391: truncate d0/d2/dd/f10 2369461 0 2024-04-22T23:46:07.693 INFO:tasks.workunit.client.1.smithi044.stdout:1/436: dread d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:07.694 INFO:tasks.workunit.client.1.smithi044.stdout:6/356: dwrite d5/dd/f6a [0,4194304] 0 2024-04-22T23:46:07.696 INFO:tasks.workunit.client.1.smithi044.stdout:7/415: rename dd/d55/d5e/d87 to dd/d55/d5e/d60/d68/d94 0 2024-04-22T23:46:07.700 INFO:tasks.workunit.client.1.smithi044.stdout:6/357: dread d5/dd/f1e [0,4194304] 0 2024-04-22T23:46:07.704 INFO:tasks.workunit.client.1.smithi044.stdout:9/281: write d2/d6/fa [28100,77147] 0 2024-04-22T23:46:07.712 INFO:tasks.workunit.client.1.smithi044.stdout:4/442: sync 2024-04-22T23:46:07.713 INFO:tasks.workunit.client.1.smithi044.stdout:1/437: mkdir d1/d8/d1c/d23/d48/d99 0 2024-04-22T23:46:07.714 INFO:tasks.workunit.client.1.smithi044.stdout:1/438: read - d1/d8/d1c/d74/f8e zero size 2024-04-22T23:46:07.714 INFO:tasks.workunit.client.1.smithi044.stdout:4/443: chown d5/d31/d24/d38/d6a/d75/f7f 13912 1 2024-04-22T23:46:07.716 INFO:tasks.workunit.client.1.smithi044.stdout:1/439: fdatasync d1/d8/d20/f2f 0 2024-04-22T23:46:07.716 INFO:tasks.workunit.client.1.smithi044.stdout:7/416: symlink dd/d55/d5e/d60/l95 0 2024-04-22T23:46:07.720 INFO:tasks.workunit.client.1.smithi044.stdout:6/358: symlink d5/d30/d73/l7a 0 2024-04-22T23:46:07.727 INFO:tasks.workunit.client.1.smithi044.stdout:6/359: dwrite d5/dd/f6a [0,4194304] 0 2024-04-22T23:46:07.737 INFO:tasks.workunit.client.1.smithi044.stdout:4/444: creat d5/d31/d24/d38/f91 x:0 0 0 2024-04-22T23:46:07.742 INFO:tasks.workunit.client.1.smithi044.stdout:3/423: link d9/f2a d9/db/d22/d38/d74/f8d 0 2024-04-22T23:46:07.743 INFO:tasks.workunit.client.1.smithi044.stdout:9/282: symlink d2/dc/d1d/d34/d67/l68 0 2024-04-22T23:46:07.744 INFO:tasks.workunit.client.1.smithi044.stdout:7/417: creat dd/d1a/d61/f96 x:0 0 0 2024-04-22T23:46:07.746 INFO:tasks.workunit.client.1.smithi044.stdout:1/440: creat d1/d8/d1c/d7b/d80/d4b/f9a x:0 0 0 2024-04-22T23:46:07.749 INFO:tasks.workunit.client.1.smithi044.stdout:0/392: dwrite d0/d2/f8e [0,4194304] 0 2024-04-22T23:46:07.751 INFO:tasks.workunit.client.1.smithi044.stdout:3/424: symlink d9/db/d15/d54/d3c/l8e 0 2024-04-22T23:46:07.758 INFO:tasks.workunit.client.1.smithi044.stdout:9/283: mkdir d2/dc/d1d/d34/d38/d5c/d69 0 2024-04-22T23:46:07.761 INFO:tasks.workunit.client.1.smithi044.stdout:0/393: unlink d0/d2/f25 0 2024-04-22T23:46:07.763 INFO:tasks.workunit.client.1.smithi044.stdout:7/418: truncate dd/d1f/d24/f19 3307874 0 2024-04-22T23:46:07.764 INFO:tasks.workunit.client.1.smithi044.stdout:7/419: fsync dd/d1a/f7e 0 2024-04-22T23:46:07.767 INFO:tasks.workunit.client.1.smithi044.stdout:3/425: write d9/db/d22/f56 [5963721,69942] 0 2024-04-22T23:46:07.768 INFO:tasks.workunit.client.1.smithi044.stdout:4/445: getdents d5/d31/d44/d20/d22 0 2024-04-22T23:46:07.772 INFO:tasks.workunit.client.1.smithi044.stdout:1/441: symlink d1/d8/d20/d47/d4a/d60/l9b 0 2024-04-22T23:46:07.779 INFO:tasks.workunit.client.1.smithi044.stdout:3/426: mkdir d9/db/d15/d2e/d48/d8f 0 2024-04-22T23:46:07.780 INFO:tasks.workunit.client.1.smithi044.stdout:7/420: dread dd/d1f/d24/f28 [0,4194304] 0 2024-04-22T23:46:07.782 INFO:tasks.workunit.client.1.smithi044.stdout:1/442: mkdir d1/d8/d1c/d7b/d80/d4b/d9c 0 2024-04-22T23:46:07.784 INFO:tasks.workunit.client.1.smithi044.stdout:1/443: truncate d1/d8/d1c/d23/d3e/f90 73689 0 2024-04-22T23:46:07.785 INFO:tasks.workunit.client.1.smithi044.stdout:9/284: truncate d2/d5/f5b 250425 0 2024-04-22T23:46:07.789 INFO:tasks.workunit.client.1.smithi044.stdout:0/394: creat d0/d2e/d2f/d77/f93 x:0 0 0 2024-04-22T23:46:07.793 INFO:tasks.workunit.client.1.smithi044.stdout:7/421: rename dd/d55/d5e/d81 to dd/d1a/d3c/d97 0 2024-04-22T23:46:07.796 INFO:tasks.workunit.client.1.smithi044.stdout:1/444: rename d1/d8/d34/f78 to d1/d8/d34/f9d 0 2024-04-22T23:46:07.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/395: creat d0/d2e/d2f/d61/f94 x:0 0 0 2024-04-22T23:46:07.800 INFO:tasks.workunit.client.1.smithi044.stdout:0/396: write d0/d2e/f53 [919196,34621] 0 2024-04-22T23:46:07.800 INFO:tasks.workunit.client.1.smithi044.stdout:3/427: fdatasync d9/db/f1a 0 2024-04-22T23:46:07.806 INFO:tasks.workunit.client.1.smithi044.stdout:9/285: rename d2/dc/d1d/d3f/f45 to d2/d9/f6a 0 2024-04-22T23:46:07.809 INFO:tasks.workunit.client.1.smithi044.stdout:1/445: mkdir d1/d8/d20/d9e 0 2024-04-22T23:46:07.812 INFO:tasks.workunit.client.1.smithi044.stdout:0/397: symlink d0/d2/dd/d3e/d40/d43/l95 0 2024-04-22T23:46:07.816 INFO:tasks.workunit.client.1.smithi044.stdout:3/428: creat d9/db/d15/d54/f90 x:0 0 0 2024-04-22T23:46:07.817 INFO:tasks.workunit.client.1.smithi044.stdout:7/422: mkdir dd/d98 0 2024-04-22T23:46:07.822 INFO:tasks.workunit.client.1.smithi044.stdout:9/286: rmdir d2/dc/d1d/d34/d38 39 2024-04-22T23:46:07.823 INFO:tasks.workunit.client.1.smithi044.stdout:1/446: fsync d1/d8/f26 0 2024-04-22T23:46:07.825 INFO:tasks.workunit.client.1.smithi044.stdout:3/429: creat d9/db/d15/d25/d67/f91 x:0 0 0 2024-04-22T23:46:07.826 INFO:tasks.workunit.client.1.smithi044.stdout:3/430: chown d9/db/d15/d25/d5b/f40 5 1 2024-04-22T23:46:07.827 INFO:tasks.workunit.client.1.smithi044.stdout:0/398: mkdir d0/d2/dd/d3e/d40/d43/d79/d96 0 2024-04-22T23:46:07.830 INFO:tasks.workunit.client.1.smithi044.stdout:3/431: unlink d9/db/d15/d25/d4c/l82 0 2024-04-22T23:46:07.832 INFO:tasks.workunit.client.1.smithi044.stdout:1/447: mknod d1/d8/d1c/d7b/d80/d4b/d9c/c9f 0 2024-04-22T23:46:07.832 INFO:tasks.workunit.client.1.smithi044.stdout:3/432: dread d9/db/d15/f21 [0,4194304] 0 2024-04-22T23:46:07.833 INFO:tasks.workunit.client.1.smithi044.stdout:0/399: mknod d0/d4c/d59/d5a/c97 0 2024-04-22T23:46:07.833 INFO:tasks.workunit.client.1.smithi044.stdout:3/433: dread - d9/db/d15/d54/d3c/f81 zero size 2024-04-22T23:46:07.834 INFO:tasks.workunit.client.1.smithi044.stdout:3/434: chown d9/db/d33/d5d/d64 1874 1 2024-04-22T23:46:07.834 INFO:tasks.workunit.client.1.smithi044.stdout:0/400: chown d0/d2/dd/d3e/d40/d43/d7b/l83 9 1 2024-04-22T23:46:07.835 INFO:tasks.workunit.client.1.smithi044.stdout:7/423: rename dd/d55/d71 to dd/d99 0 2024-04-22T23:46:07.837 INFO:tasks.workunit.client.1.smithi044.stdout:0/401: chown d0/d2/dd/d3e/l7e 23405 1 2024-04-22T23:46:07.838 INFO:tasks.workunit.client.1.smithi044.stdout:1/448: truncate d1/d8/d1c/d7b/f66 7147251 0 2024-04-22T23:46:07.842 INFO:tasks.workunit.client.1.smithi044.stdout:0/402: creat d0/d2/dd/d3e/d40/f98 x:0 0 0 2024-04-22T23:46:07.843 INFO:tasks.workunit.client.1.smithi044.stdout:0/403: stat d0/d2/f49 0 2024-04-22T23:46:07.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/435: dwrite d9/db/d15/d54/f90 [0,4194304] 0 2024-04-22T23:46:07.873 INFO:tasks.workunit.client.1.smithi044.stdout:2/367: sync 2024-04-22T23:46:07.873 INFO:tasks.workunit.client.1.smithi044.stdout:0/404: truncate d0/d4c/f52 245513 0 2024-04-22T23:46:07.875 INFO:tasks.workunit.client.1.smithi044.stdout:3/436: creat d9/d12/f92 x:0 0 0 2024-04-22T23:46:07.876 INFO:tasks.workunit.client.1.smithi044.stdout:3/437: chown d9/d12 1940 1 2024-04-22T23:46:07.876 INFO:tasks.workunit.client.1.smithi044.stdout:0/405: write d0/d2e/d2f/f65 [224374,82403] 0 2024-04-22T23:46:07.879 INFO:tasks.workunit.client.1.smithi044.stdout:3/438: fdatasync d9/db/d15/d54/d3c/f6c 0 2024-04-22T23:46:07.881 INFO:tasks.workunit.client.1.smithi044.stdout:1/449: creat d1/d8/fa0 x:0 0 0 2024-04-22T23:46:07.881 INFO:tasks.workunit.client.1.smithi044.stdout:7/424: dread dd/f4a [0,4194304] 0 2024-04-22T23:46:07.883 INFO:tasks.workunit.client.1.smithi044.stdout:2/368: symlink d8/d19/d36/d46/d63/d65/d6c/l7d 0 2024-04-22T23:46:07.886 INFO:tasks.workunit.client.1.smithi044.stdout:3/439: write d9/db/d33/f7c [2815457,60089] 0 2024-04-22T23:46:07.888 INFO:tasks.workunit.client.1.smithi044.stdout:7/425: unlink dd/f54 0 2024-04-22T23:46:07.907 INFO:tasks.workunit.client.1.smithi044.stdout:1/450: getdents d1/d8/d1c/d23/d3e 0 2024-04-22T23:46:07.911 INFO:tasks.workunit.client.1.smithi044.stdout:0/406: truncate d0/d2/dd/d3e/f6e 1482522 0 2024-04-22T23:46:07.914 INFO:tasks.workunit.client.1.smithi044.stdout:0/407: chown d0/d4c/d59/f76 328 1 2024-04-22T23:46:07.914 INFO:tasks.workunit.client.1.smithi044.stdout:2/369: dwrite d8/d19/d36/d46/f4e [0,4194304] 0 2024-04-22T23:46:07.916 INFO:tasks.workunit.client.1.smithi044.stdout:7/426: truncate dd/d1f/d24/f1c 329940 0 2024-04-22T23:46:07.919 INFO:tasks.workunit.client.1.smithi044.stdout:1/451: fsync d1/d8/d1c/d7b/f66 0 2024-04-22T23:46:07.922 INFO:tasks.workunit.client.1.smithi044.stdout:1/452: read d1/d8/f18 [3259984,31594] 0 2024-04-22T23:46:07.923 INFO:tasks.workunit.client.1.smithi044.stdout:0/408: dwrite d0/d2/d4b/f86 [0,4194304] 0 2024-04-22T23:46:07.926 INFO:tasks.workunit.client.1.smithi044.stdout:2/370: creat d8/d19/d5d/f7e x:0 0 0 2024-04-22T23:46:07.926 INFO:tasks.workunit.client.1.smithi044.stdout:0/409: stat d0/d2/c54 0 2024-04-22T23:46:07.930 INFO:tasks.workunit.client.1.smithi044.stdout:0/410: read d0/d2/dd/d3e/d40/d43/d7b/f58 [331680,43389] 0 2024-04-22T23:46:07.936 INFO:tasks.workunit.client.1.smithi044.stdout:7/427: symlink dd/d98/l9a 0 2024-04-22T23:46:07.940 INFO:tasks.workunit.client.1.smithi044.stdout:1/453: dwrite d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:07.947 INFO:tasks.workunit.client.1.smithi044.stdout:0/411: rename d0/d2/dd/d3e/d40/l62 to d0/d4c/l99 0 2024-04-22T23:46:07.948 INFO:tasks.workunit.client.1.smithi044.stdout:2/371: dwrite d8/d19/d36/d46/d54/f73 [0,4194304] 0 2024-04-22T23:46:07.951 INFO:tasks.workunit.client.1.smithi044.stdout:7/428: mkdir dd/d1a/d61/d9b 0 2024-04-22T23:46:07.957 INFO:tasks.workunit.client.1.smithi044.stdout:1/454: dwrite d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:07.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/412: chown d0/d2e/d33 204 1 2024-04-22T23:46:07.967 INFO:tasks.workunit.client.1.smithi044.stdout:1/455: mkdir d1/d8/d34/da1 0 2024-04-22T23:46:07.970 INFO:tasks.workunit.client.1.smithi044.stdout:0/413: rename d0/d2/l3 to d0/d2/dd/d3e/d40/d43/d7b/d90/l9a 0 2024-04-22T23:46:07.970 INFO:tasks.workunit.client.1.smithi044.stdout:1/456: dread d1/d8/d20/d47/f85 [0,4194304] 0 2024-04-22T23:46:07.972 INFO:tasks.workunit.client.1.smithi044.stdout:1/457: truncate d1/f1b 904049 0 2024-04-22T23:46:07.978 INFO:tasks.workunit.client.1.smithi044.stdout:0/414: link d0/d4c/d59/f76 d0/d2/dd/d3e/d40/d43/f9b 0 2024-04-22T23:46:07.980 INFO:tasks.workunit.client.1.smithi044.stdout:0/415: fdatasync d0/d2/dd/d3e/d40/f44 0 2024-04-22T23:46:07.980 INFO:tasks.workunit.client.1.smithi044.stdout:1/458: creat d1/d8/d20/d47/d92/fa2 x:0 0 0 2024-04-22T23:46:07.983 INFO:tasks.workunit.client.1.smithi044.stdout:7/429: fsync dd/d1f/d24/f1c 0 2024-04-22T23:46:07.988 INFO:tasks.workunit.client.1.smithi044.stdout:1/459: dwrite d1/d8/fa0 [0,4194304] 0 2024-04-22T23:46:07.988 INFO:tasks.workunit.client.1.smithi044.stdout:0/416: symlink d0/d2/d4b/d6b/l9c 0 2024-04-22T23:46:07.994 INFO:tasks.workunit.client.1.smithi044.stdout:7/430: dwrite dd/d79/f80 [0,4194304] 0 2024-04-22T23:46:07.999 INFO:tasks.workunit.client.1.smithi044.stdout:1/460: dwrite d1/d8/f12 [0,4194304] 0 2024-04-22T23:46:08.000 INFO:tasks.workunit.client.1.smithi044.stdout:1/461: stat d1/d8/d1c/d7b/d80/d4b 0 2024-04-22T23:46:08.003 INFO:tasks.workunit.client.1.smithi044.stdout:0/417: creat d0/d2e/d2f/d77/f9d x:0 0 0 2024-04-22T23:46:08.006 INFO:tasks.workunit.client.1.smithi044.stdout:7/431: mkdir dd/d79/d8c/d9c 0 2024-04-22T23:46:08.008 INFO:tasks.workunit.client.1.smithi044.stdout:1/462: creat d1/d8/d20/d45/d5d/fa3 x:0 0 0 2024-04-22T23:46:08.014 INFO:tasks.workunit.client.1.smithi044.stdout:0/418: symlink d0/d2e/d2f/l9e 0 2024-04-22T23:46:08.015 INFO:tasks.workunit.client.1.smithi044.stdout:7/432: dwrite dd/d55/d5e/d60/d7b/f7f [0,4194304] 0 2024-04-22T23:46:08.019 INFO:tasks.workunit.client.1.smithi044.stdout:0/419: read d0/d2/f64 [487533,30577] 0 2024-04-22T23:46:08.026 INFO:tasks.workunit.client.1.smithi044.stdout:7/433: unlink dd/d27/l2e 0 2024-04-22T23:46:08.027 INFO:tasks.workunit.client.1.smithi044.stdout:0/420: link d0/d2e/d2f/l47 d0/d4c/d59/l9f 0 2024-04-22T23:46:08.032 INFO:tasks.workunit.client.1.smithi044.stdout:0/421: creat d0/d2/dd/d3e/fa0 x:0 0 0 2024-04-22T23:46:08.040 INFO:tasks.workunit.client.1.smithi044.stdout:0/422: dwrite d0/d2/f64 [0,4194304] 0 2024-04-22T23:46:08.042 INFO:tasks.workunit.client.1.smithi044.stdout:0/423: truncate d0/d2e/d2f/d61/f94 179126 0 2024-04-22T23:46:08.045 INFO:tasks.workunit.client.1.smithi044.stdout:0/424: unlink d0/d2e/d2f/f65 0 2024-04-22T23:46:08.055 INFO:tasks.workunit.client.1.smithi044.stdout:0/425: getdents d0/d2/dd/d3e/d40/d43 0 2024-04-22T23:46:08.069 INFO:tasks.workunit.client.1.smithi044.stdout:0/426: dread d0/f8 [0,4194304] 0 2024-04-22T23:46:08.072 INFO:tasks.workunit.client.1.smithi044.stdout:0/427: unlink d0/d2e/d2f/f50 0 2024-04-22T23:46:08.077 INFO:tasks.workunit.client.1.smithi044.stdout:0/428: fsync d0/d2/dd/d3e/d40/d43/d7b/f5c 0 2024-04-22T23:46:08.079 INFO:tasks.workunit.client.1.smithi044.stdout:0/429: dread d0/d2/f6 [0,4194304] 0 2024-04-22T23:46:08.085 INFO:tasks.workunit.client.1.smithi044.stdout:0/430: rename d0/f8 to d0/d2/dd/fa1 0 2024-04-22T23:46:08.088 INFO:tasks.workunit.client.1.smithi044.stdout:0/431: symlink d0/d2e/d2f/la2 0 2024-04-22T23:46:08.090 INFO:tasks.workunit.client.1.smithi044.stdout:0/432: write d0/d2e/d2f/d77/f93 [311572,121319] 0 2024-04-22T23:46:08.092 INFO:tasks.workunit.client.1.smithi044.stdout:0/433: creat d0/d2/d4b/fa3 x:0 0 0 2024-04-22T23:46:08.093 INFO:tasks.workunit.client.1.smithi044.stdout:0/434: write d0/d2e/f7a [752525,2487] 0 2024-04-22T23:46:08.098 INFO:tasks.workunit.client.1.smithi044.stdout:0/435: rename d0/d2e/d2f/l47 to d0/d60/la4 0 2024-04-22T23:46:08.100 INFO:tasks.workunit.client.1.smithi044.stdout:0/436: mknod d0/d2/dd/d3e/ca5 0 2024-04-22T23:46:08.101 INFO:tasks.workunit.client.1.smithi044.stdout:0/437: dread - d0/d2e/d2f/d77/f9d zero size 2024-04-22T23:46:08.105 INFO:tasks.workunit.client.1.smithi044.stdout:0/438: rename d0/d2e/d2f/l9e to d0/d2/dd/la6 0 2024-04-22T23:46:08.112 INFO:tasks.workunit.client.1.smithi044.stdout:0/439: getdents d0/d2/dd/d3e 0 2024-04-22T23:46:08.115 INFO:tasks.workunit.client.1.smithi044.stdout:0/440: creat d0/d2/dd/d3e/d40/d67/fa7 x:0 0 0 2024-04-22T23:46:08.118 INFO:tasks.workunit.client.1.smithi044.stdout:0/441: symlink d0/d2/dd/d3e/d40/d43/d7b/d90/la8 0 2024-04-22T23:46:08.122 INFO:tasks.workunit.client.1.smithi044.stdout:0/442: link d0/d2e/d2f/c42 d0/d2e/d33/ca9 0 2024-04-22T23:46:08.123 INFO:tasks.workunit.client.1.smithi044.stdout:0/443: truncate d0/d2/d4b/fa3 919761 0 2024-04-22T23:46:08.127 INFO:tasks.workunit.client.1.smithi044.stdout:0/444: link d0/d2e/d2f/f4d d0/d2e/d2f/d77/faa 0 2024-04-22T23:46:08.129 INFO:tasks.workunit.client.1.smithi044.stdout:0/445: truncate d0/d2/f49 735620 0 2024-04-22T23:46:08.131 INFO:tasks.workunit.client.1.smithi044.stdout:0/446: chown d0/d2/cc 607899 1 2024-04-22T23:46:08.134 INFO:tasks.workunit.client.1.smithi044.stdout:0/447: mknod d0/cab 0 2024-04-22T23:46:08.137 INFO:tasks.workunit.client.1.smithi044.stdout:0/448: symlink d0/d2/dd/lac 0 2024-04-22T23:46:08.138 INFO:tasks.workunit.client.1.smithi044.stdout:0/449: fsync d0/d2/dd/d3e/d40/f98 0 2024-04-22T23:46:08.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/414: sync 2024-04-22T23:46:08.141 INFO:tasks.workunit.client.1.smithi044.stdout:0/450: unlink d0/d2/dd/f12 0 2024-04-22T23:46:08.145 INFO:tasks.workunit.client.1.smithi044.stdout:5/415: mkdir d15/d16/d1d/d22/d2a/d3a/d43/d80 0 2024-04-22T23:46:08.147 INFO:tasks.workunit.client.1.smithi044.stdout:0/451: mkdir d0/d2/d4b/dad 0 2024-04-22T23:46:08.151 INFO:tasks.workunit.client.1.smithi044.stdout:5/416: dwrite d15/d16/d1d/d22/f2d [0,4194304] 0 2024-04-22T23:46:08.152 INFO:tasks.workunit.client.1.smithi044.stdout:5/417: write f5 [7219445,8053] 0 2024-04-22T23:46:08.153 INFO:tasks.workunit.client.1.smithi044.stdout:5/418: chown d15/d16/d1d/d22 14 1 2024-04-22T23:46:08.162 INFO:tasks.workunit.client.1.smithi044.stdout:5/419: dwrite d15/d16/d1d/d22/d4f/f59 [4194304,4194304] 0 2024-04-22T23:46:08.166 INFO:tasks.workunit.client.1.smithi044.stdout:5/420: rmdir d15/d16/d1d/d22 39 2024-04-22T23:46:08.176 INFO:tasks.workunit.client.1.smithi044.stdout:5/421: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f4c [4194304,4194304] 0 2024-04-22T23:46:08.182 INFO:tasks.workunit.client.1.smithi044.stdout:0/452: dwrite d0/d2/dd/fa1 [0,4194304] 0 2024-04-22T23:46:08.190 INFO:tasks.workunit.client.1.smithi044.stdout:5/422: dwrite d15/d3c/f77 [0,4194304] 0 2024-04-22T23:46:08.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:07 smithi031 ceph-mon[26721]: pgmap v7: 65 pgs: 65 active+clean; 2.6 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 20 MiB/s rd, 92 MiB/s wr, 208 op/s 2024-04-22T23:46:08.193 INFO:tasks.workunit.client.1.smithi044.stdout:0/453: rename d0/d2e/d2f/d77/faa to d0/d4c/d59/d5a/fae 0 2024-04-22T23:46:08.195 INFO:tasks.workunit.client.1.smithi044.stdout:5/423: mkdir d15/d16/d1d/d22/d81 0 2024-04-22T23:46:08.199 INFO:tasks.workunit.client.1.smithi044.stdout:0/454: mknod d0/d4c/d8b/caf 0 2024-04-22T23:46:08.202 INFO:tasks.workunit.client.1.smithi044.stdout:0/455: rmdir d0/d4c/d8b 39 2024-04-22T23:46:08.203 INFO:tasks.workunit.client.1.smithi044.stdout:5/424: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f4c [4194304,4194304] 0 2024-04-22T23:46:08.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/425: mkdir d15/d5d/d82 0 2024-04-22T23:46:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:07 smithi044 ceph-mon[34463]: pgmap v7: 65 pgs: 65 active+clean; 2.6 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 20 MiB/s rd, 92 MiB/s wr, 208 op/s 2024-04-22T23:46:08.213 INFO:tasks.workunit.client.1.smithi044.stdout:0/456: rmdir d0/d2e 39 2024-04-22T23:46:08.217 INFO:tasks.workunit.client.1.smithi044.stdout:5/426: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:08.222 INFO:tasks.workunit.client.1.smithi044.stdout:0/457: dwrite d0/d2/f64 [0,4194304] 0 2024-04-22T23:46:08.224 INFO:tasks.workunit.client.1.smithi044.stdout:5/427: creat d15/d5d/d82/f83 x:0 0 0 2024-04-22T23:46:08.224 INFO:tasks.workunit.client.1.smithi044.stdout:0/458: readlink d0/d2/d4b/d6b/l9c 0 2024-04-22T23:46:08.233 INFO:tasks.workunit.client.1.smithi044.stdout:5/428: creat d15/d3c/f84 x:0 0 0 2024-04-22T23:46:08.235 INFO:tasks.workunit.client.1.smithi044.stdout:0/459: dwrite d0/d2e/d2f/d61/f94 [0,4194304] 0 2024-04-22T23:46:08.237 INFO:tasks.workunit.client.1.smithi044.stdout:5/429: creat d15/d16/d1d/d22/d2a/f85 x:0 0 0 2024-04-22T23:46:08.245 INFO:tasks.workunit.client.1.smithi044.stdout:5/430: getdents d15/d16/d1d/d22/d81 0 2024-04-22T23:46:08.248 INFO:tasks.workunit.client.1.smithi044.stdout:0/460: dwrite d0/d4c/f8f [0,4194304] 0 2024-04-22T23:46:08.248 INFO:tasks.workunit.client.1.smithi044.stdout:5/431: chown d15/d16/d1d/d22/d2a/d3a/d43/f61 863549 1 2024-04-22T23:46:08.252 INFO:tasks.workunit.client.1.smithi044.stdout:0/461: creat d0/d2/dd/d3e/d40/fb0 x:0 0 0 2024-04-22T23:46:08.253 INFO:tasks.workunit.client.1.smithi044.stdout:0/462: dread - d0/d2/d4b/d6b/f82 zero size 2024-04-22T23:46:08.254 INFO:tasks.workunit.client.1.smithi044.stdout:0/463: truncate d0/d2/dd/f1a 4445383 0 2024-04-22T23:46:08.256 INFO:tasks.workunit.client.1.smithi044.stdout:0/464: fsync d0/d2/dd/d3e/d40/d43/d7b/f58 0 2024-04-22T23:46:08.257 INFO:tasks.workunit.client.1.smithi044.stdout:0/465: readlink d0/d2/l29 0 2024-04-22T23:46:08.260 INFO:tasks.workunit.client.1.smithi044.stdout:0/466: unlink d0/d2/dd/d3e/d40/d67/d7c/c88 0 2024-04-22T23:46:08.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/467: mknod d0/d60/cb1 0 2024-04-22T23:46:08.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/468: stat d0/d2e/f32 0 2024-04-22T23:46:08.265 INFO:tasks.workunit.client.1.smithi044.stdout:5/432: dread f5 [0,4194304] 0 2024-04-22T23:46:08.266 INFO:tasks.workunit.client.1.smithi044.stdout:5/433: fdatasync d15/d16/d1d/d22/d2a/d3a/d43/f7e 0 2024-04-22T23:46:08.269 INFO:tasks.workunit.client.1.smithi044.stdout:5/434: mknod d15/d16/d1d/d22/c86 0 2024-04-22T23:46:08.271 INFO:tasks.workunit.client.1.smithi044.stdout:5/435: rmdir d15/d16/d1d/d22/d4f 39 2024-04-22T23:46:08.272 INFO:tasks.workunit.client.1.smithi044.stdout:5/436: stat d15/d5d/d64/d6f 0 2024-04-22T23:46:08.280 INFO:tasks.workunit.client.1.smithi044.stdout:5/437: getdents d15/d16 0 2024-04-22T23:46:08.283 INFO:tasks.workunit.client.1.smithi044.stdout:5/438: symlink d15/d16/l87 0 2024-04-22T23:46:08.290 INFO:tasks.workunit.client.1.smithi044.stdout:5/439: write d15/d16/d1d/f42 [367456,117203] 0 2024-04-22T23:46:08.299 INFO:tasks.workunit.client.1.smithi044.stdout:5/440: dwrite d15/d16/d2c/f54 [0,4194304] 0 2024-04-22T23:46:08.304 INFO:tasks.workunit.client.1.smithi044.stdout:5/441: rename d15/d3c/f84 to d15/d16/d1d/d22/d2a/d3a/d43/f88 0 2024-04-22T23:46:08.311 INFO:tasks.workunit.client.1.smithi044.stdout:5/442: rename d15/d16/d1d/f26 to d15/d16/d1d/f89 0 2024-04-22T23:46:08.311 INFO:tasks.workunit.client.1.smithi044.stdout:5/443: dread - d15/d5d/d82/f83 zero size 2024-04-22T23:46:08.314 INFO:tasks.workunit.client.1.smithi044.stdout:5/444: creat d15/d5d/d64/d6f/f8a x:0 0 0 2024-04-22T23:46:08.316 INFO:tasks.workunit.client.1.smithi044.stdout:5/445: truncate d15/d5d/d64/d6f/f74 267648 0 2024-04-22T23:46:08.330 INFO:tasks.workunit.client.1.smithi044.stdout:8/454: sync 2024-04-22T23:46:08.330 INFO:tasks.workunit.client.1.smithi044.stdout:6/360: sync 2024-04-22T23:46:08.331 INFO:tasks.workunit.client.1.smithi044.stdout:6/361: read d5/d3f/f22 [2339959,118873] 0 2024-04-22T23:46:08.335 INFO:tasks.workunit.client.1.smithi044.stdout:6/362: unlink d5/d3f/d4c/d40/f5b 0 2024-04-22T23:46:08.337 INFO:tasks.workunit.client.1.smithi044.stdout:6/363: symlink d5/d3f/d50/d79/l7b 0 2024-04-22T23:46:08.339 INFO:tasks.workunit.client.1.smithi044.stdout:6/364: chown d5/d3f/d4c/d23/f26 5252 1 2024-04-22T23:46:08.341 INFO:tasks.workunit.client.1.smithi044.stdout:6/365: mknod d5/d3f/d4c/d40/d53/d56/c7c 0 2024-04-22T23:46:08.343 INFO:tasks.workunit.client.1.smithi044.stdout:8/455: dread db/f1c [0,4194304] 0 2024-04-22T23:46:08.344 INFO:tasks.workunit.client.1.smithi044.stdout:8/456: truncate db/d52/f83 418932 0 2024-04-22T23:46:08.345 INFO:tasks.workunit.client.1.smithi044.stdout:8/457: truncate db/f3c 875743 0 2024-04-22T23:46:08.348 INFO:tasks.workunit.client.1.smithi044.stdout:6/366: getdents d5/dd 0 2024-04-22T23:46:08.349 INFO:tasks.workunit.client.1.smithi044.stdout:8/458: mkdir db/d68/d8b/da7 0 2024-04-22T23:46:08.352 INFO:tasks.workunit.client.1.smithi044.stdout:8/459: symlink db/d52/la8 0 2024-04-22T23:46:08.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/367: dwrite d5/d3f/f22 [4194304,4194304] 0 2024-04-22T23:46:08.358 INFO:tasks.workunit.client.1.smithi044.stdout:6/368: dread - d5/d3f/d4c/d40/f78 zero size 2024-04-22T23:46:08.360 INFO:tasks.workunit.client.1.smithi044.stdout:8/460: getdents db/d14/d72 0 2024-04-22T23:46:08.361 INFO:tasks.workunit.client.1.smithi044.stdout:6/369: write d5/dd/d28/f2c [1091762,114983] 0 2024-04-22T23:46:08.366 INFO:tasks.workunit.client.1.smithi044.stdout:8/461: symlink db/d68/d74/la9 0 2024-04-22T23:46:08.367 INFO:tasks.workunit.client.1.smithi044.stdout:6/370: creat d5/dd/f7d x:0 0 0 2024-04-22T23:46:08.368 INFO:tasks.workunit.client.1.smithi044.stdout:6/371: chown d5/d30/f44 3538 1 2024-04-22T23:46:08.369 INFO:tasks.workunit.client.1.smithi044.stdout:8/462: symlink db/d68/laa 0 2024-04-22T23:46:08.374 INFO:tasks.workunit.client.1.smithi044.stdout:8/463: mkdir db/d14/dab 0 2024-04-22T23:46:08.377 INFO:tasks.workunit.client.1.smithi044.stdout:8/464: unlink db/d14/c87 0 2024-04-22T23:46:08.380 INFO:tasks.workunit.client.1.smithi044.stdout:8/465: creat db/d68/d74/fac x:0 0 0 2024-04-22T23:46:08.383 INFO:tasks.workunit.client.1.smithi044.stdout:8/466: mkdir db/d52/d9c/dad 0 2024-04-22T23:46:08.385 INFO:tasks.workunit.client.1.smithi044.stdout:8/467: creat db/d68/d8b/da7/fae x:0 0 0 2024-04-22T23:46:08.387 INFO:tasks.workunit.client.1.smithi044.stdout:8/468: mkdir db/d68/daf 0 2024-04-22T23:46:08.391 INFO:tasks.workunit.client.1.smithi044.stdout:8/469: dread db/d14/f25 [0,4194304] 0 2024-04-22T23:46:08.392 INFO:tasks.workunit.client.1.smithi044.stdout:8/470: write db/f4f [951290,93239] 0 2024-04-22T23:46:08.393 INFO:tasks.workunit.client.1.smithi044.stdout:8/471: stat db/d14/f42 0 2024-04-22T23:46:08.396 INFO:tasks.workunit.client.1.smithi044.stdout:8/472: rename db/d52/f71 to db/d68/d74/fb0 0 2024-04-22T23:46:08.397 INFO:tasks.workunit.client.1.smithi044.stdout:8/473: dread - db/d52/f5e zero size 2024-04-22T23:46:08.399 INFO:tasks.workunit.client.1.smithi044.stdout:8/474: chown db/d14/l7c 12323000 1 2024-04-22T23:46:08.400 INFO:tasks.workunit.client.1.smithi044.stdout:8/475: write db/d68/f7a [393703,27188] 0 2024-04-22T23:46:08.404 INFO:tasks.workunit.client.1.smithi044.stdout:8/476: link db/d14/c85 db/d52/cb1 0 2024-04-22T23:46:08.406 INFO:tasks.workunit.client.1.smithi044.stdout:8/477: chown db/d52/d9c/la4 6377737 1 2024-04-22T23:46:08.407 INFO:tasks.workunit.client.1.smithi044.stdout:8/478: write db/f2a [1016473,87077] 0 2024-04-22T23:46:08.411 INFO:tasks.workunit.client.1.smithi044.stdout:8/479: getdents db/d68/daf 0 2024-04-22T23:46:08.417 INFO:tasks.workunit.client.1.smithi044.stdout:8/480: dread db/f50 [0,4194304] 0 2024-04-22T23:46:08.419 INFO:tasks.workunit.client.1.smithi044.stdout:8/481: mknod db/d14/d72/cb2 0 2024-04-22T23:46:08.421 INFO:tasks.workunit.client.1.smithi044.stdout:8/482: creat db/d52/d9c/dad/fb3 x:0 0 0 2024-04-22T23:46:08.424 INFO:tasks.workunit.client.1.smithi044.stdout:8/483: creat db/d14/d84/fb4 x:0 0 0 2024-04-22T23:46:08.425 INFO:tasks.workunit.client.1.smithi044.stdout:8/484: write db/d68/f7a [1059436,38583] 0 2024-04-22T23:46:08.425 INFO:tasks.workunit.client.1.smithi044.stdout:8/485: fsync db/d14/f46 0 2024-04-22T23:46:08.430 INFO:tasks.workunit.client.1.smithi044.stdout:8/486: dread db/f12 [4194304,4194304] 0 2024-04-22T23:46:08.430 INFO:tasks.workunit.client.1.smithi044.stdout:8/487: chown db/d14/d84 439353 1 2024-04-22T23:46:08.448 INFO:tasks.workunit.client.1.smithi044.stdout:4/446: sync 2024-04-22T23:46:08.450 INFO:tasks.workunit.client.1.smithi044.stdout:4/447: write d5/db/d16/d4d/f8a [692249,106274] 0 2024-04-22T23:46:08.451 INFO:tasks.workunit.client.1.smithi044.stdout:4/448: write d5/d31/d44/d20/f55 [2532120,33628] 0 2024-04-22T23:46:08.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/488: rename db/d68/d74/l9a to db/d14/lb5 0 2024-04-22T23:46:08.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/449: rename d5/d31/d44/d20/d22/l57 to d5/d31/d24/d38/d6a/l92 0 2024-04-22T23:46:08.463 INFO:tasks.workunit.client.1.smithi044.stdout:4/450: chown d5/db/f86 4 1 2024-04-22T23:46:08.464 INFO:tasks.workunit.client.1.smithi044.stdout:4/451: chown d5/d31/d44/f4a 11993781 1 2024-04-22T23:46:08.470 INFO:tasks.workunit.client.1.smithi044.stdout:8/489: rmdir db/d14/dab 0 2024-04-22T23:46:08.475 INFO:tasks.workunit.client.1.smithi044.stdout:8/490: rename db/d14/l2d to db/d14/d24/lb6 0 2024-04-22T23:46:08.478 INFO:tasks.workunit.client.1.smithi044.stdout:8/491: symlink db/d14/d84/lb7 0 2024-04-22T23:46:08.479 INFO:tasks.workunit.client.1.smithi044.stdout:8/492: read - db/d52/d9c/dad/fb3 zero size 2024-04-22T23:46:08.482 INFO:tasks.workunit.client.1.smithi044.stdout:8/493: mkdir db/d52/d9c/dad/db8 0 2024-04-22T23:46:08.495 INFO:tasks.workunit.client.1.smithi044.stdout:4/452: dwrite d5/d31/f76 [0,4194304] 0 2024-04-22T23:46:08.498 INFO:tasks.workunit.client.1.smithi044.stdout:8/494: dwrite db/d14/d72/f9f [0,4194304] 0 2024-04-22T23:46:08.499 INFO:tasks.workunit.client.1.smithi044.stdout:8/495: chown db 1287024915 1 2024-04-22T23:46:08.502 INFO:tasks.workunit.client.1.smithi044.stdout:4/453: mkdir d5/d31/d44/d20/d93 0 2024-04-22T23:46:08.506 INFO:tasks.workunit.client.1.smithi044.stdout:8/496: symlink db/d14/d72/lb9 0 2024-04-22T23:46:08.509 INFO:tasks.workunit.client.1.smithi044.stdout:4/454: link d5/d31/d44/d20/d22/f62 d5/d31/d44/f94 0 2024-04-22T23:46:08.512 INFO:tasks.workunit.client.1.smithi044.stdout:8/497: symlink db/d68/daf/lba 0 2024-04-22T23:46:08.514 INFO:tasks.workunit.client.1.smithi044.stdout:4/455: creat d5/d52/f95 x:0 0 0 2024-04-22T23:46:08.518 INFO:tasks.workunit.client.1.smithi044.stdout:4/456: unlink d5/d31/d24/d38/d6a/d75/f7f 0 2024-04-22T23:46:08.521 INFO:tasks.workunit.client.1.smithi044.stdout:4/457: mknod d5/d31/d44/d7e/c96 0 2024-04-22T23:46:08.521 INFO:tasks.workunit.client.1.smithi044.stdout:4/458: chown d5/d31/d24/d69 19232874 1 2024-04-22T23:46:08.526 INFO:tasks.workunit.client.1.smithi044.stdout:4/459: rename d5/d42/f48 to d5/d31/d44/d20/d22/d8d/f97 0 2024-04-22T23:46:08.534 INFO:tasks.workunit.client.1.smithi044.stdout:4/460: rename d5/db/d16/f8e to d5/d31/d44/d7e/f98 0 2024-04-22T23:46:08.535 INFO:tasks.workunit.client.1.smithi044.stdout:4/461: chown d5/db/d34/c3b 1403066010 1 2024-04-22T23:46:08.536 INFO:tasks.workunit.client.1.smithi044.stdout:4/462: write d5/d31/d24/d69/f6c [553199,111224] 0 2024-04-22T23:46:08.542 INFO:tasks.workunit.client.1.smithi044.stdout:4/463: symlink d5/d31/d44/d20/d22/d8d/l99 0 2024-04-22T23:46:08.544 INFO:tasks.workunit.client.1.smithi044.stdout:4/464: write d5/db/d3c/f82 [292173,17335] 0 2024-04-22T23:46:08.548 INFO:tasks.workunit.client.1.smithi044.stdout:4/465: mknod d5/d31/d44/d20/d22/d8d/c9a 0 2024-04-22T23:46:08.549 INFO:tasks.workunit.client.1.smithi044.stdout:4/466: write d5/db/f86 [208397,42755] 0 2024-04-22T23:46:08.557 INFO:tasks.workunit.client.1.smithi044.stdout:4/467: rename d5/ca to d5/d31/d44/c9b 0 2024-04-22T23:46:08.560 INFO:tasks.workunit.client.1.smithi044.stdout:4/468: mknod d5/d31/d24/c9c 0 2024-04-22T23:46:08.565 INFO:tasks.workunit.client.1.smithi044.stdout:4/469: link l4 d5/d31/d44/d20/d22/d8d/l9d 0 2024-04-22T23:46:08.570 INFO:tasks.workunit.client.1.smithi044.stdout:4/470: link d5/d31/d44/d20/d22/d8d/c9a d5/d52/c9e 0 2024-04-22T23:46:08.571 INFO:tasks.workunit.client.1.smithi044.stdout:4/471: fsync d5/db/f86 0 2024-04-22T23:46:08.573 INFO:tasks.workunit.client.1.smithi044.stdout:4/472: truncate d5/d31/d44/d20/f4b 4524096 0 2024-04-22T23:46:08.574 INFO:tasks.workunit.client.1.smithi044.stdout:4/473: read - d5/d31/d44/d20/d22/f67 zero size 2024-04-22T23:46:08.580 INFO:tasks.workunit.client.1.smithi044.stdout:4/474: dwrite d5/d31/d24/d38/d6a/f89 [0,4194304] 0 2024-04-22T23:46:08.590 INFO:tasks.workunit.client.1.smithi044.stdout:4/475: read d5/d31/d44/f4a [1594478,36469] 0 2024-04-22T23:46:08.592 INFO:tasks.workunit.client.1.smithi044.stdout:4/476: chown d5/d31/d44/d20/d22/l45 102030 1 2024-04-22T23:46:08.594 INFO:tasks.workunit.client.1.smithi044.stdout:4/477: dread - d5/d31/d44/d20/d22/f67 zero size 2024-04-22T23:46:08.600 INFO:tasks.workunit.client.1.smithi044.stdout:4/478: unlink d5/c7 0 2024-04-22T23:46:08.604 INFO:tasks.workunit.client.1.smithi044.stdout:4/479: creat d5/d31/d24/d38/f9f x:0 0 0 2024-04-22T23:46:08.608 INFO:tasks.workunit.client.1.smithi044.stdout:4/480: creat d5/db/d16/fa0 x:0 0 0 2024-04-22T23:46:08.610 INFO:tasks.workunit.client.1.smithi044.stdout:4/481: read d5/d31/f10 [1843070,31909] 0 2024-04-22T23:46:08.623 INFO:tasks.workunit.client.1.smithi044.stdout:4/482: dread d5/d31/d44/f29 [0,4194304] 0 2024-04-22T23:46:08.626 INFO:tasks.workunit.client.1.smithi044.stdout:4/483: mknod d5/d31/d24/d38/ca1 0 2024-04-22T23:46:08.626 INFO:tasks.workunit.client.1.smithi044.stdout:4/484: chown d5/d31/d44/d20/d22/c33 288615 1 2024-04-22T23:46:08.629 INFO:tasks.workunit.client.1.smithi044.stdout:4/485: creat d5/d31/d24/d69/fa2 x:0 0 0 2024-04-22T23:46:08.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/486: dwrite d5/d31/d44/d20/d22/f4c [0,4194304] 0 2024-04-22T23:46:08.663 INFO:tasks.workunit.client.1.smithi044.stdout:9/287: sync 2024-04-22T23:46:08.663 INFO:tasks.workunit.client.1.smithi044.stdout:2/372: sync 2024-04-22T23:46:08.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/440: sync 2024-04-22T23:46:08.666 INFO:tasks.workunit.client.1.smithi044.stdout:4/487: stat d5/d31/d24/d38/d6a/d75 0 2024-04-22T23:46:08.666 INFO:tasks.workunit.client.1.smithi044.stdout:3/441: chown d9/l7b 28 1 2024-04-22T23:46:08.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/288: dread - d2/dc/d1d/d3f/f63 zero size 2024-04-22T23:46:08.682 INFO:tasks.workunit.client.1.smithi044.stdout:9/289: getdents d2/d6 0 2024-04-22T23:46:08.693 INFO:tasks.workunit.client.1.smithi044.stdout:9/290: mknod d2/dc/d1d/d34/c6b 0 2024-04-22T23:46:08.699 INFO:tasks.workunit.client.1.smithi044.stdout:9/291: mknod d2/dc/d1d/c6c 0 2024-04-22T23:46:08.701 INFO:tasks.workunit.client.1.smithi044.stdout:4/488: dwrite d5/db/d16/f3e [0,4194304] 0 2024-04-22T23:46:08.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/373: dread d8/fe [0,4194304] 0 2024-04-22T23:46:08.705 INFO:tasks.workunit.client.1.smithi044.stdout:4/489: creat d5/db/d3c/fa3 x:0 0 0 2024-04-22T23:46:08.706 INFO:tasks.workunit.client.1.smithi044.stdout:9/292: getdents d2/dc/d1d/d34/d41 0 2024-04-22T23:46:08.710 INFO:tasks.workunit.client.1.smithi044.stdout:2/374: creat d8/d27/d5b/f7f x:0 0 0 2024-04-22T23:46:08.719 INFO:tasks.workunit.client.1.smithi044.stdout:4/490: dwrite d5/d31/d44/d20/d22/d8d/f97 [0,4194304] 0 2024-04-22T23:46:08.720 INFO:tasks.workunit.client.1.smithi044.stdout:2/375: rmdir d8/d19 39 2024-04-22T23:46:08.732 INFO:tasks.workunit.client.1.smithi044.stdout:2/376: dwrite d8/d19/d36/d46/f4e [4194304,4194304] 0 2024-04-22T23:46:08.735 INFO:tasks.workunit.client.1.smithi044.stdout:2/377: mkdir d8/d27/d5b/d74/d80 0 2024-04-22T23:46:08.737 INFO:tasks.workunit.client.1.smithi044.stdout:2/378: mkdir d8/d27/d5b/d74/d81 0 2024-04-22T23:46:08.739 INFO:tasks.workunit.client.1.smithi044.stdout:2/379: rmdir d8/d19/d1c 39 2024-04-22T23:46:08.741 INFO:tasks.workunit.client.1.smithi044.stdout:2/380: write d8/d19/d36/d44/f70 [985752,111678] 0 2024-04-22T23:46:08.747 INFO:tasks.workunit.client.1.smithi044.stdout:9/293: dread d2/dc/d1d/d34/f49 [0,4194304] 0 2024-04-22T23:46:08.749 INFO:tasks.workunit.client.1.smithi044.stdout:9/294: rmdir d2/dc/d1d 39 2024-04-22T23:46:08.754 INFO:tasks.workunit.client.1.smithi044.stdout:9/295: readlink d2/dc/d1d/d34/d38/l56 0 2024-04-22T23:46:08.757 INFO:tasks.workunit.client.1.smithi044.stdout:2/381: rmdir d8/d27/d5b 39 2024-04-22T23:46:08.758 INFO:tasks.workunit.client.1.smithi044.stdout:2/382: write d8/d19/f30 [4667451,85632] 0 2024-04-22T23:46:08.759 INFO:tasks.workunit.client.1.smithi044.stdout:2/383: fsync d8/d19/d36/d44/f52 0 2024-04-22T23:46:08.763 INFO:tasks.workunit.client.1.smithi044.stdout:9/296: link d2/d5/f50 d2/dc/d1d/d34/d38/d5c/f6d 0 2024-04-22T23:46:08.766 INFO:tasks.workunit.client.1.smithi044.stdout:2/384: dwrite d8/d19/d5d/f7c [0,4194304] 0 2024-04-22T23:46:08.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/385: chown d8/d19/d5d 21735 1 2024-04-22T23:46:08.771 INFO:tasks.workunit.client.1.smithi044.stdout:2/386: rename d8/l12 to d8/d19/d5d/l82 0 2024-04-22T23:46:08.775 INFO:tasks.workunit.client.1.smithi044.stdout:9/297: dwrite d2/dc/d1d/d34/f49 [4194304,4194304] 0 2024-04-22T23:46:08.776 INFO:tasks.workunit.client.1.smithi044.stdout:9/298: chown d2/dc/d1d/d34/d38/d5c/f65 2031040335 1 2024-04-22T23:46:08.779 INFO:tasks.workunit.client.1.smithi044.stdout:2/387: rename d8/d19/f5c to d8/d19/d5d/f83 0 2024-04-22T23:46:08.787 INFO:tasks.workunit.client.1.smithi044.stdout:2/388: creat d8/d19/d36/d46/d63/d65/f84 x:0 0 0 2024-04-22T23:46:08.792 INFO:tasks.workunit.client.1.smithi044.stdout:9/299: dwrite d2/d5/f7 [0,4194304] 0 2024-04-22T23:46:08.792 INFO:tasks.workunit.client.1.smithi044.stdout:9/300: stat d2 0 2024-04-22T23:46:08.799 INFO:tasks.workunit.client.1.smithi044.stdout:9/301: write d2/dc/f27 [3933349,9899] 0 2024-04-22T23:46:08.799 INFO:tasks.workunit.client.1.smithi044.stdout:2/389: getdents d8/d27/d5b/d74/d80 0 2024-04-22T23:46:08.801 INFO:tasks.workunit.client.1.smithi044.stdout:9/302: truncate d2/d5/f50 778247 0 2024-04-22T23:46:08.802 INFO:tasks.workunit.client.1.smithi044.stdout:2/390: rmdir d8/d19/d36/d46/d63 39 2024-04-22T23:46:08.819 INFO:tasks.workunit.client.1.smithi044.stdout:9/303: dwrite d2/dc/d1d/d3f/f48 [0,4194304] 0 2024-04-22T23:46:08.823 INFO:tasks.workunit.client.1.smithi044.stdout:9/304: fsync d2/dc/d1d/d3f/f4c 0 2024-04-22T23:46:08.825 INFO:tasks.workunit.client.1.smithi044.stdout:9/305: dread d2/dc/d1d/d34/d38/d5c/f6d [0,4194304] 0 2024-04-22T23:46:08.832 INFO:tasks.workunit.client.1.smithi044.stdout:1/463: sync 2024-04-22T23:46:08.838 INFO:tasks.workunit.client.1.smithi044.stdout:2/391: dread d8/d19/f25 [0,4194304] 0 2024-04-22T23:46:08.844 INFO:tasks.workunit.client.1.smithi044.stdout:2/392: creat d8/d19/d36/d46/d63/d65/f85 x:0 0 0 2024-04-22T23:46:08.846 INFO:tasks.workunit.client.1.smithi044.stdout:2/393: read d8/d19/f37 [2269004,91276] 0 2024-04-22T23:46:08.846 INFO:tasks.workunit.client.1.smithi044.stdout:2/394: write d8/d19/d5d/f5f [325301,10780] 0 2024-04-22T23:46:08.850 INFO:tasks.workunit.client.1.smithi044.stdout:1/464: dread d1/d8/f18 [4194304,4194304] 0 2024-04-22T23:46:08.854 INFO:tasks.workunit.client.1.smithi044.stdout:1/465: symlink d1/d8/d1c/d23/d48/d72/la4 0 2024-04-22T23:46:08.856 INFO:tasks.workunit.client.1.smithi044.stdout:1/466: rmdir d1/d8/d1c/d74 39 2024-04-22T23:46:08.861 INFO:tasks.workunit.client.1.smithi044.stdout:2/395: dread d8/d19/f20 [0,4194304] 0 2024-04-22T23:46:08.862 INFO:tasks.workunit.client.1.smithi044.stdout:1/467: dread d1/d8/d1c/d23/f5f [0,4194304] 0 2024-04-22T23:46:08.874 INFO:tasks.workunit.client.1.smithi044.stdout:2/396: dread d8/d27/f4b [0,4194304] 0 2024-04-22T23:46:08.874 INFO:tasks.workunit.client.1.smithi044.stdout:2/397: chown d8/d27 24926 1 2024-04-22T23:46:08.878 INFO:tasks.workunit.client.1.smithi044.stdout:2/398: rename d8/d27/f2c to d8/d19/d36/d46/d63/d65/d72/f86 0 2024-04-22T23:46:08.882 INFO:tasks.workunit.client.1.smithi044.stdout:2/399: rename d8/d27/d5b/c6b to d8/d19/d36/d46/d54/c87 0 2024-04-22T23:46:08.883 INFO:tasks.workunit.client.1.smithi044.stdout:2/400: write f3 [354438,108695] 0 2024-04-22T23:46:08.889 INFO:tasks.workunit.client.1.smithi044.stdout:2/401: dwrite d8/d27/f4d [0,4194304] 0 2024-04-22T23:46:08.892 INFO:tasks.workunit.client.1.smithi044.stdout:2/402: symlink d8/d19/d36/d44/l88 0 2024-04-22T23:46:08.899 INFO:tasks.workunit.client.1.smithi044.stdout:2/403: dwrite d8/d19/d36/d44/f55 [0,4194304] 0 2024-04-22T23:46:08.902 INFO:tasks.workunit.client.1.smithi044.stdout:2/404: unlink d8/f26 0 2024-04-22T23:46:08.912 INFO:tasks.workunit.client.1.smithi044.stdout:2/405: creat d8/d19/d36/d46/f89 x:0 0 0 2024-04-22T23:46:08.916 INFO:tasks.workunit.client.1.smithi044.stdout:2/406: symlink d8/d60/l8a 0 2024-04-22T23:46:08.920 INFO:tasks.workunit.client.1.smithi044.stdout:2/407: getdents d8/d19/d36/d46/d63/d65/d6c 0 2024-04-22T23:46:08.921 INFO:tasks.workunit.client.1.smithi044.stdout:2/408: read - d8/d19/d36/d46/d63/d65/f85 zero size 2024-04-22T23:46:08.936 INFO:tasks.workunit.client.1.smithi044.stdout:2/409: write d8/d27/f56 [686335,92262] 0 2024-04-22T23:46:09.059 INFO:tasks.workunit.client.1.smithi044.stdout:7/434: sync 2024-04-22T23:46:09.060 INFO:tasks.workunit.client.1.smithi044.stdout:7/435: symlink dd/d98/l9d 0 2024-04-22T23:46:09.061 INFO:tasks.workunit.client.1.smithi044.stdout:7/436: chown dd/d55/d5e/d60/d68/f73 99947591 1 2024-04-22T23:46:09.063 INFO:tasks.workunit.client.1.smithi044.stdout:7/437: write dd/d1a/d61/f96 [779764,121552] 0 2024-04-22T23:46:09.064 INFO:tasks.workunit.client.1.smithi044.stdout:7/438: write dd/ff [3844081,7373] 0 2024-04-22T23:46:09.070 INFO:tasks.workunit.client.1.smithi044.stdout:7/439: dwrite dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:09.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/440: stat dd/d55/l72 0 2024-04-22T23:46:09.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/441: fdatasync dd/d1a/f8e 0 2024-04-22T23:46:09.072 INFO:tasks.workunit.client.1.smithi044.stdout:7/442: fdatasync dd/d1a/d3c/f49 0 2024-04-22T23:46:09.090 INFO:tasks.workunit.client.1.smithi044.stdout:7/443: dread dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:09.095 INFO:tasks.workunit.client.1.smithi044.stdout:7/444: rename dd/l2d to dd/d98/l9e 0 2024-04-22T23:46:09.096 INFO:tasks.workunit.client.1.smithi044.stdout:7/445: chown fc 38 1 2024-04-22T23:46:09.099 INFO:tasks.workunit.client.1.smithi044.stdout:7/446: rmdir dd/d1a/d61/d9b 0 2024-04-22T23:46:09.111 INFO:tasks.workunit.client.1.smithi044.stdout:7/447: dwrite fc [0,4194304] 0 2024-04-22T23:46:09.115 INFO:tasks.workunit.client.1.smithi044.stdout:7/448: mknod dd/d27/c9f 0 2024-04-22T23:46:09.123 INFO:tasks.workunit.client.1.smithi044.stdout:7/449: creat dd/d1f/fa0 x:0 0 0 2024-04-22T23:46:09.124 INFO:tasks.workunit.client.1.smithi044.stdout:7/450: write dd/d1a/f90 [42589,43623] 0 2024-04-22T23:46:09.126 INFO:tasks.workunit.client.1.smithi044.stdout:7/451: creat dd/d1a/d3c/fa1 x:0 0 0 2024-04-22T23:46:09.133 INFO:tasks.workunit.client.1.smithi044.stdout:7/452: dwrite dd/d55/f58 [0,4194304] 0 2024-04-22T23:46:09.137 INFO:tasks.workunit.client.1.smithi044.stdout:7/453: creat dd/d1f/d24/fa2 x:0 0 0 2024-04-22T23:46:09.141 INFO:tasks.workunit.client.1.smithi044.stdout:7/454: creat dd/d55/d5e/d60/d68/d94/fa3 x:0 0 0 2024-04-22T23:46:09.142 INFO:tasks.workunit.client.1.smithi044.stdout:7/455: chown dd/d98 23276 1 2024-04-22T23:46:09.149 INFO:tasks.workunit.client.1.smithi044.stdout:7/456: dwrite dd/d1f/f66 [0,4194304] 0 2024-04-22T23:46:09.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/457: dwrite dd/d27/f5b [0,4194304] 0 2024-04-22T23:46:09.160 INFO:tasks.workunit.client.1.smithi044.stdout:7/458: dread - dd/d55/d5e/d60/d75/f92 zero size 2024-04-22T23:46:09.161 INFO:tasks.workunit.client.1.smithi044.stdout:7/459: read dd/d27/f46 [573060,129280] 0 2024-04-22T23:46:09.164 INFO:tasks.workunit.client.1.smithi044.stdout:7/460: mknod dd/d1f/d24/ca4 0 2024-04-22T23:46:09.170 INFO:tasks.workunit.client.1.smithi044.stdout:7/461: rename dd/d1f/l35 to dd/d1f/d24/la5 0 2024-04-22T23:46:09.172 INFO:tasks.workunit.client.1.smithi044.stdout:7/462: creat dd/d1a/d61/fa6 x:0 0 0 2024-04-22T23:46:09.182 INFO:tasks.workunit.client.1.smithi044.stdout:7/463: dwrite fa [0,4194304] 0 2024-04-22T23:46:09.183 INFO:tasks.workunit.client.1.smithi044.stdout:7/464: chown fc 11 1 2024-04-22T23:46:09.183 INFO:tasks.workunit.client.1.smithi044.stdout:7/465: chown f8 707 1 2024-04-22T23:46:09.186 INFO:tasks.workunit.client.1.smithi044.stdout:7/466: symlink dd/d1f/la7 0 2024-04-22T23:46:09.187 INFO:tasks.workunit.client.1.smithi044.stdout:7/467: fdatasync dd/d55/f58 0 2024-04-22T23:46:09.189 INFO:tasks.workunit.client.1.smithi044.stdout:0/469: sync 2024-04-22T23:46:09.191 INFO:tasks.workunit.client.1.smithi044.stdout:7/468: creat dd/d1a/d3c/fa8 x:0 0 0 2024-04-22T23:46:09.193 INFO:tasks.workunit.client.1.smithi044.stdout:7/469: creat dd/d79/d8c/fa9 x:0 0 0 2024-04-22T23:46:09.197 INFO:tasks.workunit.client.1.smithi044.stdout:7/470: rename l4 to dd/d1a/d3c/laa 0 2024-04-22T23:46:09.198 INFO:tasks.workunit.client.1.smithi044.stdout:0/470: link d0/d2/dd/d3e/d40/d43/f66 d0/d4c/d59/fb2 0 2024-04-22T23:46:09.200 INFO:tasks.workunit.client.1.smithi044.stdout:7/471: fdatasync dd/d1f/d24/f1e 0 2024-04-22T23:46:09.202 INFO:tasks.workunit.client.1.smithi044.stdout:0/471: creat d0/d4c/d59/fb3 x:0 0 0 2024-04-22T23:46:09.207 INFO:tasks.workunit.client.1.smithi044.stdout:7/472: dwrite fc [0,4194304] 0 2024-04-22T23:46:09.214 INFO:tasks.workunit.client.1.smithi044.stdout:7/473: mkdir dd/d55/d5e/d60/d68/d94/dab 0 2024-04-22T23:46:09.214 INFO:tasks.workunit.client.1.smithi044.stdout:7/474: chown dd/d1a/l69 42190649 1 2024-04-22T23:46:09.218 INFO:tasks.workunit.client.1.smithi044.stdout:7/475: symlink dd/d79/d8c/d9c/lac 0 2024-04-22T23:46:09.220 INFO:tasks.workunit.client.1.smithi044.stdout:7/476: symlink dd/d79/d8c/lad 0 2024-04-22T23:46:09.223 INFO:tasks.workunit.client.1.smithi044.stdout:7/477: mknod dd/d55/d5e/d60/d68/d94/cae 0 2024-04-22T23:46:09.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/472: dread d0/d2e/d2f/f4d [4194304,4194304] 0 2024-04-22T23:46:09.225 INFO:tasks.workunit.client.1.smithi044.stdout:7/478: dread dd/d1a/f90 [0,4194304] 0 2024-04-22T23:46:09.227 INFO:tasks.workunit.client.1.smithi044.stdout:7/479: chown dd/d1f/l42 6 1 2024-04-22T23:46:09.230 INFO:tasks.workunit.client.1.smithi044.stdout:0/473: fsync d0/d2/dd/f10 0 2024-04-22T23:46:09.232 INFO:tasks.workunit.client.1.smithi044.stdout:5/446: sync 2024-04-22T23:46:09.232 INFO:tasks.workunit.client.1.smithi044.stdout:6/372: sync 2024-04-22T23:46:09.232 INFO:tasks.workunit.client.1.smithi044.stdout:3/442: sync 2024-04-22T23:46:09.232 INFO:tasks.workunit.client.1.smithi044.stdout:8/498: sync 2024-04-22T23:46:09.232 INFO:tasks.workunit.client.1.smithi044.stdout:1/468: sync 2024-04-22T23:46:09.232 INFO:tasks.workunit.client.1.smithi044.stdout:2/410: sync 2024-04-22T23:46:09.233 INFO:tasks.workunit.client.1.smithi044.stdout:4/491: sync 2024-04-22T23:46:09.233 INFO:tasks.workunit.client.1.smithi044.stdout:9/306: sync 2024-04-22T23:46:09.234 INFO:tasks.workunit.client.1.smithi044.stdout:0/474: fdatasync d0/d2/dd/f1a 0 2024-04-22T23:46:09.238 INFO:tasks.workunit.client.1.smithi044.stdout:2/411: dread - d8/d19/d36/d46/d63/d65/f85 zero size 2024-04-22T23:46:09.238 INFO:tasks.workunit.client.1.smithi044.stdout:8/499: stat db/d14/d72/f8f 0 2024-04-22T23:46:09.238 INFO:tasks.workunit.client.1.smithi044.stdout:4/492: fdatasync d5/d31/d24/d69/fa2 0 2024-04-22T23:46:09.242 INFO:tasks.workunit.client.1.smithi044.stdout:7/480: dwrite dd/d27/f6d [0,4194304] 0 2024-04-22T23:46:09.243 INFO:tasks.workunit.client.1.smithi044.stdout:1/469: mknod d1/d8/ca5 0 2024-04-22T23:46:09.243 INFO:tasks.workunit.client.1.smithi044.stdout:2/412: readlink d8/d19/d5d/l6a 0 2024-04-22T23:46:09.248 INFO:tasks.workunit.client.1.smithi044.stdout:1/470: write d1/d14/d95/f96 [916739,35147] 0 2024-04-22T23:46:09.249 INFO:tasks.workunit.client.1.smithi044.stdout:8/500: mkdir db/d14/d84/dbb 0 2024-04-22T23:46:09.249 INFO:tasks.workunit.client.1.smithi044.stdout:2/413: write d8/d19/f37 [3431801,106498] 0 2024-04-22T23:46:09.251 INFO:tasks.workunit.client.1.smithi044.stdout:7/481: readlink dd/l43 0 2024-04-22T23:46:09.254 INFO:tasks.workunit.client.1.smithi044.stdout:9/307: link d2/f19 d2/dc/d1d/d3f/f6e 0 2024-04-22T23:46:09.256 INFO:tasks.workunit.client.1.smithi044.stdout:8/501: write db/d68/f7a [181112,25545] 0 2024-04-22T23:46:09.258 INFO:tasks.workunit.client.1.smithi044.stdout:2/414: fsync d8/d27/f6f 0 2024-04-22T23:46:09.261 INFO:tasks.workunit.client.1.smithi044.stdout:0/475: rename d0/d2/c6c to d0/d4c/cb4 0 2024-04-22T23:46:09.261 INFO:tasks.workunit.client.1.smithi044.stdout:2/415: chown d8/f11 58 1 2024-04-22T23:46:09.262 INFO:tasks.workunit.client.1.smithi044.stdout:1/471: mknod d1/d8/d20/d47/d92/ca6 0 2024-04-22T23:46:09.264 INFO:tasks.workunit.client.1.smithi044.stdout:4/493: dwrite d5/d31/d24/d38/d6a/f89 [0,4194304] 0 2024-04-22T23:46:09.267 INFO:tasks.workunit.client.1.smithi044.stdout:7/482: creat dd/d79/d8c/d9c/faf x:0 0 0 2024-04-22T23:46:09.267 INFO:tasks.workunit.client.1.smithi044.stdout:3/443: dread d9/db/d15/d25/f3b [0,4194304] 0 2024-04-22T23:46:09.272 INFO:tasks.workunit.client.1.smithi044.stdout:2/416: creat d8/d60/f8b x:0 0 0 2024-04-22T23:46:09.273 INFO:tasks.workunit.client.1.smithi044.stdout:6/373: dread d5/d3f/d4c/d23/f29 [0,4194304] 0 2024-04-22T23:46:09.278 INFO:tasks.workunit.client.1.smithi044.stdout:4/494: write d5/d31/d44/d20/d22/d8d/f97 [1807139,22837] 0 2024-04-22T23:46:09.280 INFO:tasks.workunit.client.1.smithi044.stdout:9/308: dwrite d2/d5/f7 [0,4194304] 0 2024-04-22T23:46:09.280 INFO:tasks.workunit.client.1.smithi044.stdout:8/502: dwrite db/d6c/fa0 [0,4194304] 0 2024-04-22T23:46:09.288 INFO:tasks.workunit.client.1.smithi044.stdout:3/444: symlink d9/db/d33/l93 0 2024-04-22T23:46:09.288 INFO:tasks.workunit.client.1.smithi044.stdout:2/417: write d8/d19/d36/d46/d63/f71 [391287,109035] 0 2024-04-22T23:46:09.288 INFO:tasks.workunit.client.1.smithi044.stdout:5/447: truncate d15/d16/d1d/d22/d4f/f59 6095792 0 2024-04-22T23:46:09.288 INFO:tasks.workunit.client.1.smithi044.stdout:1/472: dread d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:09.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/473: readlink d1/d8/l13 0 2024-04-22T23:46:09.294 INFO:tasks.workunit.client.1.smithi044.stdout:8/503: creat db/d68/d8b/da7/fbc x:0 0 0 2024-04-22T23:46:09.296 INFO:tasks.workunit.client.1.smithi044.stdout:4/495: creat d5/d31/d24/d38/d84/fa4 x:0 0 0 2024-04-22T23:46:09.297 INFO:tasks.workunit.client.1.smithi044.stdout:5/448: write d15/d16/d1d/d22/d2a/d3a/d43/f7e [698275,93976] 0 2024-04-22T23:46:09.299 INFO:tasks.workunit.client.1.smithi044.stdout:4/496: stat d5/d31/d44/d20/d22/f6f 0 2024-04-22T23:46:09.300 INFO:tasks.workunit.client.1.smithi044.stdout:1/474: symlink d1/d8/d20/d45/d5d/la7 0 2024-04-22T23:46:09.301 INFO:tasks.workunit.client.1.smithi044.stdout:3/445: rename d9/db/d15/d25/f3b to d9/db/d15/d54/d3c/f94 0 2024-04-22T23:46:09.301 INFO:tasks.workunit.client.1.smithi044.stdout:6/374: rmdir d5/dd/d67 39 2024-04-22T23:46:09.302 INFO:tasks.workunit.client.1.smithi044.stdout:8/504: dread - db/d14/d24/f35 zero size 2024-04-22T23:46:09.304 INFO:tasks.workunit.client.1.smithi044.stdout:3/446: chown d9/db/d33/d5d/d70/l79 23625587 1 2024-04-22T23:46:09.307 INFO:tasks.workunit.client.1.smithi044.stdout:5/449: mknod d15/d16/d1d/d22/c8b 0 2024-04-22T23:46:09.309 INFO:tasks.workunit.client.1.smithi044.stdout:1/475: mkdir d1/d8/d1c/d7b/d80/d4b/d9c/da8 0 2024-04-22T23:46:09.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/476: fdatasync d1/f7 0 2024-04-22T23:46:09.312 INFO:tasks.workunit.client.1.smithi044.stdout:6/375: symlink d5/d3f/d50/d79/l7e 0 2024-04-22T23:46:09.315 INFO:tasks.workunit.client.1.smithi044.stdout:0/476: dread d0/d2/dd/d3e/d40/d43/d7b/f55 [0,4194304] 0 2024-04-22T23:46:09.316 INFO:tasks.workunit.client.1.smithi044.stdout:5/450: dwrite d15/d5d/d64/d6f/f74 [0,4194304] 0 2024-04-22T23:46:09.318 INFO:tasks.workunit.client.1.smithi044.stdout:5/451: write d15/d16/d2c/f5a [148550,58644] 0 2024-04-22T23:46:09.322 INFO:tasks.workunit.client.1.smithi044.stdout:0/477: mknod d0/d2/dd/d3e/d40/d67/d7c/cb5 0 2024-04-22T23:46:09.326 INFO:tasks.workunit.client.1.smithi044.stdout:0/478: write d0/d2e/f53 [465284,1291] 0 2024-04-22T23:46:09.327 INFO:tasks.workunit.client.1.smithi044.stdout:0/479: chown d0/ce 7829132 1 2024-04-22T23:46:09.327 INFO:tasks.workunit.client.1.smithi044.stdout:1/477: rename d1/d8/d20/f7d to d1/d8/fa9 0 2024-04-22T23:46:09.328 INFO:tasks.workunit.client.1.smithi044.stdout:0/480: dread - d0/d2/d4b/f74 zero size 2024-04-22T23:46:09.329 INFO:tasks.workunit.client.1.smithi044.stdout:6/376: creat d5/dd/d28/d75/f7f x:0 0 0 2024-04-22T23:46:09.330 INFO:tasks.workunit.client.1.smithi044.stdout:0/481: write d0/d2e/d2f/d77/f93 [956999,75765] 0 2024-04-22T23:46:09.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/478: creat d1/d8/d20/d47/d92/faa x:0 0 0 2024-04-22T23:46:09.333 INFO:tasks.workunit.client.1.smithi044.stdout:0/482: chown d0/d2/dd/d3e/d40/d43/l95 0 1 2024-04-22T23:46:09.333 INFO:tasks.workunit.client.1.smithi044.stdout:0/483: stat d0/d2/l39 0 2024-04-22T23:46:09.338 INFO:tasks.workunit.client.1.smithi044.stdout:9/309: dread d2/dc/f27 [0,4194304] 0 2024-04-22T23:46:09.339 INFO:tasks.workunit.client.1.smithi044.stdout:0/484: creat d0/d60/fb6 x:0 0 0 2024-04-22T23:46:09.342 INFO:tasks.workunit.client.1.smithi044.stdout:2/418: dread d8/d19/d1c/f1d [0,4194304] 0 2024-04-22T23:46:09.344 INFO:tasks.workunit.client.1.smithi044.stdout:6/377: dwrite d5/dd/d28/f34 [0,4194304] 0 2024-04-22T23:46:09.345 INFO:tasks.workunit.client.1.smithi044.stdout:0/485: write d0/d4c/d59/fb3 [999552,106492] 0 2024-04-22T23:46:09.348 INFO:tasks.workunit.client.1.smithi044.stdout:5/452: dread d15/d16/d1d/f42 [0,4194304] 0 2024-04-22T23:46:09.351 INFO:tasks.workunit.client.1.smithi044.stdout:9/310: rename d2/d5/f5b to d2/dc/d1d/d34/d41/f6f 0 2024-04-22T23:46:09.352 INFO:tasks.workunit.client.1.smithi044.stdout:2/419: symlink d8/d19/d36/l8c 0 2024-04-22T23:46:09.355 INFO:tasks.workunit.client.1.smithi044.stdout:3/447: rmdir d9/db/d33 39 2024-04-22T23:46:09.358 INFO:tasks.workunit.client.1.smithi044.stdout:1/479: dread d1/d8/d1c/d7b/f65 [0,4194304] 0 2024-04-22T23:46:09.360 INFO:tasks.workunit.client.1.smithi044.stdout:0/486: mknod d0/d4c/d59/cb7 0 2024-04-22T23:46:09.365 INFO:tasks.workunit.client.1.smithi044.stdout:9/311: write d2/d5/f55 [470132,19262] 0 2024-04-22T23:46:09.368 INFO:tasks.workunit.client.1.smithi044.stdout:4/497: write d5/d31/d44/d20/d22/f62 [5052999,66691] 0 2024-04-22T23:46:09.369 INFO:tasks.workunit.client.1.smithi044.stdout:2/420: symlink d8/d19/d36/d46/d63/d65/l8d 0 2024-04-22T23:46:09.371 INFO:tasks.workunit.client.1.smithi044.stdout:9/312: dread d2/f3d [0,4194304] 0 2024-04-22T23:46:09.375 INFO:tasks.workunit.client.1.smithi044.stdout:8/505: truncate db/d14/d24/f43 3811812 0 2024-04-22T23:46:09.380 INFO:tasks.workunit.client.1.smithi044.stdout:2/421: creat d8/d19/d36/d46/d63/d65/d72/f8e x:0 0 0 2024-04-22T23:46:09.381 INFO:tasks.workunit.client.1.smithi044.stdout:3/448: rmdir d9/db/d33 39 2024-04-22T23:46:09.383 INFO:tasks.workunit.client.1.smithi044.stdout:3/449: write f8 [508087,24149] 0 2024-04-22T23:46:09.383 INFO:tasks.workunit.client.1.smithi044.stdout:5/453: dwrite d15/d5d/d64/d6f/f74 [4194304,4194304] 0 2024-04-22T23:46:09.388 INFO:tasks.workunit.client.1.smithi044.stdout:8/506: dwrite db/d14/f97 [0,4194304] 0 2024-04-22T23:46:09.390 INFO:tasks.workunit.client.1.smithi044.stdout:1/480: mkdir d1/d8/d34/da1/dab 0 2024-04-22T23:46:09.393 INFO:tasks.workunit.client.1.smithi044.stdout:6/378: write f1 [1459109,47899] 0 2024-04-22T23:46:09.402 INFO:tasks.workunit.client.1.smithi044.stdout:9/313: mkdir d2/d6/d42/d4b/d70 0 2024-04-22T23:46:09.405 INFO:tasks.workunit.client.1.smithi044.stdout:3/450: mkdir d9/db/d15/d54/d3c/d95 0 2024-04-22T23:46:09.406 INFO:tasks.workunit.client.1.smithi044.stdout:3/451: fdatasync d9/db/d15/d2e/f3f 0 2024-04-22T23:46:09.407 INFO:tasks.workunit.client.1.smithi044.stdout:3/452: chown d9/db/c44 1893689 1 2024-04-22T23:46:09.407 INFO:tasks.workunit.client.1.smithi044.stdout:0/487: rmdir d0/d2/dd/d3e/d40/d43/d79/d96 0 2024-04-22T23:46:09.409 INFO:tasks.workunit.client.1.smithi044.stdout:0/488: dread d0/d4c/f52 [0,4194304] 0 2024-04-22T23:46:09.410 INFO:tasks.workunit.client.1.smithi044.stdout:0/489: fdatasync d0/d2/d4b/f74 0 2024-04-22T23:46:09.411 INFO:tasks.workunit.client.1.smithi044.stdout:1/481: write d1/d8/fa9 [397592,52782] 0 2024-04-22T23:46:09.413 INFO:tasks.workunit.client.1.smithi044.stdout:4/498: rename d5/db/d16/l46 to d5/la5 0 2024-04-22T23:46:09.414 INFO:tasks.workunit.client.1.smithi044.stdout:6/379: creat d5/d30/f80 x:0 0 0 2024-04-22T23:46:09.415 INFO:tasks.workunit.client.1.smithi044.stdout:4/499: write d5/d31/f76 [3172926,18336] 0 2024-04-22T23:46:09.415 INFO:tasks.workunit.client.1.smithi044.stdout:6/380: stat d5/d3f/d4c/d40/d53 0 2024-04-22T23:46:09.423 INFO:tasks.workunit.client.1.smithi044.stdout:8/507: link db/d14/d24/f75 db/d14/d84/dbb/fbd 0 2024-04-22T23:46:09.425 INFO:tasks.workunit.client.1.smithi044.stdout:0/490: symlink d0/d2/dd/d3e/d40/d43/lb8 0 2024-04-22T23:46:09.430 INFO:tasks.workunit.client.1.smithi044.stdout:1/482: dread d1/d8/f1a [0,4194304] 0 2024-04-22T23:46:09.434 INFO:tasks.workunit.client.1.smithi044.stdout:4/500: mkdir d5/d31/d24/da6 0 2024-04-22T23:46:09.436 INFO:tasks.workunit.client.1.smithi044.stdout:6/381: creat d5/d3f/d50/d79/f81 x:0 0 0 2024-04-22T23:46:09.444 INFO:tasks.workunit.client.1.smithi044.stdout:1/483: symlink d1/d8/d20/d47/lac 0 2024-04-22T23:46:09.447 INFO:tasks.workunit.client.1.smithi044.stdout:9/314: dwrite d2/f19 [0,4194304] 0 2024-04-22T23:46:09.448 INFO:tasks.workunit.client.1.smithi044.stdout:4/501: dwrite d5/d31/f11 [0,4194304] 0 2024-04-22T23:46:09.455 INFO:tasks.workunit.client.1.smithi044.stdout:3/453: link d9/d12/f92 d9/db/d33/f96 0 2024-04-22T23:46:09.457 INFO:tasks.workunit.client.1.smithi044.stdout:2/422: rename d8/d19/d1c/c76 to d8/d27/c8f 0 2024-04-22T23:46:09.460 INFO:tasks.workunit.client.1.smithi044.stdout:3/454: chown d9/db/d15/d54/d3c/f81 64701 1 2024-04-22T23:46:09.460 INFO:tasks.workunit.client.1.smithi044.stdout:6/382: dwrite d5/d3f/d4c/d40/f71 [0,4194304] 0 2024-04-22T23:46:09.463 INFO:tasks.workunit.client.1.smithi044.stdout:1/484: mkdir d1/d8/d20/d45/d75/d77/dad 0 2024-04-22T23:46:09.466 INFO:tasks.workunit.client.1.smithi044.stdout:6/383: dread - d5/d3f/d50/f6d zero size 2024-04-22T23:46:09.467 INFO:tasks.workunit.client.1.smithi044.stdout:0/491: dread d0/d2e/d2f/f51 [0,4194304] 0 2024-04-22T23:46:09.470 INFO:tasks.workunit.client.1.smithi044.stdout:5/454: rename d15/d16/d1d/d22/f23 to d15/d16/d1d/d22/d2a/d3a/d43/d80/f8c 0 2024-04-22T23:46:09.472 INFO:tasks.workunit.client.1.smithi044.stdout:3/455: chown d9/db/d15/d54/f4b 633121 1 2024-04-22T23:46:09.472 INFO:tasks.workunit.client.1.smithi044.stdout:2/423: fsync d8/d19/d1c/f1d 0 2024-04-22T23:46:09.473 INFO:tasks.workunit.client.1.smithi044.stdout:8/508: dread db/d14/d24/f26 [4194304,4194304] 0 2024-04-22T23:46:09.474 INFO:tasks.workunit.client.1.smithi044.stdout:4/502: dwrite d5/d31/d24/f72 [0,4194304] 0 2024-04-22T23:46:09.476 INFO:tasks.workunit.client.1.smithi044.stdout:9/315: symlink d2/dc/d1d/d34/d38/d5c/d69/l71 0 2024-04-22T23:46:09.479 INFO:tasks.workunit.client.1.smithi044.stdout:1/485: rmdir d1/d8/d20/d47/d4a/d60 39 2024-04-22T23:46:09.480 INFO:tasks.workunit.client.1.smithi044.stdout:6/384: readlink d5/dd/d67/l1f 0 2024-04-22T23:46:09.482 INFO:tasks.workunit.client.1.smithi044.stdout:9/316: stat d2/dc/d1d/d34/d67 0 2024-04-22T23:46:09.484 INFO:tasks.workunit.client.1.smithi044.stdout:2/424: symlink d8/d19/d36/d46/d54/l90 0 2024-04-22T23:46:09.485 INFO:tasks.workunit.client.1.smithi044.stdout:8/509: symlink db/d52/d9c/lbe 0 2024-04-22T23:46:09.487 INFO:tasks.workunit.client.1.smithi044.stdout:4/503: unlink d5/d31/d24/d38/d6a/l85 0 2024-04-22T23:46:09.490 INFO:tasks.workunit.client.1.smithi044.stdout:1/486: mkdir d1/d8/d20/dae 0 2024-04-22T23:46:09.492 INFO:tasks.workunit.client.1.smithi044.stdout:6/385: dwrite d5/dd/d67/f5d [0,4194304] 0 2024-04-22T23:46:09.493 INFO:tasks.workunit.client.1.smithi044.stdout:5/455: mkdir d15/d3c/d65/d8d 0 2024-04-22T23:46:09.494 INFO:tasks.workunit.client.1.smithi044.stdout:8/510: dread db/d68/f7a [0,4194304] 0 2024-04-22T23:46:09.497 INFO:tasks.workunit.client.1.smithi044.stdout:7/483: sync 2024-04-22T23:46:09.507 INFO:tasks.workunit.client.1.smithi044.stdout:7/484: dwrite dd/d1f/d24/fa2 [0,4194304] 0 2024-04-22T23:46:09.513 INFO:tasks.workunit.client.1.smithi044.stdout:6/386: read d5/dd/f1e [617487,69586] 0 2024-04-22T23:46:09.514 INFO:tasks.workunit.client.1.smithi044.stdout:8/511: rmdir db/d14/d72 39 2024-04-22T23:46:09.516 INFO:tasks.workunit.client.1.smithi044.stdout:1/487: rename d1/d8/d1c/d23/d48/l55 to d1/d8/d1c/d23/d48/d99/laf 0 2024-04-22T23:46:09.522 INFO:tasks.workunit.client.1.smithi044.stdout:3/456: link d9/c28 d9/db/d22/d38/c97 0 2024-04-22T23:46:09.523 INFO:tasks.workunit.client.1.smithi044.stdout:0/492: write d0/d2/fa [3692460,86609] 0 2024-04-22T23:46:09.530 INFO:tasks.workunit.client.1.smithi044.stdout:2/425: link d8/d19/d1c/c75 d8/d19/d36/d44/c91 0 2024-04-22T23:46:09.532 INFO:tasks.workunit.client.1.smithi044.stdout:7/485: link dd/ff dd/d1a/d61/fb0 0 2024-04-22T23:46:09.533 INFO:tasks.workunit.client.1.smithi044.stdout:6/387: truncate d5/dd/f27 618020 0 2024-04-22T23:46:09.534 INFO:tasks.workunit.client.1.smithi044.stdout:8/512: mknod db/d14/d84/dbb/cbf 0 2024-04-22T23:46:09.535 INFO:tasks.workunit.client.1.smithi044.stdout:4/504: dwrite d5/d52/f60 [0,4194304] 0 2024-04-22T23:46:09.535 INFO:tasks.workunit.client.1.smithi044.stdout:6/388: stat d5/d3f/d4c/f3a 0 2024-04-22T23:46:09.537 INFO:tasks.workunit.client.1.smithi044.stdout:7/486: write dd/d1a/d3c/fa1 [552598,53281] 0 2024-04-22T23:46:09.538 INFO:tasks.workunit.client.1.smithi044.stdout:4/505: stat d5/d31/d24/d38 0 2024-04-22T23:46:09.543 INFO:tasks.workunit.client.1.smithi044.stdout:5/456: getdents d15/d16/d1d/d22/d2a/d3a/d43/d80 0 2024-04-22T23:46:09.544 INFO:tasks.workunit.client.1.smithi044.stdout:3/457: creat d9/db/d22/f98 x:0 0 0 2024-04-22T23:46:09.544 INFO:tasks.workunit.client.1.smithi044.stdout:8/513: mkdir db/d14/d84/dc0 0 2024-04-22T23:46:09.544 INFO:tasks.workunit.client.1.smithi044.stdout:6/389: rename d5/f38 to d5/d30/d73/f82 0 2024-04-22T23:46:09.546 INFO:tasks.workunit.client.1.smithi044.stdout:3/458: chown d9/db/d15/d25/d4c/d86 11 1 2024-04-22T23:46:09.548 INFO:tasks.workunit.client.1.smithi044.stdout:9/317: truncate d2/dc/f27 3364020 0 2024-04-22T23:46:09.550 INFO:tasks.workunit.client.1.smithi044.stdout:0/493: creat d0/d4c/fb9 x:0 0 0 2024-04-22T23:46:09.552 INFO:tasks.workunit.client.1.smithi044.stdout:4/506: dwrite d5/db/d16/fa0 [0,4194304] 0 2024-04-22T23:46:09.563 INFO:tasks.workunit.client.1.smithi044.stdout:0/494: dwrite d0/d2/dd/d3e/d40/f73 [4194304,4194304] 0 2024-04-22T23:46:09.565 INFO:tasks.workunit.client.1.smithi044.stdout:0/495: readlink d0/d2/dd/d3e/l7e 0 2024-04-22T23:46:09.572 INFO:tasks.workunit.client.1.smithi044.stdout:9/318: rename d2/dc/d1d/d34/d67 to d2/dc/d1d/d34/d38/d4a/d72 0 2024-04-22T23:46:09.574 INFO:tasks.workunit.client.1.smithi044.stdout:6/390: mkdir d5/d3f/d4c/d83 0 2024-04-22T23:46:09.577 INFO:tasks.workunit.client.1.smithi044.stdout:2/426: dwrite d8/d19/d1c/f33 [0,4194304] 0 2024-04-22T23:46:09.583 INFO:tasks.workunit.client.1.smithi044.stdout:4/507: dread d5/d31/d44/d20/d22/f62 [0,4194304] 0 2024-04-22T23:46:09.585 INFO:tasks.workunit.client.1.smithi044.stdout:2/427: write d8/d19/d36/d46/d54/f73 [2359298,74135] 0 2024-04-22T23:46:09.590 INFO:tasks.workunit.client.1.smithi044.stdout:7/487: dwrite dd/d27/f3e [0,4194304] 0 2024-04-22T23:46:09.601 INFO:tasks.workunit.client.1.smithi044.stdout:9/319: rename d2/d6/d42/d4b/d70 to d2/d6/d4d/d73 0 2024-04-22T23:46:09.608 INFO:tasks.workunit.client.1.smithi044.stdout:4/508: symlink d5/db/d16/d4d/la7 0 2024-04-22T23:46:09.608 INFO:tasks.workunit.client.1.smithi044.stdout:3/459: getdents d9/db/d15/d2e 0 2024-04-22T23:46:09.609 INFO:tasks.workunit.client.1.smithi044.stdout:4/509: readlink d5/db/d16/l41 0 2024-04-22T23:46:09.614 INFO:tasks.workunit.client.1.smithi044.stdout:7/488: rmdir dd 39 2024-04-22T23:46:09.617 INFO:tasks.workunit.client.1.smithi044.stdout:8/514: link db/d14/d72/ca6 db/d68/d8b/da7/cc1 0 2024-04-22T23:46:09.622 INFO:tasks.workunit.client.1.smithi044.stdout:4/510: creat d5/d31/d44/d20/d22/d8d/fa8 x:0 0 0 2024-04-22T23:46:09.625 INFO:tasks.workunit.client.1.smithi044.stdout:8/515: unlink db/c8a 0 2024-04-22T23:46:09.627 INFO:tasks.workunit.client.1.smithi044.stdout:0/496: dwrite d0/d4c/d59/f76 [4194304,4194304] 0 2024-04-22T23:46:09.629 INFO:tasks.workunit.client.1.smithi044.stdout:2/428: link d8/d19/d36/f3d d8/f92 0 2024-04-22T23:46:09.632 INFO:tasks.workunit.client.1.smithi044.stdout:4/511: dwrite d5/db/f21 [0,4194304] 0 2024-04-22T23:46:09.634 INFO:tasks.workunit.client.1.smithi044.stdout:3/460: symlink d9/db/d15/d2e/d48/d8f/l99 0 2024-04-22T23:46:09.635 INFO:tasks.workunit.client.1.smithi044.stdout:8/516: dread db/f48 [0,4194304] 0 2024-04-22T23:46:09.636 INFO:tasks.workunit.client.1.smithi044.stdout:8/517: readlink db/d52/l5f 0 2024-04-22T23:46:09.636 INFO:tasks.workunit.client.1.smithi044.stdout:7/489: readlink dd/d1f/d24/l29 0 2024-04-22T23:46:09.638 INFO:tasks.workunit.client.1.smithi044.stdout:2/429: dread d8/d19/d5d/f7c [0,4194304] 0 2024-04-22T23:46:09.638 INFO:tasks.workunit.client.1.smithi044.stdout:7/490: dread - dd/d1a/d61/fa6 zero size 2024-04-22T23:46:09.644 INFO:tasks.workunit.client.1.smithi044.stdout:7/491: dread dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:09.650 INFO:tasks.workunit.client.1.smithi044.stdout:4/512: mknod d5/d42/ca9 0 2024-04-22T23:46:09.651 INFO:tasks.workunit.client.1.smithi044.stdout:3/461: creat d9/db/d15/d2e/d48/f9a x:0 0 0 2024-04-22T23:46:09.653 INFO:tasks.workunit.client.1.smithi044.stdout:3/462: fdatasync d9/db/d22/f56 0 2024-04-22T23:46:09.653 INFO:tasks.workunit.client.1.smithi044.stdout:8/518: creat db/d52/d9c/dad/fc2 x:0 0 0 2024-04-22T23:46:09.657 INFO:tasks.workunit.client.1.smithi044.stdout:7/492: creat dd/d55/fb1 x:0 0 0 2024-04-22T23:46:09.659 INFO:tasks.workunit.client.1.smithi044.stdout:4/513: mkdir d5/d31/d44/d7e/daa 0 2024-04-22T23:46:09.659 INFO:tasks.workunit.client.1.smithi044.stdout:4/514: chown d5/d31/d24/d38/d84 568 1 2024-04-22T23:46:09.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/463: unlink d9/db/d15/d25/c80 0 2024-04-22T23:46:09.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/519: rename db/f3d to db/d68/fc3 0 2024-04-22T23:46:09.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/520: fsync f5 0 2024-04-22T23:46:09.672 INFO:tasks.workunit.client.1.smithi044.stdout:4/515: rename d5/d31/d24/d38/d6a/f83 to d5/d31/d24/d38/d6a/d75/fab 0 2024-04-22T23:46:09.673 INFO:tasks.workunit.client.1.smithi044.stdout:4/516: write d5/d31/d24/d69/d90/f88 [573219,47483] 0 2024-04-22T23:46:09.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/521: unlink db/d52/d9c/dad/fc2 0 2024-04-22T23:46:09.677 INFO:tasks.workunit.client.1.smithi044.stdout:4/517: symlink d5/d31/d24/lac 0 2024-04-22T23:46:09.681 INFO:tasks.workunit.client.1.smithi044.stdout:4/518: mkdir d5/db/d16/d4d/dad 0 2024-04-22T23:46:09.683 INFO:tasks.workunit.client.1.smithi044.stdout:9/320: write d2/dc/d1d/d34/d38/d5c/f6d [1708666,12986] 0 2024-04-22T23:46:09.685 INFO:tasks.workunit.client.1.smithi044.stdout:4/519: mknod d5/d31/cae 0 2024-04-22T23:46:09.693 INFO:tasks.workunit.client.1.smithi044.stdout:9/321: dread d2/d5/f50 [0,4194304] 0 2024-04-22T23:46:09.697 INFO:tasks.workunit.client.1.smithi044.stdout:9/322: creat d2/d6/f74 x:0 0 0 2024-04-22T23:46:09.699 INFO:tasks.workunit.client.1.smithi044.stdout:4/520: getdents d5/d31/d44/d20/d22/d8d 0 2024-04-22T23:46:09.704 INFO:tasks.workunit.client.1.smithi044.stdout:4/521: rmdir d5/db/d34 39 2024-04-22T23:46:09.708 INFO:tasks.workunit.client.1.smithi044.stdout:4/522: dread d5/d31/d44/d20/d22/f37 [0,4194304] 0 2024-04-22T23:46:09.708 INFO:tasks.workunit.client.1.smithi044.stdout:9/323: rmdir d2/d6/d4d/d73 0 2024-04-22T23:46:09.712 INFO:tasks.workunit.client.1.smithi044.stdout:9/324: chown d2/dc/l5f 856 1 2024-04-22T23:46:09.735 INFO:tasks.workunit.client.1.smithi044.stdout:4/523: creat d5/db/d16/d4d/dad/faf x:0 0 0 2024-04-22T23:46:09.735 INFO:tasks.workunit.client.1.smithi044.stdout:4/524: readlink d5/db/d16/l41 0 2024-04-22T23:46:09.741 INFO:tasks.workunit.client.1.smithi044.stdout:4/525: readlink l4 0 2024-04-22T23:46:09.743 INFO:tasks.workunit.client.1.smithi044.stdout:4/526: read d5/d31/f11 [3540890,102248] 0 2024-04-22T23:46:09.745 INFO:tasks.workunit.client.1.smithi044.stdout:0/497: truncate d0/d2/d4b/fa3 587743 0 2024-04-22T23:46:09.745 INFO:tasks.workunit.client.1.smithi044.stdout:9/325: dwrite d2/dc/d1d/d34/d41/f6f [0,4194304] 0 2024-04-22T23:46:09.747 INFO:tasks.workunit.client.1.smithi044.stdout:9/326: write d2/d5/f7 [3273933,108010] 0 2024-04-22T23:46:09.751 INFO:tasks.workunit.client.1.smithi044.stdout:0/498: fsync d0/d2e/d33/f3a 0 2024-04-22T23:46:09.753 INFO:tasks.workunit.client.1.smithi044.stdout:0/499: write d0/d2/fa [2776645,68999] 0 2024-04-22T23:46:09.760 INFO:tasks.workunit.client.1.smithi044.stdout:0/500: creat d0/d2e/d2f/d77/fba x:0 0 0 2024-04-22T23:46:09.760 INFO:tasks.workunit.client.1.smithi044.stdout:3/464: write d9/db/d15/d2e/f39 [223448,112467] 0 2024-04-22T23:46:09.764 INFO:tasks.workunit.client.1.smithi044.stdout:0/501: fsync d0/d2/dd/f36 0 2024-04-22T23:46:09.768 INFO:tasks.workunit.client.1.smithi044.stdout:3/465: dwrite d9/db/d15/f85 [0,4194304] 0 2024-04-22T23:46:09.770 INFO:tasks.workunit.client.1.smithi044.stdout:0/502: mkdir d0/d2/dd/d3e/d40/d43/d79/dbb 0 2024-04-22T23:46:09.774 INFO:tasks.workunit.client.1.smithi044.stdout:0/503: rmdir d0/d2/dd/d3e/d40/d43/d7b 39 2024-04-22T23:46:09.774 INFO:tasks.workunit.client.1.smithi044.stdout:0/504: chown d0/d2e/d2f 7 1 2024-04-22T23:46:09.780 INFO:tasks.workunit.client.1.smithi044.stdout:0/505: rmdir d0/d2/dd/d3e/d40/d43/d7b 39 2024-04-22T23:46:09.783 INFO:tasks.workunit.client.1.smithi044.stdout:8/522: dwrite db/f3c [0,4194304] 0 2024-04-22T23:46:09.787 INFO:tasks.workunit.client.1.smithi044.stdout:8/523: mkdir db/d52/d9c/dad/dc4 0 2024-04-22T23:46:09.789 INFO:tasks.workunit.client.1.smithi044.stdout:8/524: symlink db/d52/lc5 0 2024-04-22T23:46:09.801 INFO:tasks.workunit.client.1.smithi044.stdout:9/327: truncate d2/f3 8326505 0 2024-04-22T23:46:09.804 INFO:tasks.workunit.client.1.smithi044.stdout:4/527: dwrite d5/d42/f61 [0,4194304] 0 2024-04-22T23:46:09.805 INFO:tasks.workunit.client.1.smithi044.stdout:3/466: write d9/f2a [2000868,81711] 0 2024-04-22T23:46:09.807 INFO:tasks.workunit.client.1.smithi044.stdout:0/506: dread d0/d2/dd/d3e/d40/d43/f9b [0,4194304] 0 2024-04-22T23:46:09.807 INFO:tasks.workunit.client.1.smithi044.stdout:9/328: mkdir d2/dc/d1d/d34/d38/d5c/d69/d75 0 2024-04-22T23:46:09.816 INFO:tasks.workunit.client.1.smithi044.stdout:8/525: dread db/f27 [0,4194304] 0 2024-04-22T23:46:09.819 INFO:tasks.workunit.client.1.smithi044.stdout:0/507: dwrite d0/d2e/d2f/d77/fba [0,4194304] 0 2024-04-22T23:46:09.820 INFO:tasks.workunit.client.1.smithi044.stdout:0/508: write d0/d2e/d2f/f4d [696401,4085] 0 2024-04-22T23:46:09.824 INFO:tasks.workunit.client.1.smithi044.stdout:4/528: creat d5/fb0 x:0 0 0 2024-04-22T23:46:09.829 INFO:tasks.workunit.client.1.smithi044.stdout:3/467: creat d9/f9b x:0 0 0 2024-04-22T23:46:09.831 INFO:tasks.workunit.client.1.smithi044.stdout:0/509: mkdir d0/d2e/dbc 0 2024-04-22T23:46:09.831 INFO:tasks.workunit.client.1.smithi044.stdout:9/329: dwrite d2/d6/f43 [0,4194304] 0 2024-04-22T23:46:09.832 INFO:tasks.workunit.client.1.smithi044.stdout:3/468: write d9/db/d15/d2e/d48/f72 [1008155,91475] 0 2024-04-22T23:46:09.834 INFO:tasks.workunit.client.1.smithi044.stdout:3/469: stat d9/d14/c4e 0 2024-04-22T23:46:09.837 INFO:tasks.workunit.client.1.smithi044.stdout:0/510: mkdir d0/d4c/dbd 0 2024-04-22T23:46:09.838 INFO:tasks.workunit.client.1.smithi044.stdout:9/330: creat d2/dc/d1d/d34/d38/d5c/f76 x:0 0 0 2024-04-22T23:46:09.839 INFO:tasks.workunit.client.1.smithi044.stdout:0/511: write d0/d2e/d2f/d61/f94 [116179,128644] 0 2024-04-22T23:46:09.840 INFO:tasks.workunit.client.1.smithi044.stdout:3/470: mknod d9/db/d15/d25/d4c/d86/c9c 0 2024-04-22T23:46:09.842 INFO:tasks.workunit.client.1.smithi044.stdout:0/512: fdatasync d0/d2/dd/d3e/d40/f73 0 2024-04-22T23:46:09.844 INFO:tasks.workunit.client.1.smithi044.stdout:9/331: unlink d2/dc/d1d/d34/d38/d5c/f6d 0 2024-04-22T23:46:09.846 INFO:tasks.workunit.client.1.smithi044.stdout:3/471: symlink d9/db/d15/d54/l9d 0 2024-04-22T23:46:09.849 INFO:tasks.workunit.client.1.smithi044.stdout:9/332: mknod d2/d6/c77 0 2024-04-22T23:46:09.854 INFO:tasks.workunit.client.1.smithi044.stdout:8/526: dread db/ff [0,4194304] 0 2024-04-22T23:46:09.857 INFO:tasks.workunit.client.1.smithi044.stdout:3/472: dwrite d9/db/d15/d54/f4b [0,4194304] 0 2024-04-22T23:46:09.857 INFO:tasks.workunit.client.1.smithi044.stdout:3/473: stat d9/d12/f92 0 2024-04-22T23:46:09.863 INFO:tasks.workunit.client.1.smithi044.stdout:8/527: fdatasync db/d14/d24/f30 0 2024-04-22T23:46:09.866 INFO:tasks.workunit.client.1.smithi044.stdout:3/474: dwrite d9/db/d15/d25/d5b/d52/f8b [0,4194304] 0 2024-04-22T23:46:09.871 INFO:tasks.workunit.client.1.smithi044.stdout:9/333: getdents d2/dc 0 2024-04-22T23:46:09.873 INFO:tasks.workunit.client.1.smithi044.stdout:8/528: creat db/d52/d9c/fc6 x:0 0 0 2024-04-22T23:46:09.876 INFO:tasks.workunit.client.1.smithi044.stdout:3/475: rmdir d9/db/d33/d5d/d70 39 2024-04-22T23:46:09.877 INFO:tasks.workunit.client.1.smithi044.stdout:0/513: dwrite d0/d2/dd/d3e/f6e [0,4194304] 0 2024-04-22T23:46:09.879 INFO:tasks.workunit.client.1.smithi044.stdout:9/334: chown d2/d5/f50 5520 1 2024-04-22T23:46:09.879 INFO:tasks.workunit.client.1.smithi044.stdout:0/514: write d0/d2e/d2f/d77/f9d [943456,6727] 0 2024-04-22T23:46:09.881 INFO:tasks.workunit.client.1.smithi044.stdout:0/515: chown d0/d2/f8e 32351 1 2024-04-22T23:46:09.885 INFO:tasks.workunit.client.1.smithi044.stdout:3/476: mkdir d9/db/d22/d38/d9e 0 2024-04-22T23:46:09.888 INFO:tasks.workunit.client.1.smithi044.stdout:9/335: mkdir d2/dc/d1d/d34/d38/d5c/d69/d75/d78 0 2024-04-22T23:46:09.892 INFO:tasks.workunit.client.1.smithi044.stdout:9/336: stat d2/d5/f1f 0 2024-04-22T23:46:09.894 INFO:tasks.workunit.client.1.smithi044.stdout:3/477: read d9/f2a [3675099,130826] 0 2024-04-22T23:46:09.896 INFO:tasks.workunit.client.1.smithi044.stdout:0/516: link d0/d2/dd/d3e/d40/d43/d7b/f26 d0/d4c/fbe 0 2024-04-22T23:46:09.897 INFO:tasks.workunit.client.1.smithi044.stdout:3/478: chown d9/db/d15/d54/d3c/l8e 1116 1 2024-04-22T23:46:09.897 INFO:tasks.workunit.client.1.smithi044.stdout:8/529: dwrite db/f34 [0,4194304] 0 2024-04-22T23:46:09.898 INFO:tasks.workunit.client.1.smithi044.stdout:1/488: sync 2024-04-22T23:46:09.899 INFO:tasks.workunit.client.1.smithi044.stdout:9/337: symlink d2/dc/d1d/d34/d41/l79 0 2024-04-22T23:46:09.902 INFO:tasks.workunit.client.1.smithi044.stdout:3/479: readlink d9/db/d15/d2e/d48/d8f/l99 0 2024-04-22T23:46:09.902 INFO:tasks.workunit.client.1.smithi044.stdout:9/338: rmdir d2/d9 39 2024-04-22T23:46:09.903 INFO:tasks.workunit.client.1.smithi044.stdout:0/517: mkdir d0/d2e/dbf 0 2024-04-22T23:46:09.908 INFO:tasks.workunit.client.1.smithi044.stdout:1/489: unlink d1/d8/d20/d45/f29 0 2024-04-22T23:46:09.910 INFO:tasks.workunit.client.1.smithi044.stdout:3/480: chown d9/d14/c61 978266 1 2024-04-22T23:46:09.911 INFO:tasks.workunit.client.1.smithi044.stdout:8/530: dwrite db/d14/d24/f4e [0,4194304] 0 2024-04-22T23:46:09.912 INFO:tasks.workunit.client.1.smithi044.stdout:0/518: symlink d0/d2/d4b/lc0 0 2024-04-22T23:46:09.913 INFO:tasks.workunit.client.1.smithi044.stdout:0/519: dread - d0/d2e/d33/f85 zero size 2024-04-22T23:46:09.914 INFO:tasks.workunit.client.1.smithi044.stdout:0/520: chown d0/d2/dd/d3e/d40/d43/d7b/c69 2426290 1 2024-04-22T23:46:09.916 INFO:tasks.workunit.client.1.smithi044.stdout:9/339: write d2/d5/f7 [4260186,21415] 0 2024-04-22T23:46:09.916 INFO:tasks.workunit.client.1.smithi044.stdout:1/490: mknod d1/d8/d20/d45/d75/cb0 0 2024-04-22T23:46:09.918 INFO:tasks.workunit.client.1.smithi044.stdout:1/491: read d1/d14/f83 [337799,34318] 0 2024-04-22T23:46:09.923 INFO:tasks.workunit.client.1.smithi044.stdout:0/521: symlink d0/d2e/d2f/lc1 0 2024-04-22T23:46:09.929 INFO:tasks.workunit.client.1.smithi044.stdout:8/531: truncate db/f1c 421012 0 2024-04-22T23:46:09.930 INFO:tasks.workunit.client.1.smithi044.stdout:0/522: creat d0/d2/d4b/d6b/fc2 x:0 0 0 2024-04-22T23:46:09.933 INFO:tasks.workunit.client.1.smithi044.stdout:1/492: getdents d1/d8/d20/d9e 0 2024-04-22T23:46:09.941 INFO:tasks.workunit.client.1.smithi044.stdout:1/493: write d1/d8/d20/d47/d92/faa [811563,33521] 0 2024-04-22T23:46:09.942 INFO:tasks.workunit.client.1.smithi044.stdout:9/340: dwrite d2/d6/d4d/f5e [0,4194304] 0 2024-04-22T23:46:09.943 INFO:tasks.workunit.client.1.smithi044.stdout:3/481: getdents d9/db/d15/d2e/d48 0 2024-04-22T23:46:09.946 INFO:tasks.workunit.client.1.smithi044.stdout:0/523: symlink d0/d4c/lc3 0 2024-04-22T23:46:09.946 INFO:tasks.workunit.client.1.smithi044.stdout:8/532: dwrite db/f27 [0,4194304] 0 2024-04-22T23:46:09.948 INFO:tasks.workunit.client.1.smithi044.stdout:3/482: truncate d9/db/d15/f18 9043487 0 2024-04-22T23:46:09.950 INFO:tasks.workunit.client.1.smithi044.stdout:9/341: chown d2/dc/d1d/d34/d38/d5c/d69 2127606182 1 2024-04-22T23:46:09.954 INFO:tasks.workunit.client.1.smithi044.stdout:1/494: creat d1/d8/d20/d45/d75/d77/dad/fb1 x:0 0 0 2024-04-22T23:46:09.963 INFO:tasks.workunit.client.1.smithi044.stdout:1/495: dwrite d1/d8/d20/d47/d4a/f54 [0,4194304] 0 2024-04-22T23:46:09.983 INFO:tasks.workunit.client.1.smithi044.stdout:8/533: mkdir db/d14/dc7 0 2024-04-22T23:46:09.986 INFO:tasks.workunit.client.1.smithi044.stdout:1/496: creat d1/d8/d20/d47/d92/fb2 x:0 0 0 2024-04-22T23:46:09.990 INFO:tasks.workunit.client.1.smithi044.stdout:0/524: rename d0/d2/dd/la6 to d0/d2/dd/d3e/d40/d43/d79/dbb/lc4 0 2024-04-22T23:46:09.990 INFO:tasks.workunit.client.1.smithi044.stdout:9/342: dwrite d2/d9/f5a [0,4194304] 0 2024-04-22T23:46:09.993 INFO:tasks.workunit.client.1.smithi044.stdout:9/343: write d2/dc/d1d/d3f/f48 [4242413,28543] 0 2024-04-22T23:46:09.997 INFO:tasks.workunit.client.1.smithi044.stdout:3/483: rename d9/db/d22 to d9/db/d9f 0 2024-04-22T23:46:09.998 INFO:tasks.workunit.client.1.smithi044.stdout:8/534: symlink db/d14/d24/lc8 0 2024-04-22T23:46:10.004 INFO:tasks.workunit.client.1.smithi044.stdout:1/497: fdatasync d1/d8/d1c/d74/f8e 0 2024-04-22T23:46:10.006 INFO:tasks.workunit.client.1.smithi044.stdout:0/525: creat d0/d2/dd/d3e/fc5 x:0 0 0 2024-04-22T23:46:10.007 INFO:tasks.workunit.client.1.smithi044.stdout:0/526: readlink d0/d2e/d2f/la2 0 2024-04-22T23:46:10.011 INFO:tasks.workunit.client.1.smithi044.stdout:9/344: read d2/d6/f33 [3805955,3760] 0 2024-04-22T23:46:10.011 INFO:tasks.workunit.client.1.smithi044.stdout:3/484: rmdir d9/db/d15/d25/d5b/d52 39 2024-04-22T23:46:10.012 INFO:tasks.workunit.client.1.smithi044.stdout:9/345: write d2/dc/d1d/d34/f49 [9374667,25348] 0 2024-04-22T23:46:10.016 INFO:tasks.workunit.client.1.smithi044.stdout:0/527: creat d0/d2e/d2f/d77/fc6 x:0 0 0 2024-04-22T23:46:10.019 INFO:tasks.workunit.client.1.smithi044.stdout:8/535: mkdir db/d14/dc7/dc9 0 2024-04-22T23:46:10.020 INFO:tasks.workunit.client.1.smithi044.stdout:1/498: creat d1/d8/d20/dae/fb3 x:0 0 0 2024-04-22T23:46:10.025 INFO:tasks.workunit.client.1.smithi044.stdout:9/346: rename d2/dc/d1d/d34/d41 to d2/dc/d1d/d34/d38/d5c/d69/d7a 0 2024-04-22T23:46:10.027 INFO:tasks.workunit.client.1.smithi044.stdout:8/536: dwrite db/d14/d72/f9f [0,4194304] 0 2024-04-22T23:46:10.028 INFO:tasks.workunit.client.1.smithi044.stdout:0/528: read d0/d2/dd/d3e/f6e [3379603,67701] 0 2024-04-22T23:46:10.030 INFO:tasks.workunit.client.1.smithi044.stdout:8/537: readlink db/d14/l18 0 2024-04-22T23:46:10.030 INFO:tasks.workunit.client.1.smithi044.stdout:1/499: dwrite d1/f7 [0,4194304] 0 2024-04-22T23:46:10.031 INFO:tasks.workunit.client.1.smithi044.stdout:0/529: chown d0/d4c/f52 33974793 1 2024-04-22T23:46:10.040 INFO:tasks.workunit.client.1.smithi044.stdout:8/538: rmdir db/d14/d24 39 2024-04-22T23:46:10.042 INFO:tasks.workunit.client.1.smithi044.stdout:8/539: fsync db/f4f 0 2024-04-22T23:46:10.042 INFO:tasks.workunit.client.1.smithi044.stdout:8/540: read db/f2a [309215,5758] 0 2024-04-22T23:46:10.043 INFO:tasks.workunit.client.1.smithi044.stdout:8/541: chown db/c16 1 1 2024-04-22T23:46:10.043 INFO:tasks.workunit.client.1.smithi044.stdout:9/347: fdatasync d2/dc/f15 0 2024-04-22T23:46:10.044 INFO:tasks.workunit.client.1.smithi044.stdout:8/542: chown db/d14/c94 0 1 2024-04-22T23:46:10.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/500: symlink d1/d8/d1c/d23/d48/d99/lb4 0 2024-04-22T23:46:10.047 INFO:tasks.workunit.client.1.smithi044.stdout:3/485: link d9/db/d15/d25/d67/l87 d9/db/d15/d25/d4c/d86/la0 0 2024-04-22T23:46:10.053 INFO:tasks.workunit.client.1.smithi044.stdout:9/348: dwrite d2/dc/d1d/d34/f49 [4194304,4194304] 0 2024-04-22T23:46:10.055 INFO:tasks.workunit.client.1.smithi044.stdout:1/501: dread d1/d8/f1a [0,4194304] 0 2024-04-22T23:46:10.056 INFO:tasks.workunit.client.1.smithi044.stdout:1/502: stat d1/d8/d34/f9d 0 2024-04-22T23:46:10.060 INFO:tasks.workunit.client.1.smithi044.stdout:8/543: chown db/d14/d24 1133808 1 2024-04-22T23:46:10.066 INFO:tasks.workunit.client.1.smithi044.stdout:9/349: mkdir d2/dc/d1d/d34/d38/d5c/d7b 0 2024-04-22T23:46:10.068 INFO:tasks.workunit.client.1.smithi044.stdout:1/503: write d1/d8/d1c/d7b/f65 [3849387,117497] 0 2024-04-22T23:46:10.071 INFO:tasks.workunit.client.1.smithi044.stdout:8/544: unlink db/d14/f42 0 2024-04-22T23:46:10.072 INFO:tasks.workunit.client.1.smithi044.stdout:0/530: truncate d0/d2/d4b/f86 907941 0 2024-04-22T23:46:10.072 INFO:tasks.workunit.client.1.smithi044.stdout:9/350: dread d2/dc/f15 [0,4194304] 0 2024-04-22T23:46:10.079 INFO:tasks.workunit.client.1.smithi044.stdout:1/504: dwrite d1/d8/d20/d47/d92/fb2 [0,4194304] 0 2024-04-22T23:46:10.109 INFO:tasks.workunit.client.1.smithi044.stdout:5/457: sync 2024-04-22T23:46:10.109 INFO:tasks.workunit.client.1.smithi044.stdout:5/458: chown d15/d16/l87 17715281 1 2024-04-22T23:46:10.110 INFO:tasks.workunit.client.1.smithi044.stdout:5/459: stat d15/d16/d2c/l72 0 2024-04-22T23:46:10.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: pgmap v8: 65 pgs: 65 active+clean; 2.6 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 15 MiB/s rd, 72 MiB/s wr, 162 op/s 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:10.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:09 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: pgmap v8: 65 pgs: 65 active+clean; 2.6 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 15 MiB/s rd, 72 MiB/s wr, 162 op/s 2024-04-22T23:46:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:46:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:46:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:46:10.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:46:10.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/449232569' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:46:10.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:10.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:09 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:10.242 INFO:tasks.workunit.client.1.smithi044.stdout:3/486: write d9/db/f10 [6778834,76645] 0 2024-04-22T23:46:10.243 INFO:tasks.workunit.client.1.smithi044.stdout:8/545: mkdir db/d68/d8b/dca 0 2024-04-22T23:46:10.248 INFO:tasks.workunit.client.1.smithi044.stdout:9/351: creat d2/dc/d1d/d34/d38/d5c/d69/d75/f7c x:0 0 0 2024-04-22T23:46:10.254 INFO:tasks.workunit.client.1.smithi044.stdout:8/546: fsync db/f3f 0 2024-04-22T23:46:10.262 INFO:tasks.workunit.client.1.smithi044.stdout:3/487: unlink d9/db/d15/d25/d5b/d52/f8b 0 2024-04-22T23:46:10.265 INFO:tasks.workunit.client.1.smithi044.stdout:8/547: symlink db/d14/d84/dc0/lcb 0 2024-04-22T23:46:10.270 INFO:tasks.workunit.client.1.smithi044.stdout:8/548: creat db/d6c/fcc x:0 0 0 2024-04-22T23:46:10.271 INFO:tasks.workunit.client.1.smithi044.stdout:8/549: write db/d6c/fa5 [327083,66728] 0 2024-04-22T23:46:10.273 INFO:tasks.workunit.client.1.smithi044.stdout:3/488: link d9/db/f10 d9/db/d15/d2e/d48/fa1 0 2024-04-22T23:46:10.274 INFO:tasks.workunit.client.1.smithi044.stdout:3/489: write d9/db/d15/d2e/d48/f9a [280631,52595] 0 2024-04-22T23:46:10.275 INFO:tasks.workunit.client.1.smithi044.stdout:8/550: rmdir db/d52/d9c/dad 39 2024-04-22T23:46:10.283 INFO:tasks.workunit.client.1.smithi044.stdout:8/551: dwrite db/d14/f63 [0,4194304] 0 2024-04-22T23:46:10.283 INFO:tasks.workunit.client.1.smithi044.stdout:3/490: getdents d9/db/d15/d25/d67 0 2024-04-22T23:46:10.288 INFO:tasks.workunit.client.1.smithi044.stdout:8/552: unlink db/d52/f79 0 2024-04-22T23:46:10.292 INFO:tasks.workunit.client.1.smithi044.stdout:3/491: dwrite d9/db/d15/d54/f58 [0,4194304] 0 2024-04-22T23:46:10.293 INFO:tasks.workunit.client.1.smithi044.stdout:3/492: chown d9/db/d9f/f56 1935439 1 2024-04-22T23:46:10.301 INFO:tasks.workunit.client.1.smithi044.stdout:3/493: creat d9/db/fa2 x:0 0 0 2024-04-22T23:46:10.304 INFO:tasks.workunit.client.1.smithi044.stdout:3/494: creat d9/db/d15/d2e/d48/d8f/fa3 x:0 0 0 2024-04-22T23:46:10.309 INFO:tasks.workunit.client.1.smithi044.stdout:3/495: truncate d9/db/d15/d54/d3c/f94 2590696 0 2024-04-22T23:46:10.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/505: mkdir d1/d8/d1c/d23/d3e/db5 0 2024-04-22T23:46:10.311 INFO:tasks.workunit.client.1.smithi044.stdout:3/496: symlink d9/db/d15/d25/d67/d6b/la4 0 2024-04-22T23:46:10.312 INFO:tasks.workunit.client.1.smithi044.stdout:3/497: chown d9/db/d15/d54/f90 4279528 1 2024-04-22T23:46:10.313 INFO:tasks.workunit.client.1.smithi044.stdout:1/506: creat d1/d8/d1c/d7b/fb6 x:0 0 0 2024-04-22T23:46:10.314 INFO:tasks.workunit.client.1.smithi044.stdout:5/460: creat d15/d16/d2c/f8e x:0 0 0 2024-04-22T23:46:10.316 INFO:tasks.workunit.client.1.smithi044.stdout:3/498: dread d9/db/d15/d2e/f39 [0,4194304] 0 2024-04-22T23:46:10.317 INFO:tasks.workunit.client.1.smithi044.stdout:1/507: truncate d1/d8/d20/d47/f85 1618800 0 2024-04-22T23:46:10.320 INFO:tasks.workunit.client.1.smithi044.stdout:6/391: sync 2024-04-22T23:46:10.321 INFO:tasks.workunit.client.1.smithi044.stdout:6/392: write d5/d3f/f22 [6391252,25450] 0 2024-04-22T23:46:10.322 INFO:tasks.workunit.client.1.smithi044.stdout:6/393: write f1 [5231621,125652] 0 2024-04-22T23:46:10.326 INFO:tasks.workunit.client.1.smithi044.stdout:3/499: fdatasync d9/db/f1a 0 2024-04-22T23:46:10.332 INFO:tasks.workunit.client.1.smithi044.stdout:3/500: chown d9/db/d33/l53 208254 1 2024-04-22T23:46:10.336 INFO:tasks.workunit.client.1.smithi044.stdout:6/394: dread d5/d3f/f54 [0,4194304] 0 2024-04-22T23:46:10.337 INFO:tasks.workunit.client.1.smithi044.stdout:3/501: symlink d9/db/d33/d5d/d70/la5 0 2024-04-22T23:46:10.339 INFO:tasks.workunit.client.1.smithi044.stdout:6/395: read d5/dd/f16 [3403847,27310] 0 2024-04-22T23:46:10.340 INFO:tasks.workunit.client.1.smithi044.stdout:6/396: fsync d5/d3f/d50/f5a 0 2024-04-22T23:46:10.342 INFO:tasks.workunit.client.1.smithi044.stdout:3/502: symlink d9/db/d15/d2e/d48/d8f/la6 0 2024-04-22T23:46:10.346 INFO:tasks.workunit.client.1.smithi044.stdout:6/397: mkdir d5/dd/d67/d84 0 2024-04-22T23:46:10.354 INFO:tasks.workunit.client.1.smithi044.stdout:6/398: dwrite d5/dd/f6a [4194304,4194304] 0 2024-04-22T23:46:10.357 INFO:tasks.workunit.client.1.smithi044.stdout:6/399: write d5/dd/f1b [1752222,24881] 0 2024-04-22T23:46:10.358 INFO:tasks.workunit.client.1.smithi044.stdout:6/400: chown d5/d3f/d4c/d40/d53/d56/c7c 5656 1 2024-04-22T23:46:10.363 INFO:tasks.workunit.client.1.smithi044.stdout:6/401: readlink d5/l1c 0 2024-04-22T23:46:10.366 INFO:tasks.workunit.client.1.smithi044.stdout:6/402: mknod d5/d3f/d4c/d23/d3d/d64/c85 0 2024-04-22T23:46:10.369 INFO:tasks.workunit.client.1.smithi044.stdout:6/403: creat d5/d3f/d50/f86 x:0 0 0 2024-04-22T23:46:10.373 INFO:tasks.workunit.client.1.smithi044.stdout:6/404: creat d5/d30/d73/f87 x:0 0 0 2024-04-22T23:46:10.374 INFO:tasks.workunit.client.1.smithi044.stdout:6/405: write d5/d30/f39 [953273,68310] 0 2024-04-22T23:46:10.386 INFO:tasks.workunit.client.1.smithi044.stdout:9/352: creat d2/f7d x:0 0 0 2024-04-22T23:46:10.387 INFO:tasks.workunit.client.1.smithi044.stdout:8/553: write db/d14/d24/f43 [1222721,119672] 0 2024-04-22T23:46:10.388 INFO:tasks.workunit.client.1.smithi044.stdout:9/353: fdatasync d2/d6/f33 0 2024-04-22T23:46:10.393 INFO:tasks.workunit.client.1.smithi044.stdout:9/354: getdents d2/dc/d1d/d34 0 2024-04-22T23:46:10.397 INFO:tasks.workunit.client.1.smithi044.stdout:9/355: rmdir d2/d6/d42/d4b 0 2024-04-22T23:46:10.397 INFO:tasks.workunit.client.1.smithi044.stdout:9/356: fsync d2/d6/f53 0 2024-04-22T23:46:10.468 INFO:tasks.workunit.client.1.smithi044.stdout:1/508: mknod d1/d8/d20/d47/cb7 0 2024-04-22T23:46:10.468 INFO:tasks.workunit.client.1.smithi044.stdout:5/461: rename d15/d16/d1d/f3f to d15/d3c/d65/d8d/f8f 0 2024-04-22T23:46:10.469 INFO:tasks.workunit.client.1.smithi044.stdout:0/531: symlink d0/d2/dd/d3e/lc7 0 2024-04-22T23:46:10.473 INFO:tasks.workunit.client.1.smithi044.stdout:1/509: fdatasync d1/d8/d34/f43 0 2024-04-22T23:46:10.479 INFO:tasks.workunit.client.1.smithi044.stdout:0/532: mkdir d0/d2/d4b/dad/dc8 0 2024-04-22T23:46:10.486 INFO:tasks.workunit.client.1.smithi044.stdout:9/357: dwrite d2/dc/f66 [0,4194304] 0 2024-04-22T23:46:10.488 INFO:tasks.workunit.client.1.smithi044.stdout:8/554: dwrite db/d14/d24/f30 [0,4194304] 0 2024-04-22T23:46:10.489 INFO:tasks.workunit.client.1.smithi044.stdout:3/503: dwrite d9/db/d15/d54/d3c/f94 [0,4194304] 0 2024-04-22T23:46:10.496 INFO:tasks.workunit.client.1.smithi044.stdout:5/462: dwrite f11 [0,4194304] 0 2024-04-22T23:46:10.499 INFO:tasks.workunit.client.1.smithi044.stdout:3/504: write d9/db/d15/d54/f4b [303218,42266] 0 2024-04-22T23:46:10.501 INFO:tasks.workunit.client.1.smithi044.stdout:2/430: sync 2024-04-22T23:46:10.504 INFO:tasks.workunit.client.1.smithi044.stdout:3/505: stat d9/db/d15/f21 0 2024-04-22T23:46:10.504 INFO:tasks.workunit.client.1.smithi044.stdout:1/510: dread d1/d8/d1c/d7b/d80/d4b/f5a [0,4194304] 0 2024-04-22T23:46:10.505 INFO:tasks.workunit.client.1.smithi044.stdout:0/533: dwrite d0/d2e/f53 [0,4194304] 0 2024-04-22T23:46:10.508 INFO:tasks.workunit.client.1.smithi044.stdout:9/358: rename c0 to d2/dc/d1d/d34/d38/d5c/d69/d75/c7e 0 2024-04-22T23:46:10.513 INFO:tasks.workunit.client.1.smithi044.stdout:8/555: dread db/d14/d24/f59 [0,4194304] 0 2024-04-22T23:46:10.518 INFO:tasks.workunit.client.1.smithi044.stdout:2/431: dwrite f3 [0,4194304] 0 2024-04-22T23:46:10.518 INFO:tasks.workunit.client.1.smithi044.stdout:5/463: mknod d15/d16/d1d/d22/d81/c90 0 2024-04-22T23:46:10.521 INFO:tasks.workunit.client.1.smithi044.stdout:0/534: creat d0/d2e/dbf/fc9 x:0 0 0 2024-04-22T23:46:10.521 INFO:tasks.workunit.client.1.smithi044.stdout:9/359: dwrite d2/d6/f74 [0,4194304] 0 2024-04-22T23:46:10.525 INFO:tasks.workunit.client.1.smithi044.stdout:5/464: write d15/d16/d2c/d38/f50 [395433,45681] 0 2024-04-22T23:46:10.526 INFO:tasks.workunit.client.1.smithi044.stdout:1/511: mknod d1/d8/cb8 0 2024-04-22T23:46:10.529 INFO:tasks.workunit.client.1.smithi044.stdout:8/556: creat db/d68/d8b/da7/fcd x:0 0 0 2024-04-22T23:46:10.532 INFO:tasks.workunit.client.1.smithi044.stdout:2/432: mkdir d8/d19/d36/d93 0 2024-04-22T23:46:10.537 INFO:tasks.workunit.client.1.smithi044.stdout:3/506: link d9/l7b d9/db/d15/d54/la7 0 2024-04-22T23:46:10.538 INFO:tasks.workunit.client.1.smithi044.stdout:3/507: dread - d9/db/d15/d25/d67/f91 zero size 2024-04-22T23:46:10.539 INFO:tasks.workunit.client.1.smithi044.stdout:3/508: write d9/f9b [654902,74143] 0 2024-04-22T23:46:10.544 INFO:tasks.workunit.client.1.smithi044.stdout:8/557: creat db/d68/d8b/da7/fce x:0 0 0 2024-04-22T23:46:10.545 INFO:tasks.workunit.client.1.smithi044.stdout:8/558: fsync db/d68/d8b/da7/fbc 0 2024-04-22T23:46:10.552 INFO:tasks.workunit.client.1.smithi044.stdout:2/433: write d8/d27/f4b [4299651,125856] 0 2024-04-22T23:46:10.554 INFO:tasks.workunit.client.1.smithi044.stdout:3/509: dread d9/db/d15/d54/f90 [0,4194304] 0 2024-04-22T23:46:10.556 INFO:tasks.workunit.client.1.smithi044.stdout:5/465: creat d15/d5d/d64/d6f/d7f/f91 x:0 0 0 2024-04-22T23:46:10.558 INFO:tasks.workunit.client.1.smithi044.stdout:5/466: write d15/d16/d1d/f49 [1405638,116079] 0 2024-04-22T23:46:10.560 INFO:tasks.workunit.client.1.smithi044.stdout:7/493: sync 2024-04-22T23:46:10.560 INFO:tasks.workunit.client.1.smithi044.stdout:4/529: sync 2024-04-22T23:46:10.562 INFO:tasks.workunit.client.1.smithi044.stdout:8/559: rename db/c47 to db/d14/d9e/ccf 0 2024-04-22T23:46:10.562 INFO:tasks.workunit.client.1.smithi044.stdout:1/512: dwrite d1/d8/f18 [0,4194304] 0 2024-04-22T23:46:10.562 INFO:tasks.workunit.client.1.smithi044.stdout:4/530: chown d5/d42/f61 1483 1 2024-04-22T23:46:10.563 INFO:tasks.workunit.client.1.smithi044.stdout:8/560: chown db/d14 58 1 2024-04-22T23:46:10.563 INFO:tasks.workunit.client.1.smithi044.stdout:9/360: creat d2/f7f x:0 0 0 2024-04-22T23:46:10.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/434: mkdir d8/d19/d36/d46/d63/d65/d94 0 2024-04-22T23:46:10.568 INFO:tasks.workunit.client.1.smithi044.stdout:2/435: readlink d8/d19/d1c/l3f 0 2024-04-22T23:46:10.570 INFO:tasks.workunit.client.1.smithi044.stdout:3/510: symlink d9/db/d15/d2e/d48/d8f/la8 0 2024-04-22T23:46:10.571 INFO:tasks.workunit.client.1.smithi044.stdout:3/511: fdatasync d9/d12/f78 0 2024-04-22T23:46:10.576 INFO:tasks.workunit.client.1.smithi044.stdout:7/494: unlink dd/d1a/c33 0 2024-04-22T23:46:10.578 INFO:tasks.workunit.client.1.smithi044.stdout:1/513: chown d1/d8/d20/d47/c4c 1903 1 2024-04-22T23:46:10.579 INFO:tasks.workunit.client.1.smithi044.stdout:8/561: creat db/d68/daf/fd0 x:0 0 0 2024-04-22T23:46:10.580 INFO:tasks.workunit.client.1.smithi044.stdout:0/535: getdents d0/d2e/d2f/d77 0 2024-04-22T23:46:10.583 INFO:tasks.workunit.client.1.smithi044.stdout:4/531: creat d5/db/d34/fb1 x:0 0 0 2024-04-22T23:46:10.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/361: dwrite d2/d5/f50 [0,4194304] 0 2024-04-22T23:46:10.590 INFO:tasks.workunit.client.1.smithi044.stdout:1/514: mkdir d1/d8/d20/d9e/db9 0 2024-04-22T23:46:10.590 INFO:tasks.workunit.client.1.smithi044.stdout:1/515: write d1/d8/f18 [3784536,18786] 0 2024-04-22T23:46:10.592 INFO:tasks.workunit.client.1.smithi044.stdout:1/516: chown d1/d8/d1c/d7b/d80/d4b/f5a 262647 1 2024-04-22T23:46:10.593 INFO:tasks.workunit.client.1.smithi044.stdout:8/562: mkdir db/d14/dc7/dc9/dd1 0 2024-04-22T23:46:10.594 INFO:tasks.workunit.client.1.smithi044.stdout:8/563: stat db/d14/d24/l2f 0 2024-04-22T23:46:10.596 INFO:tasks.workunit.client.1.smithi044.stdout:3/512: truncate d9/db/d9f/d38/f4a 399797 0 2024-04-22T23:46:10.602 INFO:tasks.workunit.client.1.smithi044.stdout:4/532: readlink d5/d31/d24/d38/d6a/l92 0 2024-04-22T23:46:10.602 INFO:tasks.workunit.client.1.smithi044.stdout:4/533: stat d5/d31/f17 0 2024-04-22T23:46:10.603 INFO:tasks.workunit.client.1.smithi044.stdout:4/534: truncate d5/d31/d24/d38/f9f 678948 0 2024-04-22T23:46:10.604 INFO:tasks.workunit.client.1.smithi044.stdout:5/467: rename d15/d16/d1d/d22/d4f/f59 to d15/f92 0 2024-04-22T23:46:10.604 INFO:tasks.workunit.client.1.smithi044.stdout:4/535: stat d5/d42/l74 0 2024-04-22T23:46:10.606 INFO:tasks.workunit.client.1.smithi044.stdout:0/536: unlink d0/d4c/cb4 0 2024-04-22T23:46:10.607 INFO:tasks.workunit.client.1.smithi044.stdout:7/495: creat dd/d1f/fb2 x:0 0 0 2024-04-22T23:46:10.608 INFO:tasks.workunit.client.1.smithi044.stdout:9/362: mknod d2/d6/c80 0 2024-04-22T23:46:10.616 INFO:tasks.workunit.client.1.smithi044.stdout:5/468: fsync d15/d16/f75 0 2024-04-22T23:46:10.618 INFO:tasks.workunit.client.1.smithi044.stdout:4/536: mknod d5/d31/d44/cb2 0 2024-04-22T23:46:10.622 INFO:tasks.workunit.client.1.smithi044.stdout:7/496: dread fa [0,4194304] 0 2024-04-22T23:46:10.622 INFO:tasks.workunit.client.1.smithi044.stdout:9/363: mknod d2/d6/d42/c81 0 2024-04-22T23:46:10.623 INFO:tasks.workunit.client.1.smithi044.stdout:9/364: write d2/d6/f53 [1591985,77701] 0 2024-04-22T23:46:10.623 INFO:tasks.workunit.client.1.smithi044.stdout:8/564: getdents db/d68/d8b/dca 0 2024-04-22T23:46:10.624 INFO:tasks.workunit.client.1.smithi044.stdout:9/365: chown d2/dc/c22 1 1 2024-04-22T23:46:10.625 INFO:tasks.workunit.client.1.smithi044.stdout:8/565: chown db/d6c/c6e 26328 1 2024-04-22T23:46:10.626 INFO:tasks.workunit.client.1.smithi044.stdout:5/469: chown cd 2062278904 1 2024-04-22T23:46:10.627 INFO:tasks.workunit.client.1.smithi044.stdout:0/537: mknod d0/d2/d4b/dad/dc8/cca 0 2024-04-22T23:46:10.628 INFO:tasks.workunit.client.1.smithi044.stdout:1/517: write d1/d14/f52 [1437348,46032] 0 2024-04-22T23:46:10.632 INFO:tasks.workunit.client.1.smithi044.stdout:7/497: unlink dd/f4a 0 2024-04-22T23:46:10.633 INFO:tasks.workunit.client.1.smithi044.stdout:3/513: write d9/db/f10 [1279944,2806] 0 2024-04-22T23:46:10.634 INFO:tasks.workunit.client.1.smithi044.stdout:8/566: creat db/d68/d8b/fd2 x:0 0 0 2024-04-22T23:46:10.636 INFO:tasks.workunit.client.1.smithi044.stdout:8/567: chown db/d14/d24/l33 6 1 2024-04-22T23:46:10.637 INFO:tasks.workunit.client.1.smithi044.stdout:5/470: mknod d15/d16/d1d/d22/d2a/d3a/d43/c93 0 2024-04-22T23:46:10.640 INFO:tasks.workunit.client.1.smithi044.stdout:7/498: dwrite dd/d1f/fb2 [0,4194304] 0 2024-04-22T23:46:10.641 INFO:tasks.workunit.client.1.smithi044.stdout:7/499: readlink dd/d98/l9a 0 2024-04-22T23:46:10.641 INFO:tasks.workunit.client.1.smithi044.stdout:7/500: stat f8 0 2024-04-22T23:46:10.645 INFO:tasks.workunit.client.1.smithi044.stdout:3/514: unlink d9/db/d15/d2e/d48/f9a 0 2024-04-22T23:46:10.648 INFO:tasks.workunit.client.1.smithi044.stdout:8/568: mknod db/d14/d84/dc0/cd3 0 2024-04-22T23:46:10.649 INFO:tasks.workunit.client.1.smithi044.stdout:8/569: chown db/d14/d24/f4e 15372 1 2024-04-22T23:46:10.650 INFO:tasks.workunit.client.1.smithi044.stdout:8/570: readlink db/l1f 0 2024-04-22T23:46:10.652 INFO:tasks.workunit.client.1.smithi044.stdout:7/501: mknod dd/d79/d8c/cb3 0 2024-04-22T23:46:10.653 INFO:tasks.workunit.client.1.smithi044.stdout:7/502: write dd/f20 [1599687,61496] 0 2024-04-22T23:46:10.654 INFO:tasks.workunit.client.1.smithi044.stdout:0/538: creat d0/d2/fcb x:0 0 0 2024-04-22T23:46:10.654 INFO:tasks.workunit.client.1.smithi044.stdout:7/503: write dd/d1a/d61/f96 [922273,74600] 0 2024-04-22T23:46:10.659 INFO:tasks.workunit.client.1.smithi044.stdout:1/518: link d1/d8/d1c/d23/f5f d1/d8/d20/d47/d92/fba 0 2024-04-22T23:46:10.662 INFO:tasks.workunit.client.1.smithi044.stdout:5/471: creat d15/f94 x:0 0 0 2024-04-22T23:46:10.665 INFO:tasks.workunit.client.1.smithi044.stdout:1/519: creat d1/d8/d20/d45/d75/d77/dad/fbb x:0 0 0 2024-04-22T23:46:10.668 INFO:tasks.workunit.client.1.smithi044.stdout:7/504: symlink dd/d1f/lb4 0 2024-04-22T23:46:10.670 INFO:tasks.workunit.client.1.smithi044.stdout:5/472: dwrite f11 [8388608,4194304] 0 2024-04-22T23:46:10.672 INFO:tasks.workunit.client.1.smithi044.stdout:0/539: truncate d0/d2/dd/f1a 2038872 0 2024-04-22T23:46:10.672 INFO:tasks.workunit.client.1.smithi044.stdout:5/473: chown d15/d5d/d64 22 1 2024-04-22T23:46:10.673 INFO:tasks.workunit.client.1.smithi044.stdout:1/520: dread d1/d8/d20/d47/d92/fb2 [0,4194304] 0 2024-04-22T23:46:10.681 INFO:tasks.workunit.client.1.smithi044.stdout:4/537: truncate d5/db/d34/f59 1211055 0 2024-04-22T23:46:10.682 INFO:tasks.workunit.client.1.smithi044.stdout:5/474: symlink d15/d3c/d65/d8d/l95 0 2024-04-22T23:46:10.683 INFO:tasks.workunit.client.1.smithi044.stdout:5/475: readlink d15/d16/d2c/l46 0 2024-04-22T23:46:10.686 INFO:tasks.workunit.client.1.smithi044.stdout:4/538: dwrite d5/d31/f17 [0,4194304] 0 2024-04-22T23:46:10.687 INFO:tasks.workunit.client.1.smithi044.stdout:0/540: symlink d0/d2/dd/d3e/d40/d67/lcc 0 2024-04-22T23:46:10.687 INFO:tasks.workunit.client.1.smithi044.stdout:0/541: readlink d0/d2/dd/lac 0 2024-04-22T23:46:10.689 INFO:tasks.workunit.client.1.smithi044.stdout:5/476: dread d15/d16/d1d/f69 [0,4194304] 0 2024-04-22T23:46:10.689 INFO:tasks.workunit.client.1.smithi044.stdout:0/542: readlink d0/d2/dd/d3e/d40/l57 0 2024-04-22T23:46:10.690 INFO:tasks.workunit.client.1.smithi044.stdout:4/539: truncate d5/d31/d44/d20/d22/f6f 976379 0 2024-04-22T23:46:10.697 INFO:tasks.workunit.client.1.smithi044.stdout:0/543: readlink d0/d4c/d59/l9f 0 2024-04-22T23:46:10.698 INFO:tasks.workunit.client.1.smithi044.stdout:5/477: truncate d15/d16/d2c/f3b 421761 0 2024-04-22T23:46:10.698 INFO:tasks.workunit.client.1.smithi044.stdout:0/544: fdatasync d0/d4c/d59/fb3 0 2024-04-22T23:46:10.704 INFO:tasks.workunit.client.1.smithi044.stdout:5/478: dwrite d15/f30 [0,4194304] 0 2024-04-22T23:46:10.711 INFO:tasks.workunit.client.1.smithi044.stdout:3/515: write d9/db/d33/f96 [921314,60040] 0 2024-04-22T23:46:10.712 INFO:tasks.workunit.client.1.smithi044.stdout:5/479: truncate d15/d16/d2c/f2e 1071839 0 2024-04-22T23:46:10.712 INFO:tasks.workunit.client.1.smithi044.stdout:4/540: getdents d5/d31/d24/d38/d6a/d75 0 2024-04-22T23:46:10.714 INFO:tasks.workunit.client.1.smithi044.stdout:0/545: getdents d0/d2/d4b/d6b 0 2024-04-22T23:46:10.715 INFO:tasks.workunit.client.1.smithi044.stdout:4/541: write d5/d31/d24/d69/f81 [13716,30241] 0 2024-04-22T23:46:10.715 INFO:tasks.workunit.client.1.smithi044.stdout:8/571: write db/d68/f7a [1969734,107592] 0 2024-04-22T23:46:10.719 INFO:tasks.workunit.client.1.smithi044.stdout:3/516: dread d9/db/d15/d2e/f32 [0,4194304] 0 2024-04-22T23:46:10.721 INFO:tasks.workunit.client.1.smithi044.stdout:3/517: chown d9/db/d15/d2e/c37 13585462 1 2024-04-22T23:46:10.722 INFO:tasks.workunit.client.1.smithi044.stdout:3/518: chown d9/db/d33/d5d/f66 14369359 1 2024-04-22T23:46:10.723 INFO:tasks.workunit.client.1.smithi044.stdout:6/406: sync 2024-04-22T23:46:10.724 INFO:tasks.workunit.client.1.smithi044.stdout:3/519: readlink d9/db/d33/d5d/d70/la5 0 2024-04-22T23:46:10.728 INFO:tasks.workunit.client.1.smithi044.stdout:3/520: write d9/db/d15/d2e/d48/f72 [1321690,89778] 0 2024-04-22T23:46:10.728 INFO:tasks.workunit.client.1.smithi044.stdout:0/546: creat d0/d4c/dbd/fcd x:0 0 0 2024-04-22T23:46:10.729 INFO:tasks.workunit.client.1.smithi044.stdout:4/542: mknod d5/d31/d24/da6/cb3 0 2024-04-22T23:46:10.733 INFO:tasks.workunit.client.1.smithi044.stdout:6/407: unlink d5/dd/d28/d75/f7f 0 2024-04-22T23:46:10.736 INFO:tasks.workunit.client.1.smithi044.stdout:8/572: mknod db/d14/dc7/dc9/dd1/cd4 0 2024-04-22T23:46:10.737 INFO:tasks.workunit.client.1.smithi044.stdout:6/408: truncate d5/d30/d73/f82 152151 0 2024-04-22T23:46:10.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/409: stat d5/dd/d28/f2c 0 2024-04-22T23:46:10.747 INFO:tasks.workunit.client.1.smithi044.stdout:4/543: unlink d5/d31/d44/d20/d22/f37 0 2024-04-22T23:46:10.750 INFO:tasks.workunit.client.1.smithi044.stdout:8/573: unlink db/l4d 0 2024-04-22T23:46:10.754 INFO:tasks.workunit.client.1.smithi044.stdout:6/410: truncate d5/d3f/d4c/d23/d3d/f4a 904076 0 2024-04-22T23:46:10.758 INFO:tasks.workunit.client.1.smithi044.stdout:7/505: write f5 [2183081,125455] 0 2024-04-22T23:46:10.759 INFO:tasks.workunit.client.1.smithi044.stdout:7/506: chown dd/d1f/lb4 12626 1 2024-04-22T23:46:10.760 INFO:tasks.workunit.client.1.smithi044.stdout:8/574: dwrite db/d68/d8b/da7/fbc [0,4194304] 0 2024-04-22T23:46:10.762 INFO:tasks.workunit.client.1.smithi044.stdout:4/544: write d5/d31/d44/d7e/f98 [172558,121329] 0 2024-04-22T23:46:10.764 INFO:tasks.workunit.client.1.smithi044.stdout:7/507: rmdir dd/d1a 39 2024-04-22T23:46:10.766 INFO:tasks.workunit.client.1.smithi044.stdout:8/575: write db/d14/f88 [454579,22198] 0 2024-04-22T23:46:10.767 INFO:tasks.workunit.client.1.smithi044.stdout:1/521: truncate d1/d8/d1c/d7b/f8b 2976163 0 2024-04-22T23:46:10.769 INFO:tasks.workunit.client.1.smithi044.stdout:8/576: write db/d68/d8b/da7/fbc [531859,37607] 0 2024-04-22T23:46:10.776 INFO:tasks.workunit.client.1.smithi044.stdout:7/508: rename dd/d1f/fb2 to dd/d27/fb5 0 2024-04-22T23:46:10.776 INFO:tasks.workunit.client.1.smithi044.stdout:7/509: fdatasync dd/f3a 0 2024-04-22T23:46:10.779 INFO:tasks.workunit.client.1.smithi044.stdout:8/577: symlink db/d14/dc7/dc9/dd1/ld5 0 2024-04-22T23:46:10.783 INFO:tasks.workunit.client.1.smithi044.stdout:4/545: dread d5/d31/d44/d20/f55 [0,4194304] 0 2024-04-22T23:46:10.786 INFO:tasks.workunit.client.1.smithi044.stdout:8/578: mkdir db/d68/d8b/dca/dd6 0 2024-04-22T23:46:10.789 INFO:tasks.workunit.client.1.smithi044.stdout:7/510: dwrite dd/d1f/d24/fa2 [0,4194304] 0 2024-04-22T23:46:10.790 INFO:tasks.workunit.client.1.smithi044.stdout:4/546: rename d5/db/c36 to d5/d52/cb4 0 2024-04-22T23:46:10.797 INFO:tasks.workunit.client.1.smithi044.stdout:7/511: creat dd/d27/fb6 x:0 0 0 2024-04-22T23:46:10.798 INFO:tasks.workunit.client.1.smithi044.stdout:4/547: dwrite d5/d31/d44/d20/d22/d8d/f97 [0,4194304] 0 2024-04-22T23:46:10.800 INFO:tasks.workunit.client.1.smithi044.stdout:1/522: link d1/d8/d20/d45/d5d/l87 d1/d8/d20/d47/d4a/lbc 0 2024-04-22T23:46:10.804 INFO:tasks.workunit.client.1.smithi044.stdout:4/548: mknod d5/d31/d24/d69/cb5 0 2024-04-22T23:46:10.806 INFO:tasks.workunit.client.1.smithi044.stdout:5/480: dwrite d15/d16/d1d/d22/d2a/d3a/f4b [0,4194304] 0 2024-04-22T23:46:10.807 INFO:tasks.workunit.client.1.smithi044.stdout:1/523: creat d1/d8/d34/da1/fbd x:0 0 0 2024-04-22T23:46:10.807 INFO:tasks.workunit.client.1.smithi044.stdout:7/512: mkdir dd/d1a/db7 0 2024-04-22T23:46:10.808 INFO:tasks.workunit.client.1.smithi044.stdout:1/524: fdatasync d1/d8/d20/d45/f70 0 2024-04-22T23:46:10.809 INFO:tasks.workunit.client.1.smithi044.stdout:4/549: fsync d5/d31/d44/d20/d22/f62 0 2024-04-22T23:46:10.811 INFO:tasks.workunit.client.1.smithi044.stdout:1/525: fdatasync d1/d8/fa0 0 2024-04-22T23:46:10.813 INFO:tasks.workunit.client.1.smithi044.stdout:5/481: readlink d15/l31 0 2024-04-22T23:46:10.813 INFO:tasks.workunit.client.1.smithi044.stdout:4/550: write d5/d31/d44/d20/d22/f62 [1372531,18718] 0 2024-04-22T23:46:10.816 INFO:tasks.workunit.client.1.smithi044.stdout:7/513: mkdir dd/d79/d8c/db8 0 2024-04-22T23:46:10.818 INFO:tasks.workunit.client.1.smithi044.stdout:7/514: dread - dd/d1a/f7d zero size 2024-04-22T23:46:10.818 INFO:tasks.workunit.client.1.smithi044.stdout:4/551: mkdir d5/d31/d24/da6/db6 0 2024-04-22T23:46:10.819 INFO:tasks.workunit.client.1.smithi044.stdout:5/482: creat d15/d16/d1d/d22/d2a/d3a/f96 x:0 0 0 2024-04-22T23:46:10.825 INFO:tasks.workunit.client.1.smithi044.stdout:6/411: truncate d5/dd/d67/f20 1738384 0 2024-04-22T23:46:10.827 INFO:tasks.workunit.client.1.smithi044.stdout:1/526: dwrite d1/d8/d20/d47/f8a [0,4194304] 0 2024-04-22T23:46:10.829 INFO:tasks.workunit.client.1.smithi044.stdout:4/552: truncate d5/d31/d44/f7d 30177 0 2024-04-22T23:46:10.830 INFO:tasks.workunit.client.1.smithi044.stdout:5/483: fdatasync d15/d16/d1d/d22/f2d 0 2024-04-22T23:46:10.835 INFO:tasks.workunit.client.1.smithi044.stdout:6/412: write d5/dd/d28/f34 [4603730,35218] 0 2024-04-22T23:46:10.835 INFO:tasks.workunit.client.1.smithi044.stdout:3/521: dwrite d9/d12/f60 [0,4194304] 0 2024-04-22T23:46:10.837 INFO:tasks.workunit.client.1.smithi044.stdout:0/547: dwrite d0/d2/dd/f36 [0,4194304] 0 2024-04-22T23:46:10.837 INFO:tasks.workunit.client.1.smithi044.stdout:5/484: stat d15/d16/l87 0 2024-04-22T23:46:10.837 INFO:tasks.workunit.client.1.smithi044.stdout:4/553: chown d5/d31/f1b 120344529 1 2024-04-22T23:46:10.838 INFO:tasks.workunit.client.1.smithi044.stdout:6/413: readlink d5/l7 0 2024-04-22T23:46:10.841 INFO:tasks.workunit.client.1.smithi044.stdout:5/485: read d15/d3c/d65/d8d/f8f [370346,53397] 0 2024-04-22T23:46:10.842 INFO:tasks.workunit.client.1.smithi044.stdout:4/554: write d5/d31/f1b [2987942,11463] 0 2024-04-22T23:46:10.845 INFO:tasks.workunit.client.1.smithi044.stdout:3/522: creat d9/db/d15/d2e/d48/fa9 x:0 0 0 2024-04-22T23:46:10.847 INFO:tasks.workunit.client.1.smithi044.stdout:8/579: write db/d14/d24/f3b [67412,76535] 0 2024-04-22T23:46:10.849 INFO:tasks.workunit.client.1.smithi044.stdout:3/523: chown d9/db/d15/d54/l2c 4 1 2024-04-22T23:46:10.850 INFO:tasks.workunit.client.1.smithi044.stdout:6/414: mkdir d5/dd/d28/d75/d88 0 2024-04-22T23:46:10.851 INFO:tasks.workunit.client.1.smithi044.stdout:3/524: chown d9/d14/l16 38693 1 2024-04-22T23:46:10.853 INFO:tasks.workunit.client.1.smithi044.stdout:0/548: dwrite d0/d2/dd/d3e/d40/f84 [0,4194304] 0 2024-04-22T23:46:10.855 INFO:tasks.workunit.client.1.smithi044.stdout:3/525: write d9/db/d15/d2e/d48/d8f/fa3 [382224,110294] 0 2024-04-22T23:46:10.856 INFO:tasks.workunit.client.1.smithi044.stdout:1/527: symlink d1/d8/d20/d47/d4a/d60/lbe 0 2024-04-22T23:46:10.859 INFO:tasks.workunit.client.1.smithi044.stdout:1/528: write d1/d8/d20/d45/f70 [1427258,38757] 0 2024-04-22T23:46:10.863 INFO:tasks.workunit.client.1.smithi044.stdout:1/529: write d1/d8/d20/d45/f4f [36642,22553] 0 2024-04-22T23:46:10.866 INFO:tasks.workunit.client.1.smithi044.stdout:1/530: read - d1/d8/d20/d47/d92/fa2 zero size 2024-04-22T23:46:10.866 INFO:tasks.workunit.client.1.smithi044.stdout:7/515: dread dd/d1a/f1d [0,4194304] 0 2024-04-22T23:46:10.867 INFO:tasks.workunit.client.1.smithi044.stdout:7/516: write dd/d27/f6d [3751472,128997] 0 2024-04-22T23:46:10.887 INFO:tasks.workunit.client.1.smithi044.stdout:6/415: dread d5/dd/f16 [0,4194304] 0 2024-04-22T23:46:10.888 INFO:tasks.workunit.client.1.smithi044.stdout:7/517: mkdir dd/d1a/d3c/d97/db9 0 2024-04-22T23:46:10.888 INFO:tasks.workunit.client.1.smithi044.stdout:6/416: readlink d5/d3f/d4c/d40/d53/l6f 0 2024-04-22T23:46:10.890 INFO:tasks.workunit.client.1.smithi044.stdout:5/486: rename d15/d16/d1d/d22/d2a/f85 to d15/d16/d1d/d22/f97 0 2024-04-22T23:46:10.893 INFO:tasks.workunit.client.1.smithi044.stdout:3/526: mknod d9/db/d9f/d38/d9e/caa 0 2024-04-22T23:46:10.897 INFO:tasks.workunit.client.1.smithi044.stdout:8/580: write db/d14/d24/f75 [375308,123222] 0 2024-04-22T23:46:10.901 INFO:tasks.workunit.client.1.smithi044.stdout:8/581: chown db/d68/d74/fb0 1476839 1 2024-04-22T23:46:10.901 INFO:tasks.workunit.client.1.smithi044.stdout:1/531: rename d1/d8/d1c/d7b/d80/d4b/d81 to d1/d8/d20/d45/d75/d77/dbf 0 2024-04-22T23:46:10.902 INFO:tasks.workunit.client.1.smithi044.stdout:4/555: truncate d5/d42/f61 1205816 0 2024-04-22T23:46:10.902 INFO:tasks.workunit.client.1.smithi044.stdout:3/527: dwrite d9/db/d15/d54/d3c/f81 [0,4194304] 0 2024-04-22T23:46:10.904 INFO:tasks.workunit.client.1.smithi044.stdout:5/487: creat d15/d3c/d65/d8d/f98 x:0 0 0 2024-04-22T23:46:10.906 INFO:tasks.workunit.client.1.smithi044.stdout:0/549: dwrite d0/d2/dd/d3e/f6e [4194304,4194304] 0 2024-04-22T23:46:10.906 INFO:tasks.workunit.client.1.smithi044.stdout:5/488: write fe [109505,16568] 0 2024-04-22T23:46:10.907 INFO:tasks.workunit.client.1.smithi044.stdout:7/518: mknod dd/d79/d8c/db8/cba 0 2024-04-22T23:46:10.909 INFO:tasks.workunit.client.1.smithi044.stdout:0/550: stat d0/d4c/d59/f76 0 2024-04-22T23:46:10.910 INFO:tasks.workunit.client.1.smithi044.stdout:7/519: readlink dd/d55/d5e/l67 0 2024-04-22T23:46:10.913 INFO:tasks.workunit.client.1.smithi044.stdout:6/417: creat d5/d3f/f89 x:0 0 0 2024-04-22T23:46:10.913 INFO:tasks.workunit.client.1.smithi044.stdout:0/551: truncate d0/d4c/f8f 4899981 0 2024-04-22T23:46:10.915 INFO:tasks.workunit.client.1.smithi044.stdout:3/528: dwrite d9/db/d15/d54/d3c/f81 [0,4194304] 0 2024-04-22T23:46:10.915 INFO:tasks.workunit.client.1.smithi044.stdout:0/552: write d0/d2/fcb [91231,48325] 0 2024-04-22T23:46:10.927 INFO:tasks.workunit.client.1.smithi044.stdout:1/532: truncate d1/d8/d20/d47/d92/fb2 4261922 0 2024-04-22T23:46:10.932 INFO:tasks.workunit.client.1.smithi044.stdout:0/553: unlink d0/d2e/d2f/d61/f94 0 2024-04-22T23:46:10.933 INFO:tasks.workunit.client.1.smithi044.stdout:0/554: write d0/d2/fcb [701468,48561] 0 2024-04-22T23:46:10.940 INFO:tasks.workunit.client.1.smithi044.stdout:4/556: dwrite d5/d31/d24/f2f [4194304,4194304] 0 2024-04-22T23:46:10.943 INFO:tasks.workunit.client.1.smithi044.stdout:3/529: creat d9/db/d9f/d38/d74/fab x:0 0 0 2024-04-22T23:46:10.949 INFO:tasks.workunit.client.1.smithi044.stdout:5/489: dread f11 [4194304,4194304] 0 2024-04-22T23:46:10.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/520: dwrite dd/d1f/d24/f1e [0,4194304] 0 2024-04-22T23:46:10.950 INFO:tasks.workunit.client.1.smithi044.stdout:3/530: symlink d9/db/d15/d25/lac 0 2024-04-22T23:46:10.958 INFO:tasks.workunit.client.1.smithi044.stdout:5/490: unlink d15/d16/d1d/d22/d4f/f52 0 2024-04-22T23:46:10.965 INFO:tasks.workunit.client.1.smithi044.stdout:7/521: dwrite dd/d27/f6d [0,4194304] 0 2024-04-22T23:46:10.969 INFO:tasks.workunit.client.1.smithi044.stdout:7/522: dread dd/d1a/f1d [0,4194304] 0 2024-04-22T23:46:10.971 INFO:tasks.workunit.client.1.smithi044.stdout:4/557: truncate d5/d31/f76 1854881 0 2024-04-22T23:46:10.972 INFO:tasks.workunit.client.1.smithi044.stdout:4/558: fsync d5/d31/d24/d38/f40 0 2024-04-22T23:46:10.973 INFO:tasks.workunit.client.1.smithi044.stdout:4/559: chown d5/d52/f95 251 1 2024-04-22T23:46:10.981 INFO:tasks.workunit.client.1.smithi044.stdout:0/555: getdents d0/d2/dd/d3e/d40/d67 0 2024-04-22T23:46:10.982 INFO:tasks.workunit.client.1.smithi044.stdout:5/491: mkdir d15/d16/d1d/d22/d4f/d99 0 2024-04-22T23:46:10.983 INFO:tasks.workunit.client.1.smithi044.stdout:5/492: chown d15/d16/d1d/f42 243135 1 2024-04-22T23:46:10.983 INFO:tasks.workunit.client.1.smithi044.stdout:5/493: chown d15/d5d/d82/f83 3551 1 2024-04-22T23:46:10.984 INFO:tasks.workunit.client.1.smithi044.stdout:7/523: mkdir dd/d99/dbb 0 2024-04-22T23:46:10.986 INFO:tasks.workunit.client.1.smithi044.stdout:4/560: rename d5/db/d34/l87 to d5/d31/d24/da6/lb7 0 2024-04-22T23:46:10.987 INFO:tasks.workunit.client.1.smithi044.stdout:0/556: symlink d0/d2e/dbf/lce 0 2024-04-22T23:46:10.990 INFO:tasks.workunit.client.1.smithi044.stdout:7/524: creat dd/d79/d8c/fbc x:0 0 0 2024-04-22T23:46:10.991 INFO:tasks.workunit.client.1.smithi044.stdout:4/561: rename d5/d31/d24/d38/ca1 to d5/db/d34/cb8 0 2024-04-22T23:46:10.992 INFO:tasks.workunit.client.1.smithi044.stdout:4/562: write d5/d31/d24/d38/f91 [890500,55503] 0 2024-04-22T23:46:10.994 INFO:tasks.workunit.client.1.smithi044.stdout:4/563: truncate d5/db/f86 411092 0 2024-04-22T23:46:10.995 INFO:tasks.workunit.client.1.smithi044.stdout:3/531: getdents d9/db/d15/d54 0 2024-04-22T23:46:10.997 INFO:tasks.workunit.client.1.smithi044.stdout:3/532: truncate d9/db/d15/d54/f4b 4246973 0 2024-04-22T23:46:11.000 INFO:tasks.workunit.client.1.smithi044.stdout:7/525: mkdir dd/d55/d5e/d60/d68/d94/dbd 0 2024-04-22T23:46:11.001 INFO:tasks.workunit.client.1.smithi044.stdout:7/526: chown dd/d1f/l50 163 1 2024-04-22T23:46:11.002 INFO:tasks.workunit.client.1.smithi044.stdout:7/527: fdatasync dd/d27/f6d 0 2024-04-22T23:46:11.003 INFO:tasks.workunit.client.1.smithi044.stdout:7/528: fsync dd/d1f/d24/fa2 0 2024-04-22T23:46:11.004 INFO:tasks.workunit.client.1.smithi044.stdout:4/564: creat d5/db/d34/fb9 x:0 0 0 2024-04-22T23:46:11.007 INFO:tasks.workunit.client.1.smithi044.stdout:5/494: creat d15/d16/f9a x:0 0 0 2024-04-22T23:46:11.009 INFO:tasks.workunit.client.1.smithi044.stdout:3/533: symlink d9/db/d15/d2e/d48/d8f/lad 0 2024-04-22T23:46:11.010 INFO:tasks.workunit.client.1.smithi044.stdout:3/534: stat d9/db/d33/d5d/d70/la5 0 2024-04-22T23:46:11.011 INFO:tasks.workunit.client.1.smithi044.stdout:3/535: write d9/d12/f92 [629551,68335] 0 2024-04-22T23:46:11.012 INFO:tasks.workunit.client.1.smithi044.stdout:7/529: creat dd/d55/d5e/d60/d68/d94/fbe x:0 0 0 2024-04-22T23:46:11.014 INFO:tasks.workunit.client.1.smithi044.stdout:5/495: write f5 [6791559,64804] 0 2024-04-22T23:46:11.018 INFO:tasks.workunit.client.1.smithi044.stdout:3/536: unlink d9/db/d15/d25/d5b/f40 0 2024-04-22T23:46:11.019 INFO:tasks.workunit.client.1.smithi044.stdout:7/530: dwrite dd/d55/d5e/d60/d7b/f7f [0,4194304] 0 2024-04-22T23:46:11.024 INFO:tasks.workunit.client.1.smithi044.stdout:3/537: mknod d9/db/d33/d5d/d70/cae 0 2024-04-22T23:46:11.028 INFO:tasks.workunit.client.1.smithi044.stdout:5/496: creat d15/d16/d1d/d22/d2a/d3a/d43/f9b x:0 0 0 2024-04-22T23:46:11.032 INFO:tasks.workunit.client.1.smithi044.stdout:3/538: dwrite d9/db/d33/d5d/f66 [0,4194304] 0 2024-04-22T23:46:11.034 INFO:tasks.workunit.client.1.smithi044.stdout:5/497: symlink d15/d16/d1d/d22/d2a/l9c 0 2024-04-22T23:46:11.035 INFO:tasks.workunit.client.1.smithi044.stdout:5/498: read - d15/d16/d1d/d22/d2a/d3a/f96 zero size 2024-04-22T23:46:11.035 INFO:tasks.workunit.client.1.smithi044.stdout:3/539: read d9/db/d15/f21 [74577,47993] 0 2024-04-22T23:46:11.037 INFO:tasks.workunit.client.1.smithi044.stdout:0/557: write d0/d2/dd/f1a [1778690,41957] 0 2024-04-22T23:46:11.039 INFO:tasks.workunit.client.1.smithi044.stdout:0/558: write d0/d2/d4b/d6b/fc2 [749518,16719] 0 2024-04-22T23:46:11.042 INFO:tasks.workunit.client.1.smithi044.stdout:5/499: fdatasync d15/d16/d2c/f8e 0 2024-04-22T23:46:11.045 INFO:tasks.workunit.client.1.smithi044.stdout:3/540: rmdir d9/db/d33/d5d/d70 39 2024-04-22T23:46:11.045 INFO:tasks.workunit.client.1.smithi044.stdout:2/436: sync 2024-04-22T23:46:11.051 INFO:tasks.workunit.client.1.smithi044.stdout:5/500: mknod d15/d16/d1d/d22/d2a/d3a/d43/d80/c9d 0 2024-04-22T23:46:11.053 INFO:tasks.workunit.client.1.smithi044.stdout:4/565: dwrite d5/d31/d44/d20/f4b [0,4194304] 0 2024-04-22T23:46:11.055 INFO:tasks.workunit.client.1.smithi044.stdout:3/541: dwrite f8 [0,4194304] 0 2024-04-22T23:46:11.057 INFO:tasks.workunit.client.1.smithi044.stdout:2/437: symlink d8/d27/d5b/l95 0 2024-04-22T23:46:11.058 INFO:tasks.workunit.client.1.smithi044.stdout:3/542: write d9/db/d15/d2e/d48/fa1 [9356515,25309] 0 2024-04-22T23:46:11.063 INFO:tasks.workunit.client.1.smithi044.stdout:2/438: dread - d8/d19/d36/d46/d63/d65/f84 zero size 2024-04-22T23:46:11.063 INFO:tasks.workunit.client.1.smithi044.stdout:3/543: chown d9/db/d33/l93 204 1 2024-04-22T23:46:11.069 INFO:tasks.workunit.client.1.smithi044.stdout:7/531: dwrite dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:11.070 INFO:tasks.workunit.client.1.smithi044.stdout:4/566: dwrite d5/d31/d24/d69/d90/f88 [0,4194304] 0 2024-04-22T23:46:11.070 INFO:tasks.workunit.client.1.smithi044.stdout:3/544: truncate d9/db/d15/d54/d3c/f6c 1162725 0 2024-04-22T23:46:11.075 INFO:tasks.workunit.client.1.smithi044.stdout:2/439: dwrite d8/d60/f69 [0,4194304] 0 2024-04-22T23:46:11.173 INFO:tasks.workunit.client.1.smithi044.stdout:5/501: symlink d15/d16/d1d/d22/d2a/d3a/l9e 0 2024-04-22T23:46:11.173 INFO:tasks.workunit.client.1.smithi044.stdout:5/502: chown d15/d16/d1d/d22/l71 11504 1 2024-04-22T23:46:11.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:10 smithi044 ceph-mon[34463]: mgrmap e29: smithi031.fazdcv(active, since 11s), standbys: smithi044.iglcti 2024-04-22T23:46:11.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:10 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:46:11.244 INFO:tasks.workunit.client.1.smithi044.stdout:9/366: sync 2024-04-22T23:46:11.247 INFO:tasks.workunit.client.1.smithi044.stdout:7/532: creat dd/d1f/d24/fbf x:0 0 0 2024-04-22T23:46:11.248 INFO:tasks.workunit.client.1.smithi044.stdout:7/533: write dd/d1a/f7e [986709,38475] 0 2024-04-22T23:46:11.248 INFO:tasks.workunit.client.1.smithi044.stdout:4/567: rmdir d5/d31/d44/d20/d22 39 2024-04-22T23:46:11.252 INFO:tasks.workunit.client.1.smithi044.stdout:3/545: rename d9/db/d9f/d38/d74/f8d to d9/db/d33/faf 0 2024-04-22T23:46:11.253 INFO:tasks.workunit.client.1.smithi044.stdout:5/503: rmdir d15/d16/d1d/d22 39 2024-04-22T23:46:11.254 INFO:tasks.workunit.client.1.smithi044.stdout:9/367: dread d2/d5/f1f [0,4194304] 0 2024-04-22T23:46:11.255 INFO:tasks.workunit.client.1.smithi044.stdout:9/368: write d2/d5/f7 [5180157,111194] 0 2024-04-22T23:46:11.256 INFO:tasks.workunit.client.1.smithi044.stdout:0/559: link d0/d2/dd/d3e/d40/d43/d7b/c1f d0/d4c/d8b/ccf 0 2024-04-22T23:46:11.257 INFO:tasks.workunit.client.1.smithi044.stdout:0/560: dread - d0/d2e/d2f/d77/fc6 zero size 2024-04-22T23:46:11.258 INFO:tasks.workunit.client.1.smithi044.stdout:7/534: unlink dd/d55/d5e/d60/l95 0 2024-04-22T23:46:11.260 INFO:tasks.workunit.client.1.smithi044.stdout:0/561: read d0/d2/d4b/d6b/fc2 [298049,128581] 0 2024-04-22T23:46:11.261 INFO:tasks.workunit.client.1.smithi044.stdout:7/535: write dd/d27/fb6 [59544,86639] 0 2024-04-22T23:46:11.262 INFO:tasks.workunit.client.1.smithi044.stdout:4/568: symlink d5/db/d16/d4d/lba 0 2024-04-22T23:46:11.264 INFO:tasks.workunit.client.1.smithi044.stdout:0/562: chown d0/d2e/d33/f85 6016 1 2024-04-22T23:46:11.265 INFO:tasks.workunit.client.1.smithi044.stdout:9/369: dwrite d2/dc/d1d/d3f/f4c [0,4194304] 0 2024-04-22T23:46:11.267 INFO:tasks.workunit.client.1.smithi044.stdout:9/370: dread - d2/dc/d1d/d34/d38/d5c/d69/d75/f7c zero size 2024-04-22T23:46:11.267 INFO:tasks.workunit.client.1.smithi044.stdout:9/371: fsync d2/f7d 0 2024-04-22T23:46:11.270 INFO:tasks.workunit.client.1.smithi044.stdout:3/546: mknod d9/db/cb0 0 2024-04-22T23:46:11.272 INFO:tasks.workunit.client.1.smithi044.stdout:7/536: mkdir dd/d79/d8c/dc0 0 2024-04-22T23:46:11.275 INFO:tasks.workunit.client.1.smithi044.stdout:4/569: creat d5/d31/d24/d38/fbb x:0 0 0 2024-04-22T23:46:11.278 INFO:tasks.workunit.client.1.smithi044.stdout:0/563: write d0/d2/dd/d3e/fa0 [299915,94652] 0 2024-04-22T23:46:11.280 INFO:tasks.workunit.client.1.smithi044.stdout:0/564: write d0/d2/fa [5074260,115286] 0 2024-04-22T23:46:11.281 INFO:tasks.workunit.client.1.smithi044.stdout:0/565: fsync d0/d2e/d2f/f5e 0 2024-04-22T23:46:11.282 INFO:tasks.workunit.client.1.smithi044.stdout:0/566: dread - d0/d4c/fb9 zero size 2024-04-22T23:46:11.282 INFO:tasks.workunit.client.1.smithi044.stdout:9/372: symlink d2/dc/d1d/d34/d38/d5c/d69/d7a/l82 0 2024-04-22T23:46:11.285 INFO:tasks.workunit.client.1.smithi044.stdout:7/537: fdatasync dd/d1f/d24/f3f 0 2024-04-22T23:46:11.287 INFO:tasks.workunit.client.1.smithi044.stdout:4/570: creat d5/d31/d24/d69/fbc x:0 0 0 2024-04-22T23:46:11.288 INFO:tasks.workunit.client.1.smithi044.stdout:9/373: dwrite d2/dc/f18 [0,4194304] 0 2024-04-22T23:46:11.289 INFO:tasks.workunit.client.1.smithi044.stdout:9/374: dread - d2/f7f zero size 2024-04-22T23:46:11.291 INFO:tasks.workunit.client.1.smithi044.stdout:9/375: chown d2/dc/d1d/d3f/f63 1063741 1 2024-04-22T23:46:11.296 INFO:tasks.workunit.client.1.smithi044.stdout:8/582: sync 2024-04-22T23:46:11.296 INFO:tasks.workunit.client.1.smithi044.stdout:8/583: chown f5 27 1 2024-04-22T23:46:11.297 INFO:tasks.workunit.client.1.smithi044.stdout:8/584: readlink db/d14/d24/l45 0 2024-04-22T23:46:11.304 INFO:tasks.workunit.client.1.smithi044.stdout:0/567: dwrite d0/d2/dd/f11 [0,4194304] 0 2024-04-22T23:46:11.304 INFO:tasks.workunit.client.1.smithi044.stdout:4/571: symlink d5/d31/d24/d69/lbd 0 2024-04-22T23:46:11.309 INFO:tasks.workunit.client.1.smithi044.stdout:4/572: truncate d5/d31/d24/d38/f9f 1391255 0 2024-04-22T23:46:11.311 INFO:tasks.workunit.client.1.smithi044.stdout:8/585: unlink db/d14/l2e 0 2024-04-22T23:46:11.312 INFO:tasks.workunit.client.1.smithi044.stdout:3/547: dwrite f7 [0,4194304] 0 2024-04-22T23:46:11.313 INFO:tasks.workunit.client.1.smithi044.stdout:4/573: write d5/d31/d44/d7e/f98 [361542,113607] 0 2024-04-22T23:46:11.319 INFO:tasks.workunit.client.1.smithi044.stdout:7/538: link dd/d1f/d24/f1e dd/d55/d5e/fc1 0 2024-04-22T23:46:11.319 INFO:tasks.workunit.client.1.smithi044.stdout:5/504: getdents d15/d16/d1d/d22/d4f/d68 0 2024-04-22T23:46:11.320 INFO:tasks.workunit.client.1.smithi044.stdout:8/586: truncate db/d14/d84/dbb/fbd 1320836 0 2024-04-22T23:46:11.322 INFO:tasks.workunit.client.1.smithi044.stdout:9/376: dwrite d2/d5/f1f [0,4194304] 0 2024-04-22T23:46:11.326 INFO:tasks.workunit.client.1.smithi044.stdout:3/548: mkdir d9/d12/db1 0 2024-04-22T23:46:11.333 INFO:tasks.workunit.client.1.smithi044.stdout:5/505: readlink d15/d16/l35 0 2024-04-22T23:46:11.335 INFO:tasks.workunit.client.1.smithi044.stdout:4/574: dwrite d5/d31/d44/d20/d22/f6f [0,4194304] 0 2024-04-22T23:46:11.338 INFO:tasks.workunit.client.1.smithi044.stdout:8/587: readlink db/d14/d24/lb6 0 2024-04-22T23:46:11.340 INFO:tasks.workunit.client.1.smithi044.stdout:0/568: dread d0/d4c/f8f [0,4194304] 0 2024-04-22T23:46:11.341 INFO:tasks.workunit.client.1.smithi044.stdout:8/588: write db/d68/f7a [955231,31704] 0 2024-04-22T23:46:11.348 INFO:tasks.workunit.client.1.smithi044.stdout:3/549: mknod d9/d12/cb2 0 2024-04-22T23:46:11.352 INFO:tasks.workunit.client.1.smithi044.stdout:5/506: dread - d15/d16/f7b zero size 2024-04-22T23:46:11.355 INFO:tasks.workunit.client.1.smithi044.stdout:4/575: rename d5/d31/d24/d69/f6c to d5/d42/fbe 0 2024-04-22T23:46:11.358 INFO:tasks.workunit.client.1.smithi044.stdout:8/589: truncate db/d14/f78 179174 0 2024-04-22T23:46:11.359 INFO:tasks.workunit.client.1.smithi044.stdout:8/590: write db/f34 [3561300,86207] 0 2024-04-22T23:46:11.362 INFO:tasks.workunit.client.1.smithi044.stdout:3/550: dread d9/db/d9f/d38/f41 [0,4194304] 0 2024-04-22T23:46:11.363 INFO:tasks.workunit.client.1.smithi044.stdout:9/377: dread d2/f2c [0,4194304] 0 2024-04-22T23:46:11.363 INFO:tasks.workunit.client.1.smithi044.stdout:5/507: mkdir d15/d16/d2c/d38/d9f 0 2024-04-22T23:46:11.366 INFO:tasks.workunit.client.1.smithi044.stdout:4/576: unlink d5/d31/d44/d20/d22/d8d/l9d 0 2024-04-22T23:46:11.370 INFO:tasks.workunit.client.1.smithi044.stdout:8/591: creat db/d14/d24/fd7 x:0 0 0 2024-04-22T23:46:11.373 INFO:tasks.workunit.client.1.smithi044.stdout:3/551: mknod d9/db/d15/d25/d67/d6b/cb3 0 2024-04-22T23:46:11.374 INFO:tasks.workunit.client.1.smithi044.stdout:3/552: write d9/db/d9f/f56 [6164054,101843] 0 2024-04-22T23:46:11.374 INFO:tasks.workunit.client.1.smithi044.stdout:3/553: chown d9/db 3 1 2024-04-22T23:46:11.377 INFO:tasks.workunit.client.1.smithi044.stdout:5/508: symlink d15/d3c/d65/d8d/la0 0 2024-04-22T23:46:11.380 INFO:tasks.workunit.client.1.smithi044.stdout:9/378: creat d2/dc/d1d/d34/d38/d5c/f83 x:0 0 0 2024-04-22T23:46:11.381 INFO:tasks.workunit.client.1.smithi044.stdout:8/592: rmdir db/d14/dc7/dc9/dd1 39 2024-04-22T23:46:11.383 INFO:tasks.workunit.client.1.smithi044.stdout:0/569: truncate d0/d2e/f70 2138645 0 2024-04-22T23:46:11.385 INFO:tasks.workunit.client.1.smithi044.stdout:0/570: chown d0/d2/dd/f1a 137390151 1 2024-04-22T23:46:11.385 INFO:tasks.workunit.client.1.smithi044.stdout:3/554: unlink d9/db/d33/f7c 0 2024-04-22T23:46:11.386 INFO:tasks.workunit.client.1.smithi044.stdout:9/379: dwrite d2/d6/f74 [0,4194304] 0 2024-04-22T23:46:11.387 INFO:tasks.workunit.client.1.smithi044.stdout:5/509: unlink d15/d5d/d64/f6a 0 2024-04-22T23:46:11.393 INFO:tasks.workunit.client.1.smithi044.stdout:5/510: dwrite d15/d16/d1d/f49 [0,4194304] 0 2024-04-22T23:46:11.396 INFO:tasks.workunit.client.1.smithi044.stdout:8/593: unlink db/l1f 0 2024-04-22T23:46:11.404 INFO:tasks.workunit.client.1.smithi044.stdout:3/555: unlink d9/d12/cb2 0 2024-04-22T23:46:11.407 INFO:tasks.workunit.client.1.smithi044.stdout:0/571: symlink d0/d2e/d2f/ld0 0 2024-04-22T23:46:11.408 INFO:tasks.workunit.client.1.smithi044.stdout:0/572: readlink d0/d2/dd/d3e/d40/d67/lcc 0 2024-04-22T23:46:11.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/594: rename db/c54 to db/d68/d8b/da7/cd8 0 2024-04-22T23:46:11.416 INFO:tasks.workunit.client.1.smithi044.stdout:5/511: dwrite d15/d16/d2c/f66 [0,4194304] 0 2024-04-22T23:46:11.420 INFO:tasks.workunit.client.1.smithi044.stdout:8/595: dread - db/f3f zero size 2024-04-22T23:46:11.423 INFO:tasks.workunit.client.1.smithi044.stdout:0/573: dwrite d0/d2/d4b/f68 [0,4194304] 0 2024-04-22T23:46:11.426 INFO:tasks.workunit.client.1.smithi044.stdout:3/556: rename d9/db/d15/d54/f90 to d9/d14/d7d/fb4 0 2024-04-22T23:46:11.431 INFO:tasks.workunit.client.1.smithi044.stdout:5/512: mkdir d15/d16/d1d/d22/d2a/d3a/d43/da1 0 2024-04-22T23:46:11.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/596: creat db/d6c/fd9 x:0 0 0 2024-04-22T23:46:11.435 INFO:tasks.workunit.client.1.smithi044.stdout:9/380: write d2/f2c [4058089,52028] 0 2024-04-22T23:46:11.438 INFO:tasks.workunit.client.1.smithi044.stdout:1/533: sync 2024-04-22T23:46:11.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/440: sync 2024-04-22T23:46:11.438 INFO:tasks.workunit.client.1.smithi044.stdout:6/418: sync 2024-04-22T23:46:11.440 INFO:tasks.workunit.client.1.smithi044.stdout:1/534: fdatasync d1/d8/d20/d45/f4f 0 2024-04-22T23:46:11.443 INFO:tasks.workunit.client.1.smithi044.stdout:3/557: dread d9/db/d15/d25/d4c/f6a [0,4194304] 0 2024-04-22T23:46:11.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:10 smithi031 ceph-mon[26721]: mgrmap e29: smithi031.fazdcv(active, since 11s), standbys: smithi044.iglcti 2024-04-22T23:46:11.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:10 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:46:11.444 INFO:tasks.workunit.client.1.smithi044.stdout:9/381: creat d2/dc/d1d/d34/f84 x:0 0 0 2024-04-22T23:46:11.445 INFO:tasks.workunit.client.1.smithi044.stdout:9/382: chown d2/d6/fa 9 1 2024-04-22T23:46:11.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/513: mkdir d15/d16/d1d/da2 0 2024-04-22T23:46:11.452 INFO:tasks.workunit.client.1.smithi044.stdout:1/535: creat d1/d8/d1c/d23/d48/fc0 x:0 0 0 2024-04-22T23:46:11.453 INFO:tasks.workunit.client.1.smithi044.stdout:8/597: symlink db/d14/dc7/dc9/lda 0 2024-04-22T23:46:11.460 INFO:tasks.workunit.client.1.smithi044.stdout:5/514: symlink d15/d16/d1d/d22/d2a/d3a/d43/la3 0 2024-04-22T23:46:11.462 INFO:tasks.workunit.client.1.smithi044.stdout:1/536: mknod d1/d8/d20/d45/d5d/cc1 0 2024-04-22T23:46:11.467 INFO:tasks.workunit.client.1.smithi044.stdout:0/574: link d0/d2/dd/d3e/f4f d0/d2/dd/d3e/d40/d43/d79/dbb/fd1 0 2024-04-22T23:46:11.470 INFO:tasks.workunit.client.1.smithi044.stdout:6/419: write d5/dd/f10 [2788713,29682] 0 2024-04-22T23:46:11.471 INFO:tasks.workunit.client.1.smithi044.stdout:6/420: stat d5/d3f/d4c/d23/f26 0 2024-04-22T23:46:11.472 INFO:tasks.workunit.client.1.smithi044.stdout:6/421: write d5/dd/d28/f2e [423684,91676] 0 2024-04-22T23:46:11.473 INFO:tasks.workunit.client.1.smithi044.stdout:6/422: readlink d5/d3f/lb 0 2024-04-22T23:46:11.476 INFO:tasks.workunit.client.1.smithi044.stdout:2/441: rename d8/d19/d36/d46/d54 to d8/d19/d96 0 2024-04-22T23:46:11.477 INFO:tasks.workunit.client.1.smithi044.stdout:8/598: symlink db/d68/d8b/dca/dd6/ldb 0 2024-04-22T23:46:11.478 INFO:tasks.workunit.client.1.smithi044.stdout:3/558: truncate d9/db/f71 2140202 0 2024-04-22T23:46:11.480 INFO:tasks.workunit.client.1.smithi044.stdout:1/537: mkdir d1/d8/d20/d45/d75/d77/dbf/dc2 0 2024-04-22T23:46:11.481 INFO:tasks.workunit.client.1.smithi044.stdout:1/538: write d1/d14/d95/f96 [1798816,78314] 0 2024-04-22T23:46:11.483 INFO:tasks.workunit.client.1.smithi044.stdout:1/539: write d1/d8/d1c/d7b/d80/d4b/f88 [571466,126341] 0 2024-04-22T23:46:11.484 INFO:tasks.workunit.client.1.smithi044.stdout:1/540: read d1/d8/fa9 [122267,50521] 0 2024-04-22T23:46:11.491 INFO:tasks.workunit.client.1.smithi044.stdout:1/541: dwrite d1/d8/f12 [0,4194304] 0 2024-04-22T23:46:11.496 INFO:tasks.workunit.client.1.smithi044.stdout:5/515: mknod d15/d16/ca4 0 2024-04-22T23:46:11.496 INFO:tasks.workunit.client.1.smithi044.stdout:5/516: chown d15/d5d/d64/d6f/d7f 1 1 2024-04-22T23:46:11.498 INFO:tasks.workunit.client.1.smithi044.stdout:5/517: write d15/d5d/d64/d6f/f8a [906835,70314] 0 2024-04-22T23:46:11.499 INFO:tasks.workunit.client.1.smithi044.stdout:5/518: chown d15/f94 7281 1 2024-04-22T23:46:11.500 INFO:tasks.workunit.client.1.smithi044.stdout:6/423: rename d5/d3f/d50/d79/l7e to d5/dd/d28/d75/d88/l8a 0 2024-04-22T23:46:11.502 INFO:tasks.workunit.client.1.smithi044.stdout:9/383: dread d2/dc/d1d/d34/f49 [0,4194304] 0 2024-04-22T23:46:11.505 INFO:tasks.workunit.client.1.smithi044.stdout:2/442: fsync d8/d19/d5d/f83 0 2024-04-22T23:46:11.509 INFO:tasks.workunit.client.1.smithi044.stdout:6/424: dwrite d5/d3f/d50/d79/f81 [0,4194304] 0 2024-04-22T23:46:11.516 INFO:tasks.workunit.client.1.smithi044.stdout:5/519: creat d15/d16/d2c/d38/fa5 x:0 0 0 2024-04-22T23:46:11.520 INFO:tasks.workunit.client.1.smithi044.stdout:2/443: creat d8/d19/d36/d44/f97 x:0 0 0 2024-04-22T23:46:11.522 INFO:tasks.workunit.client.1.smithi044.stdout:1/542: dwrite d1/d8/d1c/d7b/d80/d46/f68 [0,4194304] 0 2024-04-22T23:46:11.524 INFO:tasks.workunit.client.1.smithi044.stdout:0/575: link d0/d2/dd/d3e/d40/d43/d7b/c24 d0/d4c/d8b/cd2 0 2024-04-22T23:46:11.527 INFO:tasks.workunit.client.1.smithi044.stdout:0/576: write d0/d2/dd/f11 [4593808,120534] 0 2024-04-22T23:46:11.529 INFO:tasks.workunit.client.1.smithi044.stdout:3/559: rename d9/db/d9f/d38/c97 to d9/db/d15/d25/cb5 0 2024-04-22T23:46:11.531 INFO:tasks.workunit.client.1.smithi044.stdout:9/384: mknod d2/d5/c85 0 2024-04-22T23:46:11.534 INFO:tasks.workunit.client.1.smithi044.stdout:2/444: creat d8/d19/d1c/f98 x:0 0 0 2024-04-22T23:46:11.535 INFO:tasks.workunit.client.1.smithi044.stdout:2/445: fdatasync d8/d27/f4d 0 2024-04-22T23:46:11.538 INFO:tasks.workunit.client.1.smithi044.stdout:9/385: dwrite d2/f2c [4194304,4194304] 0 2024-04-22T23:46:11.543 INFO:tasks.workunit.client.1.smithi044.stdout:0/577: unlink d0/d2/dd/d3e/fc5 0 2024-04-22T23:46:11.545 INFO:tasks.workunit.client.1.smithi044.stdout:5/520: rename d15/d16/d1d/d22/l34 to d15/d16/d1d/d22/d81/la6 0 2024-04-22T23:46:11.548 INFO:tasks.workunit.client.1.smithi044.stdout:3/560: creat d9/db/d15/d25/fb6 x:0 0 0 2024-04-22T23:46:11.548 INFO:tasks.workunit.client.1.smithi044.stdout:3/561: read - d9/db/fa2 zero size 2024-04-22T23:46:11.549 INFO:tasks.workunit.client.1.smithi044.stdout:3/562: write d9/db/d15/d54/d3c/f81 [4619662,102974] 0 2024-04-22T23:46:11.552 INFO:tasks.workunit.client.1.smithi044.stdout:6/425: link d5/d30/d73/l7a d5/dd/d28/d75/l8b 0 2024-04-22T23:46:11.556 INFO:tasks.workunit.client.1.smithi044.stdout:9/386: fdatasync d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f 0 2024-04-22T23:46:11.560 INFO:tasks.workunit.client.1.smithi044.stdout:1/543: dwrite d1/d8/d1c/d23/f42 [0,4194304] 0 2024-04-22T23:46:11.562 INFO:tasks.workunit.client.1.smithi044.stdout:9/387: dread d2/d5/f55 [0,4194304] 0 2024-04-22T23:46:11.566 INFO:tasks.workunit.client.1.smithi044.stdout:3/563: creat d9/db/d15/d54/d3c/fb7 x:0 0 0 2024-04-22T23:46:11.567 INFO:tasks.workunit.client.1.smithi044.stdout:6/426: creat d5/d3f/d50/d79/f8c x:0 0 0 2024-04-22T23:46:11.568 INFO:tasks.workunit.client.1.smithi044.stdout:9/388: dread d2/d6/f53 [0,4194304] 0 2024-04-22T23:46:11.570 INFO:tasks.workunit.client.1.smithi044.stdout:9/389: write d2/dc/d1d/d3f/f6e [3458160,18142] 0 2024-04-22T23:46:11.576 INFO:tasks.workunit.client.1.smithi044.stdout:0/578: dwrite d0/d2/dd/f2a [0,4194304] 0 2024-04-22T23:46:11.578 INFO:tasks.workunit.client.1.smithi044.stdout:6/427: stat d5/d30/d73/l7a 0 2024-04-22T23:46:11.580 INFO:tasks.workunit.client.1.smithi044.stdout:6/428: write d5/dd/f1b [284535,2803] 0 2024-04-22T23:46:11.581 INFO:tasks.workunit.client.1.smithi044.stdout:9/390: mknod d2/dc/d1d/d3f/c86 0 2024-04-22T23:46:11.583 INFO:tasks.workunit.client.1.smithi044.stdout:2/446: link d8/d19/d36/d46/c50 d8/d19/d36/d46/d63/d65/d6c/c99 0 2024-04-22T23:46:11.589 INFO:tasks.workunit.client.1.smithi044.stdout:2/447: dwrite f3 [0,4194304] 0 2024-04-22T23:46:11.590 INFO:tasks.workunit.client.1.smithi044.stdout:7/539: sync 2024-04-22T23:46:11.590 INFO:tasks.workunit.client.1.smithi044.stdout:2/448: chown d8/d27/d5b 21084199 1 2024-04-22T23:46:11.591 INFO:tasks.workunit.client.1.smithi044.stdout:5/521: rename d15/d16/d1d/d22/l71 to d15/d16/d1d/la7 0 2024-04-22T23:46:11.591 INFO:tasks.workunit.client.1.smithi044.stdout:7/540: read dd/d1f/d24/fa2 [2566854,121594] 0 2024-04-22T23:46:11.597 INFO:tasks.workunit.client.1.smithi044.stdout:5/522: dread d15/d16/d1d/d22/d2a/d3a/f4b [0,4194304] 0 2024-04-22T23:46:11.601 INFO:tasks.workunit.client.1.smithi044.stdout:9/391: creat d2/dc/f87 x:0 0 0 2024-04-22T23:46:11.609 INFO:tasks.workunit.client.1.smithi044.stdout:1/544: rename d1/d8/d20/d45/l6c to d1/d8/d20/d45/d75/d77/dbf/dc2/lc3 0 2024-04-22T23:46:11.615 INFO:tasks.workunit.client.1.smithi044.stdout:5/523: rmdir d15/d5d/d64 39 2024-04-22T23:46:11.616 INFO:tasks.workunit.client.1.smithi044.stdout:0/579: dread d0/d2/dd/d3e/d40/d43/d79/dbb/fd1 [0,4194304] 0 2024-04-22T23:46:11.622 INFO:tasks.workunit.client.1.smithi044.stdout:1/545: readlink d1/d14/l30 0 2024-04-22T23:46:11.624 INFO:tasks.workunit.client.1.smithi044.stdout:6/429: rename d5/dd/d28/l70 to d5/d30/d73/l8d 0 2024-04-22T23:46:11.627 INFO:tasks.workunit.client.1.smithi044.stdout:2/449: dwrite f6 [0,4194304] 0 2024-04-22T23:46:11.628 INFO:tasks.workunit.client.1.smithi044.stdout:5/524: creat d15/d16/d1d/d22/d2a/d3a/d43/fa8 x:0 0 0 2024-04-22T23:46:11.631 INFO:tasks.workunit.client.1.smithi044.stdout:1/546: dwrite d1/d8/d20/d47/d4a/f54 [0,4194304] 0 2024-04-22T23:46:11.634 INFO:tasks.workunit.client.1.smithi044.stdout:1/547: write d1/d8/d1c/d23/d3e/f7e [843524,35434] 0 2024-04-22T23:46:11.636 INFO:tasks.workunit.client.1.smithi044.stdout:0/580: rename d0/d2/dd/d3e/d40/d43/f9b to d0/d4c/d59/d5a/fd3 0 2024-04-22T23:46:11.642 INFO:tasks.workunit.client.1.smithi044.stdout:2/450: dwrite d8/d19/d36/d46/f4e [0,4194304] 0 2024-04-22T23:46:11.644 INFO:tasks.workunit.client.1.smithi044.stdout:9/392: dwrite d2/d6/f53 [0,4194304] 0 2024-04-22T23:46:11.645 INFO:tasks.workunit.client.1.smithi044.stdout:1/548: creat d1/d8/d1c/d23/d48/d99/fc4 x:0 0 0 2024-04-22T23:46:11.653 INFO:tasks.workunit.client.1.smithi044.stdout:0/581: rename d0/d2/dd/d3e/d40/f84 to d0/d2/d4b/d6b/fd4 0 2024-04-22T23:46:11.654 INFO:tasks.workunit.client.1.smithi044.stdout:0/582: dread - d0/d2e/d33/f7d zero size 2024-04-22T23:46:11.655 INFO:tasks.workunit.client.1.smithi044.stdout:0/583: chown d0/l8c 1487157 1 2024-04-22T23:46:11.656 INFO:tasks.workunit.client.1.smithi044.stdout:0/584: write d0/d2/f8e [3729657,64247] 0 2024-04-22T23:46:11.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/393: fsync d2/dc/f3a 0 2024-04-22T23:46:11.662 INFO:tasks.workunit.client.1.smithi044.stdout:9/394: fsync d2/dc/f87 0 2024-04-22T23:46:11.668 INFO:tasks.workunit.client.1.smithi044.stdout:2/451: link d8/d19/d5d/f83 d8/d27/d5b/f9a 0 2024-04-22T23:46:11.670 INFO:tasks.workunit.client.1.smithi044.stdout:1/549: mknod d1/d8/d34/da1/dab/cc5 0 2024-04-22T23:46:11.672 INFO:tasks.workunit.client.1.smithi044.stdout:5/525: getdents d15/d3c 0 2024-04-22T23:46:11.677 INFO:tasks.workunit.client.1.smithi044.stdout:6/430: write d5/f4d [690891,58946] 0 2024-04-22T23:46:11.684 INFO:tasks.workunit.client.1.smithi044.stdout:0/585: dread d0/d2/dd/d3e/d40/f73 [4194304,4194304] 0 2024-04-22T23:46:11.684 INFO:tasks.workunit.client.1.smithi044.stdout:1/550: truncate d1/d8/f26 1174827 0 2024-04-22T23:46:11.684 INFO:tasks.workunit.client.1.smithi044.stdout:9/395: dread d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f [0,4194304] 0 2024-04-22T23:46:11.685 INFO:tasks.workunit.client.1.smithi044.stdout:0/586: fdatasync d0/d2e/dbf/fc9 0 2024-04-22T23:46:11.685 INFO:tasks.workunit.client.1.smithi044.stdout:1/551: write d1/d8/f19 [4041656,52667] 0 2024-04-22T23:46:11.687 INFO:tasks.workunit.client.1.smithi044.stdout:0/587: fdatasync d0/d2e/d33/f85 0 2024-04-22T23:46:11.688 INFO:tasks.workunit.client.1.smithi044.stdout:9/396: chown d2/dc/d1d/d34/d38/d5c/d69/d7a/l79 26 1 2024-04-22T23:46:11.689 INFO:tasks.workunit.client.1.smithi044.stdout:9/397: chown d2/f7f 5929 1 2024-04-22T23:46:11.690 INFO:tasks.workunit.client.1.smithi044.stdout:6/431: dread - d5/dd/f68 zero size 2024-04-22T23:46:11.693 INFO:tasks.workunit.client.1.smithi044.stdout:1/552: dwrite d1/d8/d1c/d7b/f65 [0,4194304] 0 2024-04-22T23:46:11.694 INFO:tasks.workunit.client.1.smithi044.stdout:1/553: readlink d1/d8/d34/l35 0 2024-04-22T23:46:11.695 INFO:tasks.workunit.client.1.smithi044.stdout:5/526: link d15/d16/d1d/d22/l2b d15/d5d/d82/la9 0 2024-04-22T23:46:11.698 INFO:tasks.workunit.client.1.smithi044.stdout:0/588: mknod d0/d2/dd/d3e/d40/d67/cd5 0 2024-04-22T23:46:11.700 INFO:tasks.workunit.client.1.smithi044.stdout:1/554: mknod d1/d8/d1c/d23/d3e/cc6 0 2024-04-22T23:46:11.706 INFO:tasks.workunit.client.1.smithi044.stdout:9/398: dread d2/f4 [0,4194304] 0 2024-04-22T23:46:11.707 INFO:tasks.workunit.client.1.smithi044.stdout:5/527: creat d15/d5d/d64/d6f/faa x:0 0 0 2024-04-22T23:46:11.708 INFO:tasks.workunit.client.1.smithi044.stdout:6/432: creat d5/d3f/d4c/d40/d53/f8e x:0 0 0 2024-04-22T23:46:11.710 INFO:tasks.workunit.client.1.smithi044.stdout:6/433: chown d5/d3f/d4c/d4f/l66 416556 1 2024-04-22T23:46:11.712 INFO:tasks.workunit.client.1.smithi044.stdout:9/399: mknod d2/dc/d1d/d34/d38/d4a/c88 0 2024-04-22T23:46:11.712 INFO:tasks.workunit.client.1.smithi044.stdout:1/555: dwrite d1/d8/d20/d45/f2a [4194304,4194304] 0 2024-04-22T23:46:11.713 INFO:tasks.workunit.client.1.smithi044.stdout:6/434: write d5/dd/f6a [88529,45948] 0 2024-04-22T23:46:11.714 INFO:tasks.workunit.client.1.smithi044.stdout:9/400: chown d2/dc/d1d/d34/f84 1540866 1 2024-04-22T23:46:11.716 INFO:tasks.workunit.client.1.smithi044.stdout:1/556: creat d1/d14/fc7 x:0 0 0 2024-04-22T23:46:11.719 INFO:tasks.workunit.client.1.smithi044.stdout:1/557: mknod d1/d8/d1c/d23/d48/cc8 0 2024-04-22T23:46:11.738 INFO:tasks.workunit.client.1.smithi044.stdout:4/577: sync 2024-04-22T23:46:11.738 INFO:tasks.workunit.client.1.smithi044.stdout:8/599: sync 2024-04-22T23:46:11.743 INFO:tasks.workunit.client.1.smithi044.stdout:8/600: dwrite db/f44 [0,4194304] 0 2024-04-22T23:46:11.765 INFO:tasks.workunit.client.1.smithi044.stdout:4/578: rename d5/db/f79 to d5/d31/d44/d20/d22/fbf 0 2024-04-22T23:46:11.772 INFO:tasks.workunit.client.1.smithi044.stdout:4/579: symlink d5/db/d16/lc0 0 2024-04-22T23:46:11.775 INFO:tasks.workunit.client.1.smithi044.stdout:4/580: truncate d5/db/d34/fb1 971162 0 2024-04-22T23:46:11.776 INFO:tasks.workunit.client.1.smithi044.stdout:8/601: creat db/d14/dc7/dc9/fdc x:0 0 0 2024-04-22T23:46:11.778 INFO:tasks.workunit.client.1.smithi044.stdout:8/602: dread - db/d14/d84/fb4 zero size 2024-04-22T23:46:11.779 INFO:tasks.workunit.client.1.smithi044.stdout:2/452: dwrite d8/d19/d36/f3d [0,4194304] 0 2024-04-22T23:46:11.779 INFO:tasks.workunit.client.1.smithi044.stdout:4/581: stat d5/d31/d44/d20/d22/f35 0 2024-04-22T23:46:11.781 INFO:tasks.workunit.client.1.smithi044.stdout:4/582: write d5/db/d34/fb9 [489385,51778] 0 2024-04-22T23:46:11.793 INFO:tasks.workunit.client.1.smithi044.stdout:4/583: creat d5/d52/fc1 x:0 0 0 2024-04-22T23:46:11.794 INFO:tasks.workunit.client.1.smithi044.stdout:8/603: mkdir db/ddd 0 2024-04-22T23:46:11.797 INFO:tasks.workunit.client.1.smithi044.stdout:2/453: rename d8/f10 to d8/d19/d96/f9b 0 2024-04-22T23:46:11.802 INFO:tasks.workunit.client.1.smithi044.stdout:8/604: symlink db/d68/d8b/dca/lde 0 2024-04-22T23:46:11.803 INFO:tasks.workunit.client.1.smithi044.stdout:0/589: write d0/d2/dd/d3e/d40/d43/d7b/f5c [81685,111376] 0 2024-04-22T23:46:11.805 INFO:tasks.workunit.client.1.smithi044.stdout:4/584: getdents d5/d42 0 2024-04-22T23:46:11.809 INFO:tasks.workunit.client.1.smithi044.stdout:0/590: mkdir d0/dd6 0 2024-04-22T23:46:11.810 INFO:tasks.workunit.client.1.smithi044.stdout:9/401: rmdir d2 39 2024-04-22T23:46:11.811 INFO:tasks.workunit.client.1.smithi044.stdout:8/605: dwrite db/d14/d72/f81 [0,4194304] 0 2024-04-22T23:46:11.812 INFO:tasks.workunit.client.1.smithi044.stdout:0/591: fsync d0/d2/dd/d3e/f3f 0 2024-04-22T23:46:11.816 INFO:tasks.workunit.client.1.smithi044.stdout:9/402: chown d2/c31 7 1 2024-04-22T23:46:11.816 INFO:tasks.workunit.client.1.smithi044.stdout:5/528: dwrite d15/d16/d2c/f5a [4194304,4194304] 0 2024-04-22T23:46:11.819 INFO:tasks.workunit.client.1.smithi044.stdout:0/592: creat d0/d60/fd7 x:0 0 0 2024-04-22T23:46:11.823 INFO:tasks.workunit.client.1.smithi044.stdout:9/403: creat d2/dc/f89 x:0 0 0 2024-04-22T23:46:11.824 INFO:tasks.workunit.client.1.smithi044.stdout:8/606: dwrite db/d14/d84/dbb/fbd [0,4194304] 0 2024-04-22T23:46:11.827 INFO:tasks.workunit.client.1.smithi044.stdout:9/404: write d2/f7d [699344,79800] 0 2024-04-22T23:46:11.827 INFO:tasks.workunit.client.1.smithi044.stdout:1/558: truncate d1/d8/d1c/d23/d3e/f7e 258400 0 2024-04-22T23:46:11.830 INFO:tasks.workunit.client.1.smithi044.stdout:8/607: read db/d68/d74/fb0 [374829,104288] 0 2024-04-22T23:46:11.831 INFO:tasks.workunit.client.1.smithi044.stdout:4/585: rmdir d5/d52 39 2024-04-22T23:46:11.831 INFO:tasks.workunit.client.1.smithi044.stdout:5/529: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:11.832 INFO:tasks.workunit.client.1.smithi044.stdout:4/586: readlink d5/d31/d44/d20/l70 0 2024-04-22T23:46:11.838 INFO:tasks.workunit.client.1.smithi044.stdout:9/405: stat d2/dc/d1d/d3f/l47 0 2024-04-22T23:46:11.844 INFO:tasks.workunit.client.1.smithi044.stdout:1/559: dread d1/d8/d20/d45/f70 [0,4194304] 0 2024-04-22T23:46:11.844 INFO:tasks.workunit.client.1.smithi044.stdout:8/608: dwrite db/f44 [0,4194304] 0 2024-04-22T23:46:11.845 INFO:tasks.workunit.client.1.smithi044.stdout:8/609: dread - db/d14/d84/f99 zero size 2024-04-22T23:46:11.850 INFO:tasks.workunit.client.1.smithi044.stdout:9/406: fdatasync d2/d5/f55 0 2024-04-22T23:46:11.853 INFO:tasks.workunit.client.1.smithi044.stdout:1/560: chown d1/d8/d20/d45/l76 3479687 1 2024-04-22T23:46:11.856 INFO:tasks.workunit.client.1.smithi044.stdout:4/587: link d5/db/d16/fa0 d5/d31/d24/da6/fc2 0 2024-04-22T23:46:11.856 INFO:tasks.workunit.client.1.smithi044.stdout:8/610: mknod db/d14/d9e/cdf 0 2024-04-22T23:46:11.858 INFO:tasks.workunit.client.1.smithi044.stdout:9/407: symlink d2/d6/d4d/l8a 0 2024-04-22T23:46:11.859 INFO:tasks.workunit.client.1.smithi044.stdout:1/561: creat d1/d8/d1c/d7b/d80/d46/fc9 x:0 0 0 2024-04-22T23:46:11.862 INFO:tasks.workunit.client.1.smithi044.stdout:5/530: rename d15/d16/d2c to d15/d16/d1d/dab 0 2024-04-22T23:46:11.863 INFO:tasks.workunit.client.1.smithi044.stdout:5/531: fsync d15/d5d/d64/d6f/f74 0 2024-04-22T23:46:11.865 INFO:tasks.workunit.client.1.smithi044.stdout:8/611: dread db/d14/f5a [0,4194304] 0 2024-04-22T23:46:11.865 INFO:tasks.workunit.client.1.smithi044.stdout:1/562: truncate d1/d8/d34/f43 1238350 0 2024-04-22T23:46:11.867 INFO:tasks.workunit.client.1.smithi044.stdout:4/588: rename d5/fb0 to d5/db/d16/d4d/fc3 0 2024-04-22T23:46:11.869 INFO:tasks.workunit.client.1.smithi044.stdout:4/589: chown d5/db/d34/f51 731 1 2024-04-22T23:46:11.870 INFO:tasks.workunit.client.1.smithi044.stdout:1/563: truncate d1/d8/d1c/d7b/d80/d4b/f9a 1018231 0 2024-04-22T23:46:11.872 INFO:tasks.workunit.client.1.smithi044.stdout:1/564: chown d1/d8/d20/f2f 7232 1 2024-04-22T23:46:11.873 INFO:tasks.workunit.client.1.smithi044.stdout:5/532: dwrite d15/d16/d1d/dab/d38/f50 [0,4194304] 0 2024-04-22T23:46:11.874 INFO:tasks.workunit.client.1.smithi044.stdout:8/612: creat db/d14/d9e/fe0 x:0 0 0 2024-04-22T23:46:11.875 INFO:tasks.workunit.client.1.smithi044.stdout:8/613: chown db/d68/d8b/l96 101342 1 2024-04-22T23:46:11.876 INFO:tasks.workunit.client.1.smithi044.stdout:4/590: dread - d5/d52/fc1 zero size 2024-04-22T23:46:11.884 INFO:tasks.workunit.client.1.smithi044.stdout:1/565: creat d1/d8/d20/d9e/db9/fca x:0 0 0 2024-04-22T23:46:11.884 INFO:tasks.workunit.client.1.smithi044.stdout:5/533: dwrite d15/d16/d1d/f42 [0,4194304] 0 2024-04-22T23:46:11.889 INFO:tasks.workunit.client.1.smithi044.stdout:9/408: write d2/dc/f15 [3186148,112563] 0 2024-04-22T23:46:11.891 INFO:tasks.workunit.client.1.smithi044.stdout:5/534: mknod d15/d16/d1d/d22/d81/cac 0 2024-04-22T23:46:11.895 INFO:tasks.workunit.client.1.smithi044.stdout:1/566: mkdir d1/d8/d20/d45/d75/d77/dad/dcb 0 2024-04-22T23:46:11.900 INFO:tasks.workunit.client.1.smithi044.stdout:1/567: mkdir d1/d8/d20/d45/d75/d77/dbf/dc2/dcc 0 2024-04-22T23:46:11.902 INFO:tasks.workunit.client.1.smithi044.stdout:5/535: mknod d15/d16/d1d/d22/cad 0 2024-04-22T23:46:11.904 INFO:tasks.workunit.client.1.smithi044.stdout:8/614: link db/d14/f41 db/d14/dc7/dc9/dd1/fe1 0 2024-04-22T23:46:11.906 INFO:tasks.workunit.client.1.smithi044.stdout:5/536: write d15/d16/d1d/d22/d2a/d3a/d43/d80/f8c [4366441,118540] 0 2024-04-22T23:46:11.906 INFO:tasks.workunit.client.1.smithi044.stdout:1/568: chown d1/d8/d20/d47/f85 0 1 2024-04-22T23:46:11.907 INFO:tasks.workunit.client.1.smithi044.stdout:8/615: fsync db/d68/d8b/da7/fcd 0 2024-04-22T23:46:11.908 INFO:tasks.workunit.client.1.smithi044.stdout:8/616: readlink db/d52/la8 0 2024-04-22T23:46:11.908 INFO:tasks.workunit.client.1.smithi044.stdout:1/569: chown d1/l2b 306447 1 2024-04-22T23:46:11.910 INFO:tasks.workunit.client.1.smithi044.stdout:4/591: dwrite d5/d31/d44/f29 [0,4194304] 0 2024-04-22T23:46:11.910 INFO:tasks.workunit.client.1.smithi044.stdout:9/409: getdents d2/d9 0 2024-04-22T23:46:11.911 INFO:tasks.workunit.client.1.smithi044.stdout:1/570: stat d1/d8/d1c/d23/d48/d72 0 2024-04-22T23:46:11.911 INFO:tasks.workunit.client.1.smithi044.stdout:9/410: readlink d2/d6/l21 0 2024-04-22T23:46:11.913 INFO:tasks.workunit.client.1.smithi044.stdout:1/571: stat d1/d8/d20 0 2024-04-22T23:46:11.914 INFO:tasks.workunit.client.1.smithi044.stdout:9/411: write d2/f7d [1468766,21686] 0 2024-04-22T23:46:11.914 INFO:tasks.workunit.client.1.smithi044.stdout:8/617: write db/d52/d9c/dad/fb3 [777816,117370] 0 2024-04-22T23:46:11.915 INFO:tasks.workunit.client.1.smithi044.stdout:4/592: dread d5/d31/d24/d38/f40 [0,4194304] 0 2024-04-22T23:46:11.917 INFO:tasks.workunit.client.1.smithi044.stdout:1/572: write d1/d14/f52 [1889089,108720] 0 2024-04-22T23:46:11.918 INFO:tasks.workunit.client.1.smithi044.stdout:4/593: stat d5/d31/d24/da6/cb3 0 2024-04-22T23:46:11.919 INFO:tasks.workunit.client.1.smithi044.stdout:1/573: stat d1/d8/d20/d45/d75/d77/f89 0 2024-04-22T23:46:11.920 INFO:tasks.workunit.client.1.smithi044.stdout:1/574: read d1/d8/d20/d47/d4a/f54 [3684484,101244] 0 2024-04-22T23:46:11.923 INFO:tasks.workunit.client.1.smithi044.stdout:8/618: mkdir db/d14/d72/de2 0 2024-04-22T23:46:11.925 INFO:tasks.workunit.client.1.smithi044.stdout:4/594: unlink d5/d31/d24/d69/lbd 0 2024-04-22T23:46:11.928 INFO:tasks.workunit.client.1.smithi044.stdout:3/564: sync 2024-04-22T23:46:11.928 INFO:tasks.workunit.client.1.smithi044.stdout:7/541: sync 2024-04-22T23:46:11.933 INFO:tasks.workunit.client.1.smithi044.stdout:5/537: truncate d15/d16/d1d/d22/d4f/d68/f6c 3227570 0 2024-04-22T23:46:11.933 INFO:tasks.workunit.client.1.smithi044.stdout:1/575: mknod d1/d8/d20/ccd 0 2024-04-22T23:46:11.934 INFO:tasks.workunit.client.1.smithi044.stdout:7/542: dwrite dd/d55/f58 [0,4194304] 0 2024-04-22T23:46:11.938 INFO:tasks.workunit.client.1.smithi044.stdout:1/576: truncate d1/d8/d20/d45/d75/d77/f89 649843 0 2024-04-22T23:46:11.940 INFO:tasks.workunit.client.1.smithi044.stdout:9/412: dwrite d2/d6/f33 [4194304,4194304] 0 2024-04-22T23:46:11.941 INFO:tasks.workunit.client.1.smithi044.stdout:3/565: rename l4 to d9/db/d33/d5d/d64/lb8 0 2024-04-22T23:46:11.943 INFO:tasks.workunit.client.1.smithi044.stdout:1/577: write d1/d8/d20/d45/f4f [3327957,113690] 0 2024-04-22T23:46:11.944 INFO:tasks.workunit.client.1.smithi044.stdout:8/619: creat db/ddd/fe3 x:0 0 0 2024-04-22T23:46:11.945 INFO:tasks.workunit.client.1.smithi044.stdout:5/538: write d15/d16/d1d/d22/d2a/f79 [4991686,119979] 0 2024-04-22T23:46:11.948 INFO:tasks.workunit.client.1.smithi044.stdout:4/595: link d5/d31/d24/d69/fa2 d5/d31/d44/d20/d93/fc4 0 2024-04-22T23:46:11.949 INFO:tasks.workunit.client.1.smithi044.stdout:7/543: dwrite dd/d27/f3e [0,4194304] 0 2024-04-22T23:46:11.964 INFO:tasks.workunit.client.1.smithi044.stdout:9/413: mkdir d2/d6/d42/d8b 0 2024-04-22T23:46:11.965 INFO:tasks.workunit.client.1.smithi044.stdout:1/578: readlink d1/d8/d20/d47/d4a/lbc 0 2024-04-22T23:46:11.965 INFO:tasks.workunit.client.1.smithi044.stdout:5/539: symlink d15/d5d/d64/lae 0 2024-04-22T23:46:11.965 INFO:tasks.workunit.client.1.smithi044.stdout:8/620: creat db/d68/d8b/da7/fe4 x:0 0 0 2024-04-22T23:46:11.969 INFO:tasks.workunit.client.1.smithi044.stdout:7/544: creat dd/d99/fc2 x:0 0 0 2024-04-22T23:46:11.989 INFO:tasks.workunit.client.1.smithi044.stdout:3/566: dwrite d9/db/f13 [0,4194304] 0 2024-04-22T23:46:12.002 INFO:tasks.workunit.client.1.smithi044.stdout:9/414: fdatasync d2/f19 0 2024-04-22T23:46:12.003 INFO:tasks.workunit.client.1.smithi044.stdout:9/415: chown d2/d6/d42/d8b 329 1 2024-04-22T23:46:12.003 INFO:tasks.workunit.client.1.smithi044.stdout:8/621: fsync db/d14/d24/f26 0 2024-04-22T23:46:12.006 INFO:tasks.workunit.client.1.smithi044.stdout:5/540: chown d15/d16/d1d/d22/d2a/d3a/d43/c44 41 1 2024-04-22T23:46:12.006 INFO:tasks.workunit.client.1.smithi044.stdout:5/541: chown d15/d16/f9a 356 1 2024-04-22T23:46:12.007 INFO:tasks.workunit.client.1.smithi044.stdout:1/579: rename d1/d8/d1c/d7b/d80/d4b/c71 to d1/d8/d34/da1/dab/cce 0 2024-04-22T23:46:12.012 INFO:tasks.workunit.client.1.smithi044.stdout:3/567: rmdir d9/db/d15/d2e/d48 39 2024-04-22T23:46:12.014 INFO:tasks.workunit.client.1.smithi044.stdout:1/580: dwrite d1/d8/d20/d47/d92/faa [0,4194304] 0 2024-04-22T23:46:12.016 INFO:tasks.workunit.client.1.smithi044.stdout:8/622: mknod db/d14/d84/dc0/ce5 0 2024-04-22T23:46:12.017 INFO:tasks.workunit.client.1.smithi044.stdout:9/416: mknod d2/dc/d1d/d34/d38/d5c/d69/d7a/c8c 0 2024-04-22T23:46:12.020 INFO:tasks.workunit.client.1.smithi044.stdout:7/545: symlink dd/d79/d8c/dc0/lc3 0 2024-04-22T23:46:12.023 INFO:tasks.workunit.client.1.smithi044.stdout:8/623: dread db/d14/f25 [0,4194304] 0 2024-04-22T23:46:12.025 INFO:tasks.workunit.client.1.smithi044.stdout:8/624: readlink db/d52/la8 0 2024-04-22T23:46:12.025 INFO:tasks.workunit.client.1.smithi044.stdout:1/581: unlink d1/d14/c17 0 2024-04-22T23:46:12.029 INFO:tasks.workunit.client.1.smithi044.stdout:1/582: write d1/d8/d20/d45/f2a [1740881,32381] 0 2024-04-22T23:46:12.029 INFO:tasks.workunit.client.1.smithi044.stdout:9/417: symlink d2/d6/l8d 0 2024-04-22T23:46:12.031 INFO:tasks.workunit.client.1.smithi044.stdout:3/568: dwrite d9/d14/d7d/fb4 [0,4194304] 0 2024-04-22T23:46:12.032 INFO:tasks.workunit.client.1.smithi044.stdout:9/418: fsync d2/dc/d1d/d3f/f4c 0 2024-04-22T23:46:12.035 INFO:tasks.workunit.client.1.smithi044.stdout:7/546: dread fa [0,4194304] 0 2024-04-22T23:46:12.038 INFO:tasks.workunit.client.1.smithi044.stdout:9/419: read d2/f19 [2651726,853] 0 2024-04-22T23:46:12.039 INFO:tasks.workunit.client.1.smithi044.stdout:8/625: unlink db/d68/fc3 0 2024-04-22T23:46:12.039 INFO:tasks.workunit.client.1.smithi044.stdout:9/420: read d2/dc/d1d/d3f/f48 [3003010,48819] 0 2024-04-22T23:46:12.041 INFO:tasks.workunit.client.1.smithi044.stdout:5/542: dread f14 [0,4194304] 0 2024-04-22T23:46:12.043 INFO:tasks.workunit.client.1.smithi044.stdout:1/583: creat d1/d8/d1c/d23/d3e/fcf x:0 0 0 2024-04-22T23:46:12.051 INFO:tasks.workunit.client.1.smithi044.stdout:9/421: fsync d2/dc/f32 0 2024-04-22T23:46:12.057 INFO:tasks.workunit.client.1.smithi044.stdout:5/543: mkdir d15/d16/d1d/daf 0 2024-04-22T23:46:12.059 INFO:tasks.workunit.client.1.smithi044.stdout:9/422: rmdir d2/dc 39 2024-04-22T23:46:12.061 INFO:tasks.workunit.client.1.smithi044.stdout:3/569: creat d9/fb9 x:0 0 0 2024-04-22T23:46:12.062 INFO:tasks.workunit.client.1.smithi044.stdout:7/547: rmdir dd 39 2024-04-22T23:46:12.065 INFO:tasks.workunit.client.1.smithi044.stdout:3/570: dread d9/f9b [0,4194304] 0 2024-04-22T23:46:12.066 INFO:tasks.workunit.client.1.smithi044.stdout:5/544: mkdir d15/d3c/d65/db0 0 2024-04-22T23:46:12.067 INFO:tasks.workunit.client.1.smithi044.stdout:5/545: write d15/d16/d1d/dab/d38/fa5 [392224,105094] 0 2024-04-22T23:46:12.071 INFO:tasks.workunit.client.1.smithi044.stdout:9/423: dread - d2/dc/d1d/d34/d38/d5c/f76 zero size 2024-04-22T23:46:12.075 INFO:tasks.workunit.client.1.smithi044.stdout:9/424: dread d2/d6/fa [0,4194304] 0 2024-04-22T23:46:12.075 INFO:tasks.workunit.client.1.smithi044.stdout:9/425: chown d2/dc/d51 242271 1 2024-04-22T23:46:12.080 INFO:tasks.workunit.client.1.smithi044.stdout:8/626: write db/d14/d24/f35 [204023,79203] 0 2024-04-22T23:46:12.082 INFO:tasks.workunit.client.1.smithi044.stdout:3/571: dread d9/db/d15/f63 [0,4194304] 0 2024-04-22T23:46:12.082 INFO:tasks.workunit.client.1.smithi044.stdout:8/627: truncate db/d14/d24/fd7 732224 0 2024-04-22T23:46:12.088 INFO:tasks.workunit.client.1.smithi044.stdout:3/572: dwrite d9/db/d15/f18 [4194304,4194304] 0 2024-04-22T23:46:12.099 INFO:tasks.workunit.client.1.smithi044.stdout:7/548: symlink dd/d1a/lc4 0 2024-04-22T23:46:12.100 INFO:tasks.workunit.client.1.smithi044.stdout:6/435: sync 2024-04-22T23:46:12.105 INFO:tasks.workunit.client.1.smithi044.stdout:8/628: mknod db/d52/ce6 0 2024-04-22T23:46:12.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/426: mkdir d2/dc/d1d/d34/d8e 0 2024-04-22T23:46:12.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/427: chown d2/d6/d4d/f5e 2046790156 1 2024-04-22T23:46:12.112 INFO:tasks.workunit.client.1.smithi044.stdout:9/428: truncate d2/f7f 198481 0 2024-04-22T23:46:12.114 INFO:tasks.workunit.client.1.smithi044.stdout:6/436: creat d5/d3f/d4c/d4f/f8f x:0 0 0 2024-04-22T23:46:12.116 INFO:tasks.workunit.client.1.smithi044.stdout:8/629: creat db/d14/dc7/fe7 x:0 0 0 2024-04-22T23:46:12.117 INFO:tasks.workunit.client.1.smithi044.stdout:8/630: stat db/f1c 0 2024-04-22T23:46:12.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/546: creat d15/d5d/fb1 x:0 0 0 2024-04-22T23:46:12.119 INFO:tasks.workunit.client.1.smithi044.stdout:6/437: dread d5/dd/d28/f34 [4194304,4194304] 0 2024-04-22T23:46:12.120 INFO:tasks.workunit.client.1.smithi044.stdout:6/438: write d5/d30/f39 [2313408,10211] 0 2024-04-22T23:46:12.122 INFO:tasks.workunit.client.1.smithi044.stdout:9/429: fdatasync d2/d6/fa 0 2024-04-22T23:46:12.125 INFO:tasks.workunit.client.1.smithi044.stdout:5/547: stat d15/d16/d1d/d22/l2b 0 2024-04-22T23:46:12.125 INFO:tasks.workunit.client.1.smithi044.stdout:5/548: chown d15/d16/d1d/dab/d38/fa5 470122447 1 2024-04-22T23:46:12.127 INFO:tasks.workunit.client.1.smithi044.stdout:5/549: write d15/d16/d1d/d22/d2a/d3a/d43/fa8 [415615,69262] 0 2024-04-22T23:46:12.131 INFO:tasks.workunit.client.1.smithi044.stdout:7/549: link dd/d1f/d24/l6c dd/d98/lc5 0 2024-04-22T23:46:12.135 INFO:tasks.workunit.client.1.smithi044.stdout:9/430: write d2/dc/d1d/d34/d38/d5c/f76 [466984,58211] 0 2024-04-22T23:46:12.136 INFO:tasks.workunit.client.1.smithi044.stdout:3/573: rename d9/db/d15/d25/d67/l87 to d9/db/d15/d25/d5b/lba 0 2024-04-22T23:46:12.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/550: unlink d15/d3c/f77 0 2024-04-22T23:46:12.140 INFO:tasks.workunit.client.1.smithi044.stdout:6/439: write d5/dd/d67/f47 [811157,69763] 0 2024-04-22T23:46:12.142 INFO:tasks.workunit.client.1.smithi044.stdout:7/550: creat dd/d1a/d3c/fc6 x:0 0 0 2024-04-22T23:46:12.143 INFO:tasks.workunit.client.1.smithi044.stdout:7/551: fsync dd/d55/d5e/d60/d68/d94/fbe 0 2024-04-22T23:46:12.144 INFO:tasks.workunit.client.1.smithi044.stdout:9/431: creat d2/dc/d1d/d34/d38/d4a/d72/f8f x:0 0 0 2024-04-22T23:46:12.144 INFO:tasks.workunit.client.1.smithi044.stdout:7/552: readlink dd/d55/d5e/d60/d68/l82 0 2024-04-22T23:46:12.149 INFO:tasks.workunit.client.1.smithi044.stdout:8/631: rename db/d14/d24/fa2 to db/d52/d9c/dad/db8/fe8 0 2024-04-22T23:46:12.149 INFO:tasks.workunit.client.1.smithi044.stdout:8/632: chown db/f3c 29428933 1 2024-04-22T23:46:12.152 INFO:tasks.workunit.client.1.smithi044.stdout:7/553: mkdir dd/d79/dc7 0 2024-04-22T23:46:12.157 INFO:tasks.workunit.client.1.smithi044.stdout:5/551: symlink d15/d16/d1d/d22/d4f/d99/lb2 0 2024-04-22T23:46:12.163 INFO:tasks.workunit.client.1.smithi044.stdout:5/552: rmdir d15/d16/d1d/d22 39 2024-04-22T23:46:12.166 INFO:tasks.workunit.client.1.smithi044.stdout:3/574: getdents d9/d14/d7d 0 2024-04-22T23:46:12.170 INFO:tasks.workunit.client.1.smithi044.stdout:8/633: rename db/d68/d8b/fd2 to db/d14/dc7/dc9/fe9 0 2024-04-22T23:46:12.170 INFO:tasks.workunit.client.1.smithi044.stdout:8/634: stat db/d14/c53 0 2024-04-22T23:46:12.172 INFO:tasks.workunit.client.1.smithi044.stdout:3/575: fdatasync d9/f2a 0 2024-04-22T23:46:12.173 INFO:tasks.workunit.client.1.smithi044.stdout:9/432: write d2/f3 [4992873,83859] 0 2024-04-22T23:46:12.176 INFO:tasks.workunit.client.1.smithi044.stdout:9/433: dread d2/f4 [0,4194304] 0 2024-04-22T23:46:12.177 INFO:tasks.workunit.client.1.smithi044.stdout:7/554: rename dd/d1a/d3c to dd/d1a/db7/dc8 0 2024-04-22T23:46:12.178 INFO:tasks.workunit.client.1.smithi044.stdout:3/576: dread d9/d12/f92 [0,4194304] 0 2024-04-22T23:46:12.178 INFO:tasks.workunit.client.1.smithi044.stdout:5/553: mknod d15/d16/d1d/d22/d2a/d3a/cb3 0 2024-04-22T23:46:12.180 INFO:tasks.workunit.client.1.smithi044.stdout:5/554: write d15/d16/d1d/d22/d2a/d3a/f96 [913235,22076] 0 2024-04-22T23:46:12.181 INFO:tasks.workunit.client.1.smithi044.stdout:8/635: mknod db/d68/d8b/dca/dd6/cea 0 2024-04-22T23:46:12.182 INFO:tasks.workunit.client.1.smithi044.stdout:5/555: fdatasync d15/d16/d1d/d22/d2a/d3a/d43/d80/f8c 0 2024-04-22T23:46:12.189 INFO:tasks.workunit.client.1.smithi044.stdout:9/434: creat d2/dc/d1d/d34/d38/d5c/d69/d75/d78/f90 x:0 0 0 2024-04-22T23:46:12.189 INFO:tasks.workunit.client.1.smithi044.stdout:7/555: mkdir dd/d55/d5e/d60/d68/d94/dbd/dc9 0 2024-04-22T23:46:12.191 INFO:tasks.workunit.client.1.smithi044.stdout:5/556: symlink d15/d5d/lb4 0 2024-04-22T23:46:12.195 INFO:tasks.workunit.client.1.smithi044.stdout:9/435: symlink d2/dc/d1d/d34/d38/d4a/d72/l91 0 2024-04-22T23:46:12.196 INFO:tasks.workunit.client.1.smithi044.stdout:7/556: dwrite dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:12.203 INFO:tasks.workunit.client.1.smithi044.stdout:9/436: chown d2/dc/d1d/c2e 98 1 2024-04-22T23:46:12.204 INFO:tasks.workunit.client.1.smithi044.stdout:9/437: dread - d2/dc/d1d/d34/d38/d5c/f83 zero size 2024-04-22T23:46:12.207 INFO:tasks.workunit.client.1.smithi044.stdout:7/557: rename dd/d79/f80 to dd/d98/fca 0 2024-04-22T23:46:12.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/557: dread d15/d16/d1d/d22/d2a/d3a/f7a [0,4194304] 0 2024-04-22T23:46:12.211 INFO:tasks.workunit.client.1.smithi044.stdout:3/577: getdents d9/db/d15/d2e/d48/d8f 0 2024-04-22T23:46:12.211 INFO:tasks.workunit.client.1.smithi044.stdout:9/438: dread d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f [0,4194304] 0 2024-04-22T23:46:12.211 INFO:tasks.workunit.client.1.smithi044.stdout:8/636: truncate db/d14/d24/f35 128334 0 2024-04-22T23:46:12.213 INFO:tasks.workunit.client.1.smithi044.stdout:9/439: read - d2/dc/d1d/d34/d38/d5c/f65 zero size 2024-04-22T23:46:12.214 INFO:tasks.workunit.client.1.smithi044.stdout:7/558: mkdir dd/d55/dcb 0 2024-04-22T23:46:12.219 INFO:tasks.workunit.client.1.smithi044.stdout:3/578: mkdir d9/db/d15/d25/d5b/d52/dbb 0 2024-04-22T23:46:12.221 INFO:tasks.workunit.client.1.smithi044.stdout:7/559: creat dd/d1f/d24/fcc x:0 0 0 2024-04-22T23:46:12.222 INFO:tasks.workunit.client.1.smithi044.stdout:7/560: fdatasync dd/d55/d5e/d60/d75/f92 0 2024-04-22T23:46:12.224 INFO:tasks.workunit.client.1.smithi044.stdout:8/637: stat db/d14/d72/f95 0 2024-04-22T23:46:12.227 INFO:tasks.workunit.client.1.smithi044.stdout:3/579: creat d9/db/d15/d54/d3c/fbc x:0 0 0 2024-04-22T23:46:12.229 INFO:tasks.workunit.client.1.smithi044.stdout:8/638: mkdir db/d14/dc7/dc9/dd1/deb 0 2024-04-22T23:46:12.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/558: dwrite d15/d16/d1d/d22/d2a/d3a/f4b [0,4194304] 0 2024-04-22T23:46:12.239 INFO:tasks.workunit.client.1.smithi044.stdout:8/639: dwrite db/f3c [0,4194304] 0 2024-04-22T23:46:12.240 INFO:tasks.workunit.client.1.smithi044.stdout:9/440: getdents d2/d6 0 2024-04-22T23:46:12.240 INFO:tasks.workunit.client.1.smithi044.stdout:8/640: truncate db/f3f 971106 0 2024-04-22T23:46:12.244 INFO:tasks.workunit.client.1.smithi044.stdout:3/580: getdents d9/db/d15/d2e/d48/d8f 0 2024-04-22T23:46:12.245 INFO:tasks.workunit.client.1.smithi044.stdout:7/561: write dd/d1f/d24/f19 [3401058,87913] 0 2024-04-22T23:46:12.248 INFO:tasks.workunit.client.1.smithi044.stdout:3/581: write d9/db/f1a [2359723,79626] 0 2024-04-22T23:46:12.252 INFO:tasks.workunit.client.1.smithi044.stdout:5/559: unlink d15/d16/d1d/c33 0 2024-04-22T23:46:12.254 INFO:tasks.workunit.client.1.smithi044.stdout:8/641: dwrite db/d14/d84/dbb/fbd [0,4194304] 0 2024-04-22T23:46:12.255 INFO:tasks.workunit.client.1.smithi044.stdout:9/441: dwrite d2/dc/d1d/d34/d38/d5c/f76 [0,4194304] 0 2024-04-22T23:46:12.256 INFO:tasks.workunit.client.1.smithi044.stdout:5/560: chown d15/d5d/d64/d6f/f74 595358582 1 2024-04-22T23:46:12.259 INFO:tasks.workunit.client.1.smithi044.stdout:5/561: write d15/d16/d1d/dab/f54 [7581012,18798] 0 2024-04-22T23:46:12.260 INFO:tasks.workunit.client.1.smithi044.stdout:7/562: dwrite dd/d1a/db7/dc8/fa1 [0,4194304] 0 2024-04-22T23:46:12.263 INFO:tasks.workunit.client.1.smithi044.stdout:8/642: mknod db/d68/daf/cec 0 2024-04-22T23:46:12.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/582: creat d9/d12/db1/fbd x:0 0 0 2024-04-22T23:46:12.266 INFO:tasks.workunit.client.1.smithi044.stdout:7/563: truncate dd/d27/fb6 506152 0 2024-04-22T23:46:12.269 INFO:tasks.workunit.client.1.smithi044.stdout:5/562: truncate d15/d16/d1d/d22/d2a/d3a/f96 1529013 0 2024-04-22T23:46:12.270 INFO:tasks.workunit.client.1.smithi044.stdout:9/442: creat d2/dc/d1d/d34/f92 x:0 0 0 2024-04-22T23:46:12.270 INFO:tasks.workunit.client.1.smithi044.stdout:8/643: mkdir db/d68/d8b/ded 0 2024-04-22T23:46:12.272 INFO:tasks.workunit.client.1.smithi044.stdout:7/564: mknod dd/d1f/d24/ccd 0 2024-04-22T23:46:12.276 INFO:tasks.workunit.client.1.smithi044.stdout:7/565: creat dd/d55/d5e/d60/d7b/fce x:0 0 0 2024-04-22T23:46:12.277 INFO:tasks.workunit.client.1.smithi044.stdout:9/443: mknod d2/dc/d1d/d34/d38/d5c/d7b/c93 0 2024-04-22T23:46:12.279 INFO:tasks.workunit.client.1.smithi044.stdout:8/644: dwrite db/d6c/fcc [0,4194304] 0 2024-04-22T23:46:12.279 INFO:tasks.workunit.client.1.smithi044.stdout:7/566: dread - dd/d1a/db7/dc8/fa8 zero size 2024-04-22T23:46:12.280 INFO:tasks.workunit.client.1.smithi044.stdout:5/563: unlink d15/d3c/d65/d8d/la0 0 2024-04-22T23:46:12.284 INFO:tasks.workunit.client.1.smithi044.stdout:7/567: mkdir dd/d55/dcf 0 2024-04-22T23:46:12.287 INFO:tasks.workunit.client.1.smithi044.stdout:8/645: creat db/d52/d9c/dad/fee x:0 0 0 2024-04-22T23:46:12.289 INFO:tasks.workunit.client.1.smithi044.stdout:7/568: unlink dd/d79/d8c/d9c/lac 0 2024-04-22T23:46:12.292 INFO:tasks.workunit.client.1.smithi044.stdout:5/564: rename d15/d3c/d65/db0 to d15/d5d/db5 0 2024-04-22T23:46:12.297 INFO:tasks.workunit.client.1.smithi044.stdout:7/569: creat dd/d99/dbb/fd0 x:0 0 0 2024-04-22T23:46:12.298 INFO:tasks.workunit.client.1.smithi044.stdout:2/454: sync 2024-04-22T23:46:12.299 INFO:tasks.workunit.client.1.smithi044.stdout:0/593: sync 2024-04-22T23:46:12.300 INFO:tasks.workunit.client.1.smithi044.stdout:7/570: symlink dd/d55/ld1 0 2024-04-22T23:46:12.303 INFO:tasks.workunit.client.1.smithi044.stdout:9/444: dread d2/f7d [0,4194304] 0 2024-04-22T23:46:12.304 INFO:tasks.workunit.client.1.smithi044.stdout:5/565: rename d15/d16/c5c to d15/d16/d1d/d22/d81/cb6 0 2024-04-22T23:46:12.305 INFO:tasks.workunit.client.1.smithi044.stdout:3/583: dread d9/db/d15/d2e/d48/fa1 [4194304,4194304] 0 2024-04-22T23:46:12.306 INFO:tasks.workunit.client.1.smithi044.stdout:9/445: fsync d2/dc/d1d/d3f/f6e 0 2024-04-22T23:46:12.307 INFO:tasks.workunit.client.1.smithi044.stdout:7/571: unlink dd/d1a/db7/dc8/l57 0 2024-04-22T23:46:12.307 INFO:tasks.workunit.client.1.smithi044.stdout:5/566: dread - d15/d16/d1d/d22/f97 zero size 2024-04-22T23:46:12.313 INFO:tasks.workunit.client.1.smithi044.stdout:2/455: rename d8/d19/d36/f3e to d8/d19/d53/f9c 0 2024-04-22T23:46:12.314 INFO:tasks.workunit.client.1.smithi044.stdout:3/584: mkdir d9/db/d33/d5d/d64/dbe 0 2024-04-22T23:46:12.315 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:11 smithi044 ceph-mon[34463]: pgmap v9: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 33 MiB/s rd, 117 MiB/s wr, 273 op/s 2024-04-22T23:46:12.316 INFO:tasks.workunit.client.1.smithi044.stdout:5/567: fsync f12 0 2024-04-22T23:46:12.317 INFO:tasks.workunit.client.1.smithi044.stdout:3/585: write d9/db/d15/d54/f58 [3984435,115750] 0 2024-04-22T23:46:12.318 INFO:tasks.workunit.client.1.smithi044.stdout:0/594: rename d0/d2e/d33/f7d to d0/d2/dd/d3e/d40/d43/d7b/d90/fd8 0 2024-04-22T23:46:12.320 INFO:tasks.workunit.client.1.smithi044.stdout:2/456: truncate d8/ff 2945475 0 2024-04-22T23:46:12.322 INFO:tasks.workunit.client.1.smithi044.stdout:8/646: truncate db/d52/d9c/dad/db8/fe8 604127 0 2024-04-22T23:46:12.329 INFO:tasks.workunit.client.1.smithi044.stdout:0/595: mkdir d0/d2e/d2f/d61/dd9 0 2024-04-22T23:46:12.329 INFO:tasks.workunit.client.1.smithi044.stdout:7/572: dread dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:12.330 INFO:tasks.workunit.client.1.smithi044.stdout:7/573: stat dd/d27/f5b 0 2024-04-22T23:46:12.333 INFO:tasks.workunit.client.1.smithi044.stdout:2/457: mknod d8/d19/d36/d44/c9d 0 2024-04-22T23:46:12.335 INFO:tasks.workunit.client.1.smithi044.stdout:8/647: symlink db/d14/d9e/lef 0 2024-04-22T23:46:12.335 INFO:tasks.workunit.client.1.smithi044.stdout:8/648: chown f6 338844 1 2024-04-22T23:46:12.336 INFO:tasks.workunit.client.1.smithi044.stdout:8/649: stat db/d14/l7c 0 2024-04-22T23:46:12.341 INFO:tasks.workunit.client.1.smithi044.stdout:2/458: dwrite d8/d19/d36/d46/d63/d65/f85 [0,4194304] 0 2024-04-22T23:46:12.341 INFO:tasks.workunit.client.1.smithi044.stdout:3/586: unlink d9/db/d15/d54/la7 0 2024-04-22T23:46:12.343 INFO:tasks.workunit.client.1.smithi044.stdout:2/459: write d8/d19/d96/f73 [1945908,77854] 0 2024-04-22T23:46:12.352 INFO:tasks.workunit.client.1.smithi044.stdout:8/650: unlink db/d14/d84/dc0/ce5 0 2024-04-22T23:46:12.352 INFO:tasks.workunit.client.1.smithi044.stdout:0/596: dread d0/d2/dd/f13 [0,4194304] 0 2024-04-22T23:46:12.353 INFO:tasks.workunit.client.1.smithi044.stdout:8/651: write db/d6c/fa5 [64810,102128] 0 2024-04-22T23:46:12.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/597: readlink d0/d2/d4b/d6b/l9c 0 2024-04-22T23:46:12.356 INFO:tasks.workunit.client.1.smithi044.stdout:0/598: dread - d0/d2/dd/d3e/d40/d43/d7b/d90/fd8 zero size 2024-04-22T23:46:12.360 INFO:tasks.workunit.client.1.smithi044.stdout:2/460: truncate d8/d19/f29 309514 0 2024-04-22T23:46:12.362 INFO:tasks.workunit.client.1.smithi044.stdout:7/574: link dd/d1a/db7/dc8/f8f dd/d79/d8c/db8/fd2 0 2024-04-22T23:46:12.366 INFO:tasks.workunit.client.1.smithi044.stdout:5/568: getdents d15/d16 0 2024-04-22T23:46:12.371 INFO:tasks.workunit.client.1.smithi044.stdout:7/575: dwrite dd/d55/d5e/d60/d68/d94/fbe [0,4194304] 0 2024-04-22T23:46:12.372 INFO:tasks.workunit.client.1.smithi044.stdout:7/576: dread - dd/d1a/f7d zero size 2024-04-22T23:46:12.381 INFO:tasks.workunit.client.1.smithi044.stdout:7/577: dwrite dd/d1a/f7e [0,4194304] 0 2024-04-22T23:46:12.382 INFO:tasks.workunit.client.1.smithi044.stdout:0/599: write d0/d2/dd/d3e/d40/d43/d79/dbb/fd1 [4807643,53822] 0 2024-04-22T23:46:12.387 INFO:tasks.workunit.client.1.smithi044.stdout:8/652: truncate db/d14/f66 3630171 0 2024-04-22T23:46:12.389 INFO:tasks.workunit.client.1.smithi044.stdout:5/569: write d15/d16/d1d/d22/d2a/d3a/d43/f61 [173066,54712] 0 2024-04-22T23:46:12.391 INFO:tasks.workunit.client.1.smithi044.stdout:7/578: mknod dd/d55/d5e/d89/cd3 0 2024-04-22T23:46:12.391 INFO:tasks.workunit.client.1.smithi044.stdout:5/570: fsync d15/d3c/d65/d8d/f8f 0 2024-04-22T23:46:12.392 INFO:tasks.workunit.client.1.smithi044.stdout:0/600: symlink d0/d2/dd/d3e/d40/d67/lda 0 2024-04-22T23:46:12.393 INFO:tasks.workunit.client.1.smithi044.stdout:0/601: chown d0/d2/dd/f13 187210 1 2024-04-22T23:46:12.395 INFO:tasks.workunit.client.1.smithi044.stdout:0/602: truncate d0/d2/dd/d3e/f3f 1310280 0 2024-04-22T23:46:12.395 INFO:tasks.workunit.client.1.smithi044.stdout:0/603: chown d0/d4c/f8f 734 1 2024-04-22T23:46:12.396 INFO:tasks.workunit.client.1.smithi044.stdout:5/571: symlink d15/d16/d1d/dab/lb7 0 2024-04-22T23:46:12.397 INFO:tasks.workunit.client.1.smithi044.stdout:5/572: readlink d15/d16/d1d/l5b 0 2024-04-22T23:46:12.398 INFO:tasks.workunit.client.1.smithi044.stdout:7/579: symlink dd/d55/d5e/d60/d68/d94/dbd/ld4 0 2024-04-22T23:46:12.398 INFO:tasks.workunit.client.1.smithi044.stdout:5/573: truncate d15/d16/f7b 407579 0 2024-04-22T23:46:12.399 INFO:tasks.workunit.client.1.smithi044.stdout:2/461: getdents d8/d19/d36/d46 0 2024-04-22T23:46:12.400 INFO:tasks.workunit.client.1.smithi044.stdout:5/574: write d15/f94 [337311,34854] 0 2024-04-22T23:46:12.401 INFO:tasks.workunit.client.1.smithi044.stdout:5/575: write d15/d16/f9a [156143,108248] 0 2024-04-22T23:46:12.403 INFO:tasks.workunit.client.1.smithi044.stdout:8/653: symlink db/d14/lf0 0 2024-04-22T23:46:12.404 INFO:tasks.workunit.client.1.smithi044.stdout:0/604: mknod d0/d2/dd/d3e/d40/d67/cdb 0 2024-04-22T23:46:12.406 INFO:tasks.workunit.client.1.smithi044.stdout:0/605: write d0/d2/fcb [1414904,7520] 0 2024-04-22T23:46:12.412 INFO:tasks.workunit.client.1.smithi044.stdout:2/462: mkdir d8/d19/d9e 0 2024-04-22T23:46:12.412 INFO:tasks.workunit.client.1.smithi044.stdout:0/606: creat d0/d2/dd/d3e/d40/d43/d7b/d90/fdc x:0 0 0 2024-04-22T23:46:12.414 INFO:tasks.workunit.client.1.smithi044.stdout:5/576: mknod d15/d3c/cb8 0 2024-04-22T23:46:12.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/577: dread d15/d16/d1d/f42 [0,4194304] 0 2024-04-22T23:46:12.421 INFO:tasks.workunit.client.1.smithi044.stdout:0/607: dwrite d0/d2/dd/d3e/d40/d43/f5f [0,4194304] 0 2024-04-22T23:46:12.426 INFO:tasks.workunit.client.1.smithi044.stdout:7/580: getdents dd/d98 0 2024-04-22T23:46:12.429 INFO:tasks.workunit.client.1.smithi044.stdout:8/654: getdents db/d68/d8b/da7 0 2024-04-22T23:46:12.430 INFO:tasks.workunit.client.1.smithi044.stdout:5/578: creat d15/d16/d1d/d22/d4f/d99/fb9 x:0 0 0 2024-04-22T23:46:12.433 INFO:tasks.workunit.client.1.smithi044.stdout:5/579: rename d15/d16/d1d/d22/d2a/d3a to d15/d16/d1d/d22/d2a/d3a/d43/dba 22 2024-04-22T23:46:12.436 INFO:tasks.workunit.client.1.smithi044.stdout:2/463: dwrite d8/d19/d5d/f5e [0,4194304] 0 2024-04-22T23:46:12.440 INFO:tasks.workunit.client.1.smithi044.stdout:0/608: link d0/d2/f64 d0/d2e/dbc/fdd 0 2024-04-22T23:46:12.441 INFO:tasks.workunit.client.1.smithi044.stdout:0/609: write d0/d4c/d59/fb3 [1065328,5410] 0 2024-04-22T23:46:12.442 INFO:tasks.workunit.client.1.smithi044.stdout:8/655: read db/d14/d24/f4e [2850507,115967] 0 2024-04-22T23:46:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:11 smithi031 ceph-mon[26721]: pgmap v9: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 33 MiB/s rd, 117 MiB/s wr, 273 op/s 2024-04-22T23:46:12.444 INFO:tasks.workunit.client.1.smithi044.stdout:2/464: fsync d8/d19/f47 0 2024-04-22T23:46:12.449 INFO:tasks.workunit.client.1.smithi044.stdout:8/656: rename db/d14/f97 to db/d14/d24/ff1 0 2024-04-22T23:46:12.452 INFO:tasks.workunit.client.1.smithi044.stdout:2/465: dwrite d8/d19/d36/d44/f52 [0,4194304] 0 2024-04-22T23:46:12.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/657: fsync db/d14/f15 0 2024-04-22T23:46:12.461 INFO:tasks.workunit.client.1.smithi044.stdout:5/580: getdents d15/d16/d1d/d22/d81 0 2024-04-22T23:46:12.461 INFO:tasks.workunit.client.1.smithi044.stdout:5/581: fdatasync fe 0 2024-04-22T23:46:12.468 INFO:tasks.workunit.client.1.smithi044.stdout:5/582: creat d15/d5d/d64/d6f/d7f/fbb x:0 0 0 2024-04-22T23:46:12.469 INFO:tasks.workunit.client.1.smithi044.stdout:5/583: dread - d15/d5d/d82/f83 zero size 2024-04-22T23:46:12.471 INFO:tasks.workunit.client.1.smithi044.stdout:0/610: rename d0/d2e/d2f/d61 to d0/d2/dd/d3e/d40/d43/dde 0 2024-04-22T23:46:12.472 INFO:tasks.workunit.client.1.smithi044.stdout:0/611: stat d0/d4c/dbd 0 2024-04-22T23:46:12.473 INFO:tasks.workunit.client.1.smithi044.stdout:8/658: write db/d14/d24/f4e [4203177,55486] 0 2024-04-22T23:46:12.483 INFO:tasks.workunit.client.1.smithi044.stdout:6/440: sync 2024-04-22T23:46:12.483 INFO:tasks.workunit.client.1.smithi044.stdout:1/584: sync 2024-04-22T23:46:12.483 INFO:tasks.workunit.client.1.smithi044.stdout:4/596: sync 2024-04-22T23:46:12.484 INFO:tasks.workunit.client.1.smithi044.stdout:5/584: mkdir d15/d16/dbc 0 2024-04-22T23:46:12.485 INFO:tasks.workunit.client.1.smithi044.stdout:2/466: rename f3 to d8/d19/d36/f9f 0 2024-04-22T23:46:12.486 INFO:tasks.workunit.client.1.smithi044.stdout:4/597: chown d5/d31/d44/d20 1295 1 2024-04-22T23:46:12.491 INFO:tasks.workunit.client.1.smithi044.stdout:8/659: dread db/d14/f66 [0,4194304] 0 2024-04-22T23:46:12.492 INFO:tasks.workunit.client.1.smithi044.stdout:6/441: unlink d5/d3f/f89 0 2024-04-22T23:46:12.500 INFO:tasks.workunit.client.1.smithi044.stdout:4/598: mkdir d5/db/d16/d4d/dad/dc5 0 2024-04-22T23:46:12.505 INFO:tasks.workunit.client.1.smithi044.stdout:5/585: stat d15/d16/d1d/d22/d81/cac 0 2024-04-22T23:46:12.506 INFO:tasks.workunit.client.1.smithi044.stdout:5/586: dread - d15/d16/d1d/dab/f8e zero size 2024-04-22T23:46:12.507 INFO:tasks.workunit.client.1.smithi044.stdout:5/587: read - d15/d5d/fb1 zero size 2024-04-22T23:46:12.508 INFO:tasks.workunit.client.1.smithi044.stdout:5/588: write d15/d16/d1d/f49 [1866802,97025] 0 2024-04-22T23:46:12.509 INFO:tasks.workunit.client.1.smithi044.stdout:4/599: fdatasync d5/d31/d24/d38/f63 0 2024-04-22T23:46:12.509 INFO:tasks.workunit.client.1.smithi044.stdout:5/589: chown cd 8 1 2024-04-22T23:46:12.511 INFO:tasks.workunit.client.1.smithi044.stdout:4/600: truncate d5/d31/d44/f7d 128927 0 2024-04-22T23:46:12.513 INFO:tasks.workunit.client.1.smithi044.stdout:0/612: rename d0/d2/dd/d3e/d40/d43/d7b/d90 to d0/d2e/ddf 0 2024-04-22T23:46:12.516 INFO:tasks.workunit.client.1.smithi044.stdout:1/585: creat d1/d8/d1c/d7b/d80/fd0 x:0 0 0 2024-04-22T23:46:12.516 INFO:tasks.workunit.client.1.smithi044.stdout:8/660: write db/d14/d24/f58 [972205,117963] 0 2024-04-22T23:46:12.519 INFO:tasks.workunit.client.1.smithi044.stdout:6/442: creat d5/f90 x:0 0 0 2024-04-22T23:46:12.521 INFO:tasks.workunit.client.1.smithi044.stdout:0/613: dwrite d0/d2/fcb [0,4194304] 0 2024-04-22T23:46:12.522 INFO:tasks.workunit.client.1.smithi044.stdout:6/443: dread - d5/d3f/d4c/d23/d3d/d64/f77 zero size 2024-04-22T23:46:12.525 INFO:tasks.workunit.client.1.smithi044.stdout:4/601: unlink d5/db/c6d 0 2024-04-22T23:46:12.528 INFO:tasks.workunit.client.1.smithi044.stdout:2/467: rename d8/d19/d96/l90 to d8/d19/d9e/la0 0 2024-04-22T23:46:12.530 INFO:tasks.workunit.client.1.smithi044.stdout:5/590: dread d15/d16/d1d/d22/d2a/d3a/d43/f4c [4194304,4194304] 0 2024-04-22T23:46:12.532 INFO:tasks.workunit.client.1.smithi044.stdout:2/468: read d8/d27/f2a [753589,114738] 0 2024-04-22T23:46:12.533 INFO:tasks.workunit.client.1.smithi044.stdout:0/614: mknod d0/d4c/d8b/ce0 0 2024-04-22T23:46:12.534 INFO:tasks.workunit.client.1.smithi044.stdout:6/444: unlink d5/dd/d67/c43 0 2024-04-22T23:46:12.536 INFO:tasks.workunit.client.1.smithi044.stdout:6/445: chown d5/dd/f6a 14377 1 2024-04-22T23:46:12.537 INFO:tasks.workunit.client.1.smithi044.stdout:4/602: unlink d5/d31/d44/d20/d22/d8d/l99 0 2024-04-22T23:46:12.538 INFO:tasks.workunit.client.1.smithi044.stdout:5/591: dwrite d15/d16/d1d/dab/d38/f50 [0,4194304] 0 2024-04-22T23:46:12.541 INFO:tasks.workunit.client.1.smithi044.stdout:5/592: chown d15/d16/d1d/d22/d2a/d3a/d43/c73 2979288 1 2024-04-22T23:46:12.541 INFO:tasks.workunit.client.1.smithi044.stdout:5/593: dread - d15/d5d/d82/f83 zero size 2024-04-22T23:46:12.546 INFO:tasks.workunit.client.1.smithi044.stdout:0/615: creat d0/d2e/d33/fe1 x:0 0 0 2024-04-22T23:46:12.547 INFO:tasks.workunit.client.1.smithi044.stdout:0/616: fsync d0/d2/f8e 0 2024-04-22T23:46:12.550 INFO:tasks.workunit.client.1.smithi044.stdout:4/603: fsync d5/d31/d44/d20/d22/f35 0 2024-04-22T23:46:12.560 INFO:tasks.workunit.client.1.smithi044.stdout:0/617: creat d0/d60/fe2 x:0 0 0 2024-04-22T23:46:12.563 INFO:tasks.workunit.client.1.smithi044.stdout:6/446: symlink d5/d3f/d4c/l91 0 2024-04-22T23:46:12.563 INFO:tasks.workunit.client.1.smithi044.stdout:6/447: readlink d5/l7 0 2024-04-22T23:46:12.566 INFO:tasks.workunit.client.1.smithi044.stdout:4/604: rename d5/db/d16/c2b to d5/db/d16/d4d/dad/dc5/cc6 0 2024-04-22T23:46:12.567 INFO:tasks.workunit.client.1.smithi044.stdout:1/586: link d1/d8/d20/d47/d92/fba d1/d8/d1c/d7b/fd1 0 2024-04-22T23:46:12.569 INFO:tasks.workunit.client.1.smithi044.stdout:8/661: dwrite db/d52/d9c/dad/db8/fe8 [0,4194304] 0 2024-04-22T23:46:12.569 INFO:tasks.workunit.client.1.smithi044.stdout:5/594: truncate d15/d3c/f53 3443124 0 2024-04-22T23:46:12.572 INFO:tasks.workunit.client.1.smithi044.stdout:0/618: creat d0/d2e/d2f/fe3 x:0 0 0 2024-04-22T23:46:12.574 INFO:tasks.workunit.client.1.smithi044.stdout:4/605: dread d5/db/d16/d4d/f8a [0,4194304] 0 2024-04-22T23:46:12.579 INFO:tasks.workunit.client.1.smithi044.stdout:1/587: creat d1/d14/fd2 x:0 0 0 2024-04-22T23:46:12.589 INFO:tasks.workunit.client.1.smithi044.stdout:1/588: dread - d1/d8/d1c/d23/f7c zero size 2024-04-22T23:46:12.592 INFO:tasks.workunit.client.1.smithi044.stdout:1/589: write d1/d8/d20/f2f [3179859,57162] 0 2024-04-22T23:46:12.593 INFO:tasks.workunit.client.1.smithi044.stdout:6/448: dwrite d5/d3f/d4c/f3a [0,4194304] 0 2024-04-22T23:46:12.595 INFO:tasks.workunit.client.1.smithi044.stdout:6/449: write d5/dd/f1b [2757540,1160] 0 2024-04-22T23:46:12.598 INFO:tasks.workunit.client.1.smithi044.stdout:4/606: dread d5/d31/d24/d69/d90/f88 [0,4194304] 0 2024-04-22T23:46:12.599 INFO:tasks.workunit.client.1.smithi044.stdout:0/619: mkdir d0/d2/dd/d3e/d40/d43/de4 0 2024-04-22T23:46:12.601 INFO:tasks.workunit.client.1.smithi044.stdout:1/590: mkdir d1/d8/d1c/d23/d48/d72/dd3 0 2024-04-22T23:46:12.601 INFO:tasks.workunit.client.1.smithi044.stdout:5/595: dread d15/d16/d1d/d22/d4f/d68/f6c [0,4194304] 0 2024-04-22T23:46:12.602 INFO:tasks.workunit.client.1.smithi044.stdout:1/591: read - d1/d8/d20/d45/d5d/fa3 zero size 2024-04-22T23:46:12.602 INFO:tasks.workunit.client.1.smithi044.stdout:8/662: truncate db/f44 1706007 0 2024-04-22T23:46:12.604 INFO:tasks.workunit.client.1.smithi044.stdout:5/596: truncate d15/d5d/d64/d6f/d7f/f91 566213 0 2024-04-22T23:46:12.605 INFO:tasks.workunit.client.1.smithi044.stdout:1/592: chown d1/d8/d20/d45/d75/d77/dad/fb1 94052769 1 2024-04-22T23:46:12.613 INFO:tasks.workunit.client.1.smithi044.stdout:4/607: rename d5/d31/d44/c2d to d5/d31/d44/cc7 0 2024-04-22T23:46:12.614 INFO:tasks.workunit.client.1.smithi044.stdout:4/608: chown d5/db/d16/lc0 2531846 1 2024-04-22T23:46:12.616 INFO:tasks.workunit.client.1.smithi044.stdout:8/663: rename db/d14/dc7/dc9/dd1 to db/ddd/df2 0 2024-04-22T23:46:12.616 INFO:tasks.workunit.client.1.smithi044.stdout:4/609: read d5/d31/f32 [8858,25439] 0 2024-04-22T23:46:12.619 INFO:tasks.workunit.client.1.smithi044.stdout:4/610: read d5/d31/d44/f29 [1566280,130246] 0 2024-04-22T23:46:12.619 INFO:tasks.workunit.client.1.smithi044.stdout:8/664: write db/d52/d9c/dad/fee [543143,75106] 0 2024-04-22T23:46:12.620 INFO:tasks.workunit.client.1.smithi044.stdout:5/597: write d15/d3c/d65/d8d/f8f [4136899,73880] 0 2024-04-22T23:46:12.620 INFO:tasks.workunit.client.1.smithi044.stdout:4/611: readlink d5/l5c 0 2024-04-22T23:46:12.622 INFO:tasks.workunit.client.1.smithi044.stdout:4/612: readlink d5/d31/d24/d38/d6a/l7a 0 2024-04-22T23:46:12.624 INFO:tasks.workunit.client.1.smithi044.stdout:1/593: rename d1/d8/d1c/d7b/d80/fd0 to d1/d8/d1c/d23/d48/d72/fd4 0 2024-04-22T23:46:12.625 INFO:tasks.workunit.client.1.smithi044.stdout:0/620: write d0/d2/dd/d3e/d40/d43/d7b/f26 [3028408,59340] 0 2024-04-22T23:46:12.628 INFO:tasks.workunit.client.1.smithi044.stdout:6/450: dread d5/dd/d28/f34 [0,4194304] 0 2024-04-22T23:46:12.629 INFO:tasks.workunit.client.1.smithi044.stdout:8/665: write db/d14/d72/f9f [1321452,99889] 0 2024-04-22T23:46:12.631 INFO:tasks.workunit.client.1.smithi044.stdout:0/621: dread d0/d2/dd/d3e/f3f [0,4194304] 0 2024-04-22T23:46:12.635 INFO:tasks.workunit.client.1.smithi044.stdout:1/594: creat d1/d14/d95/fd5 x:0 0 0 2024-04-22T23:46:12.637 INFO:tasks.workunit.client.1.smithi044.stdout:8/666: dwrite db/d14/d24/f3b [0,4194304] 0 2024-04-22T23:46:12.643 INFO:tasks.workunit.client.1.smithi044.stdout:1/595: dwrite d1/d8/d20/d45/d5d/fa3 [0,4194304] 0 2024-04-22T23:46:12.645 INFO:tasks.workunit.client.1.smithi044.stdout:5/598: write d15/d16/d1d/d22/d4f/d68/f6c [1542514,24694] 0 2024-04-22T23:46:12.655 INFO:tasks.workunit.client.1.smithi044.stdout:4/613: rename d5/d31/d44/d20/l70 to d5/db/lc8 0 2024-04-22T23:46:12.657 INFO:tasks.workunit.client.1.smithi044.stdout:5/599: creat d15/d3c/d65/fbd x:0 0 0 2024-04-22T23:46:12.664 INFO:tasks.workunit.client.1.smithi044.stdout:5/600: creat d15/d5d/d64/d6f/fbe x:0 0 0 2024-04-22T23:46:12.667 INFO:tasks.workunit.client.1.smithi044.stdout:6/451: getdents d5/dd/d28/d75 0 2024-04-22T23:46:12.668 INFO:tasks.workunit.client.1.smithi044.stdout:0/622: link d0/d2e/dbc/fdd d0/d2/dd/d3e/d40/d43/fe5 0 2024-04-22T23:46:12.671 INFO:tasks.workunit.client.1.smithi044.stdout:0/623: read d0/d2e/d2f/d77/f9d [385976,110275] 0 2024-04-22T23:46:12.672 INFO:tasks.workunit.client.1.smithi044.stdout:5/601: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f7e [0,4194304] 0 2024-04-22T23:46:12.674 INFO:tasks.workunit.client.1.smithi044.stdout:8/667: rename db/d6c/d93 to db/d14/dc7/dc9/df3 0 2024-04-22T23:46:12.674 INFO:tasks.workunit.client.1.smithi044.stdout:0/624: read d0/d2/dd/d3e/d40/d43/d7b/f58 [238327,125762] 0 2024-04-22T23:46:12.675 INFO:tasks.workunit.client.1.smithi044.stdout:6/452: fdatasync d5/d3f/d4c/d23/d3d/f51 0 2024-04-22T23:46:12.679 INFO:tasks.workunit.client.1.smithi044.stdout:4/614: link d5/db/f8f d5/db/d16/fc9 0 2024-04-22T23:46:12.681 INFO:tasks.workunit.client.1.smithi044.stdout:0/625: mkdir d0/d60/de6 0 2024-04-22T23:46:12.687 INFO:tasks.workunit.client.1.smithi044.stdout:6/453: dread - d5/d3f/d4c/d40/d53/d56/f6e zero size 2024-04-22T23:46:12.694 INFO:tasks.workunit.client.1.smithi044.stdout:4/615: mknod d5/d31/d44/d20/d22/cca 0 2024-04-22T23:46:12.698 INFO:tasks.workunit.client.1.smithi044.stdout:0/626: dread d0/d2/dd/d3e/f4f [0,4194304] 0 2024-04-22T23:46:12.699 INFO:tasks.workunit.client.1.smithi044.stdout:4/616: dread d5/d31/d24/d69/d90/f88 [0,4194304] 0 2024-04-22T23:46:12.702 INFO:tasks.workunit.client.1.smithi044.stdout:5/602: link d15/d16/d1d/f42 d15/d5d/d64/d6f/fbf 0 2024-04-22T23:46:12.706 INFO:tasks.workunit.client.1.smithi044.stdout:6/454: mkdir d5/d3f/d4c/d92 0 2024-04-22T23:46:12.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/617: dwrite d5/d31/d24/f2f [8388608,4194304] 0 2024-04-22T23:46:12.712 INFO:tasks.workunit.client.1.smithi044.stdout:5/603: rename d15/d16/d1d/dab to d15/d5d/db5/dc0 0 2024-04-22T23:46:12.715 INFO:tasks.workunit.client.1.smithi044.stdout:5/604: dread - d15/d16/d1d/d22/d2a/d3a/d43/f88 zero size 2024-04-22T23:46:12.716 INFO:tasks.workunit.client.1.smithi044.stdout:4/618: dwrite d5/db/d34/fb9 [0,4194304] 0 2024-04-22T23:46:12.722 INFO:tasks.workunit.client.1.smithi044.stdout:0/627: dread d0/d4c/d59/d5a/fd3 [4194304,4194304] 0 2024-04-22T23:46:12.722 INFO:tasks.workunit.client.1.smithi044.stdout:4/619: write d5/d31/d24/d69/fbc [485377,104799] 0 2024-04-22T23:46:12.722 INFO:tasks.workunit.client.1.smithi044.stdout:4/620: dread - d5/d52/fc1 zero size 2024-04-22T23:46:12.722 INFO:tasks.workunit.client.1.smithi044.stdout:0/628: chown d0/d2/dd/d3e/d40/d43/d79/dbb 7705 1 2024-04-22T23:46:12.724 INFO:tasks.workunit.client.1.smithi044.stdout:0/629: write d0/d4c/fb9 [248588,84427] 0 2024-04-22T23:46:12.726 INFO:tasks.workunit.client.1.smithi044.stdout:5/605: dwrite d15/d5d/fb1 [0,4194304] 0 2024-04-22T23:46:12.728 INFO:tasks.workunit.client.1.smithi044.stdout:0/630: chown d0/d2/dd/c2c 2 1 2024-04-22T23:46:12.728 INFO:tasks.workunit.client.1.smithi044.stdout:4/621: creat d5/d31/d24/d38/d84/fcb x:0 0 0 2024-04-22T23:46:12.729 INFO:tasks.workunit.client.1.smithi044.stdout:0/631: fdatasync d0/d4c/d59/d5a/fae 0 2024-04-22T23:46:12.730 INFO:tasks.workunit.client.1.smithi044.stdout:0/632: chown d0/d4c/fb9 389073553 1 2024-04-22T23:46:12.732 INFO:tasks.workunit.client.1.smithi044.stdout:5/606: symlink d15/d16/d1d/d22/d81/lc1 0 2024-04-22T23:46:12.736 INFO:tasks.workunit.client.1.smithi044.stdout:4/622: rename d5/d31/d44/c1a to d5/db/d16/d4d/dad/dc5/ccc 0 2024-04-22T23:46:12.738 INFO:tasks.workunit.client.1.smithi044.stdout:6/455: dwrite d5/d3f/f63 [0,4194304] 0 2024-04-22T23:46:12.739 INFO:tasks.workunit.client.1.smithi044.stdout:5/607: symlink d15/d16/d1d/d22/d2a/lc2 0 2024-04-22T23:46:12.743 INFO:tasks.workunit.client.1.smithi044.stdout:4/623: read d5/d31/d44/d20/d22/f35 [1961316,23718] 0 2024-04-22T23:46:12.745 INFO:tasks.workunit.client.1.smithi044.stdout:6/456: creat d5/dd/d28/f93 x:0 0 0 2024-04-22T23:46:12.746 INFO:tasks.workunit.client.1.smithi044.stdout:6/457: write d5/dd/f10 [844560,2875] 0 2024-04-22T23:46:12.747 INFO:tasks.workunit.client.1.smithi044.stdout:5/608: creat d15/d16/d1d/d22/d4f/d68/fc3 x:0 0 0 2024-04-22T23:46:12.749 INFO:tasks.workunit.client.1.smithi044.stdout:0/633: rename d0/d2e/ddf/l9a to d0/d60/de6/le7 0 2024-04-22T23:46:12.749 INFO:tasks.workunit.client.1.smithi044.stdout:4/624: fsync d5/d31/d44/d20/d22/fbf 0 2024-04-22T23:46:12.757 INFO:tasks.workunit.client.1.smithi044.stdout:4/625: dwrite d5/d31/d24/d38/d84/fa4 [0,4194304] 0 2024-04-22T23:46:12.762 INFO:tasks.workunit.client.1.smithi044.stdout:0/634: creat d0/d2/dd/d3e/d40/d43/d79/dbb/fe8 x:0 0 0 2024-04-22T23:46:12.762 INFO:tasks.workunit.client.1.smithi044.stdout:6/458: fsync d5/d3f/d4c/d23/f3b 0 2024-04-22T23:46:12.764 INFO:tasks.workunit.client.1.smithi044.stdout:6/459: write d5/d3f/d4c/d23/d3d/d64/f77 [352668,67544] 0 2024-04-22T23:46:12.765 INFO:tasks.workunit.client.1.smithi044.stdout:5/609: creat d15/d5d/db5/dc0/d38/d9f/fc4 x:0 0 0 2024-04-22T23:46:12.767 INFO:tasks.workunit.client.1.smithi044.stdout:4/626: fdatasync d5/d31/f54 0 2024-04-22T23:46:12.768 INFO:tasks.workunit.client.1.smithi044.stdout:4/627: write d5/d31/d44/f7d [324046,20144] 0 2024-04-22T23:46:12.772 INFO:tasks.workunit.client.1.smithi044.stdout:5/610: dwrite fe [0,4194304] 0 2024-04-22T23:46:12.773 INFO:tasks.workunit.client.1.smithi044.stdout:6/460: fdatasync d5/dd/f27 0 2024-04-22T23:46:12.781 INFO:tasks.workunit.client.1.smithi044.stdout:6/461: dwrite d5/dd/d67/f24 [0,4194304] 0 2024-04-22T23:46:12.782 INFO:tasks.workunit.client.1.smithi044.stdout:6/462: chown d5/dd/f6a 0 1 2024-04-22T23:46:12.786 INFO:tasks.workunit.client.1.smithi044.stdout:6/463: dread d5/d3f/d4c/d40/d53/d56/f58 [0,4194304] 0 2024-04-22T23:46:12.788 INFO:tasks.workunit.client.1.smithi044.stdout:5/611: mkdir d15/d5d/d64/dc5 0 2024-04-22T23:46:12.790 INFO:tasks.workunit.client.1.smithi044.stdout:0/635: dwrite d0/d2/dd/d3e/d40/d43/d7b/f58 [0,4194304] 0 2024-04-22T23:46:12.790 INFO:tasks.workunit.client.1.smithi044.stdout:6/464: write d5/d3f/d4c/d23/f29 [4894294,3007] 0 2024-04-22T23:46:12.792 INFO:tasks.workunit.client.1.smithi044.stdout:4/628: unlink d5/d31/d24/l5e 0 2024-04-22T23:46:12.796 INFO:tasks.workunit.client.1.smithi044.stdout:5/612: creat d15/d16/d1d/d22/d2a/d3a/d43/fc6 x:0 0 0 2024-04-22T23:46:12.797 INFO:tasks.workunit.client.1.smithi044.stdout:6/465: creat d5/d3f/d4c/d23/d3d/d64/f94 x:0 0 0 2024-04-22T23:46:12.800 INFO:tasks.workunit.client.1.smithi044.stdout:4/629: creat d5/d31/d24/d69/fcd x:0 0 0 2024-04-22T23:46:12.801 INFO:tasks.workunit.client.1.smithi044.stdout:4/630: write d5/d31/d44/d20/d22/f62 [5170064,18884] 0 2024-04-22T23:46:12.801 INFO:tasks.workunit.client.1.smithi044.stdout:6/466: symlink d5/d3f/d4c/d40/l95 0 2024-04-22T23:46:12.805 INFO:tasks.workunit.client.1.smithi044.stdout:5/613: creat d15/d16/dbc/fc7 x:0 0 0 2024-04-22T23:46:12.810 INFO:tasks.workunit.client.1.smithi044.stdout:0/636: link d0/d2/f8e d0/d2/dd/d3e/fe9 0 2024-04-22T23:46:12.812 INFO:tasks.workunit.client.1.smithi044.stdout:5/614: dwrite d15/d16/dbc/fc7 [0,4194304] 0 2024-04-22T23:46:12.813 INFO:tasks.workunit.client.1.smithi044.stdout:5/615: fdatasync d15/d5d/d64/d6f/f74 0 2024-04-22T23:46:12.822 INFO:tasks.workunit.client.1.smithi044.stdout:0/637: mknod d0/d2/cea 0 2024-04-22T23:46:12.822 INFO:tasks.workunit.client.1.smithi044.stdout:5/616: dwrite d15/d5d/db5/dc0/f66 [0,4194304] 0 2024-04-22T23:46:12.824 INFO:tasks.workunit.client.1.smithi044.stdout:0/638: read d0/d4c/d59/fb3 [170532,95242] 0 2024-04-22T23:46:12.826 INFO:tasks.workunit.client.1.smithi044.stdout:4/631: dread d5/d31/d44/f94 [0,4194304] 0 2024-04-22T23:46:12.827 INFO:tasks.workunit.client.1.smithi044.stdout:6/467: getdents d5/dd/d28/d75/d88 0 2024-04-22T23:46:12.828 INFO:tasks.workunit.client.1.smithi044.stdout:6/468: fsync d5/dd/f10 0 2024-04-22T23:46:12.833 INFO:tasks.workunit.client.1.smithi044.stdout:5/617: chown d15/d3c/f53 85032 1 2024-04-22T23:46:12.833 INFO:tasks.workunit.client.1.smithi044.stdout:5/618: stat d15/f30 0 2024-04-22T23:46:12.836 INFO:tasks.workunit.client.1.smithi044.stdout:6/469: dwrite d5/dd/f27 [0,4194304] 0 2024-04-22T23:46:12.842 INFO:tasks.workunit.client.1.smithi044.stdout:5/619: rename d15/d16/d1d/d22/d4f/d99/fb9 to d15/d3c/d65/d8d/fc8 0 2024-04-22T23:46:12.843 INFO:tasks.workunit.client.1.smithi044.stdout:5/620: dread - d15/d5d/d64/d6f/faa zero size 2024-04-22T23:46:12.843 INFO:tasks.workunit.client.1.smithi044.stdout:6/470: mkdir d5/d3f/d50/d79/d96 0 2024-04-22T23:46:12.844 INFO:tasks.workunit.client.1.smithi044.stdout:6/471: write d5/d3f/d4c/f3a [928286,68410] 0 2024-04-22T23:46:12.848 INFO:tasks.workunit.client.1.smithi044.stdout:4/632: creat d5/d31/fce x:0 0 0 2024-04-22T23:46:12.851 INFO:tasks.workunit.client.1.smithi044.stdout:5/621: dwrite d15/d5d/d82/f83 [0,4194304] 0 2024-04-22T23:46:12.855 INFO:tasks.workunit.client.1.smithi044.stdout:0/639: getdents d0/d2e/d2f 0 2024-04-22T23:46:12.859 INFO:tasks.workunit.client.1.smithi044.stdout:4/633: creat d5/d31/d24/d38/d6a/fcf x:0 0 0 2024-04-22T23:46:12.862 INFO:tasks.workunit.client.1.smithi044.stdout:0/640: mknod d0/d4c/d8b/ceb 0 2024-04-22T23:46:12.867 INFO:tasks.workunit.client.1.smithi044.stdout:4/634: dwrite d5/db/d34/fb1 [0,4194304] 0 2024-04-22T23:46:12.868 INFO:tasks.workunit.client.1.smithi044.stdout:0/641: truncate d0/d2/dd/d3e/d40/d43/d7b/f5c 844752 0 2024-04-22T23:46:12.871 INFO:tasks.workunit.client.1.smithi044.stdout:0/642: creat d0/d2e/dbf/fec x:0 0 0 2024-04-22T23:46:12.874 INFO:tasks.workunit.client.1.smithi044.stdout:6/472: truncate d5/d3f/d4c/d23/f3b 1246530 0 2024-04-22T23:46:12.875 INFO:tasks.workunit.client.1.smithi044.stdout:5/622: truncate d15/d5d/db5/dc0/f5a 3787442 0 2024-04-22T23:46:12.928 INFO:tasks.workunit.client.1.smithi044.stdout:4/635: dwrite d5/d31/d24/d69/fcd [0,4194304] 0 2024-04-22T23:46:12.928 INFO:tasks.workunit.client.1.smithi044.stdout:6/473: write d5/d3f/f54 [3295255,12407] 0 2024-04-22T23:46:12.932 INFO:tasks.workunit.client.1.smithi044.stdout:0/643: dwrite d0/d60/fe2 [0,4194304] 0 2024-04-22T23:46:12.933 INFO:tasks.workunit.client.1.smithi044.stdout:4/636: dread - d5/d31/d24/d69/fa2 zero size 2024-04-22T23:46:12.933 INFO:tasks.workunit.client.1.smithi044.stdout:6/474: dread - d5/d3f/d4c/d23/d3d/f51 zero size 2024-04-22T23:46:12.940 INFO:tasks.workunit.client.1.smithi044.stdout:5/623: rename d15/d3c/d65 to d15/d16/d1d/d22/d4f/dc9 0 2024-04-22T23:46:12.943 INFO:tasks.workunit.client.1.smithi044.stdout:4/637: dwrite d5/d31/d24/d69/fbc [0,4194304] 0 2024-04-22T23:46:12.951 INFO:tasks.workunit.client.1.smithi044.stdout:4/638: dwrite d5/d31/d24/d69/f81 [0,4194304] 0 2024-04-22T23:46:12.953 INFO:tasks.workunit.client.1.smithi044.stdout:5/624: stat d15/c29 0 2024-04-22T23:46:12.954 INFO:tasks.workunit.client.1.smithi044.stdout:0/644: symlink d0/d2/dd/d3e/led 0 2024-04-22T23:46:12.959 INFO:tasks.workunit.client.1.smithi044.stdout:6/475: mkdir d5/d3f/d4c/d92/d97 0 2024-04-22T23:46:12.961 INFO:tasks.workunit.client.1.smithi044.stdout:6/476: readlink d5/d3f/d4c/l33 0 2024-04-22T23:46:12.962 INFO:tasks.workunit.client.1.smithi044.stdout:4/639: dwrite d5/d31/d44/d20/d22/f67 [0,4194304] 0 2024-04-22T23:46:12.963 INFO:tasks.workunit.client.1.smithi044.stdout:0/645: creat d0/d4c/dbd/fee x:0 0 0 2024-04-22T23:46:12.969 INFO:tasks.workunit.client.1.smithi044.stdout:0/646: fdatasync d0/d2e/dbf/fec 0 2024-04-22T23:46:12.983 INFO:tasks.workunit.client.1.smithi044.stdout:5/625: dread d15/d16/d1d/d22/d2a/d3a/d43/f61 [0,4194304] 0 2024-04-22T23:46:12.995 INFO:tasks.workunit.client.1.smithi044.stdout:6/477: creat d5/d3f/d4c/d92/f98 x:0 0 0 2024-04-22T23:46:12.995 INFO:tasks.workunit.client.1.smithi044.stdout:6/478: read - d5/dd/f7d zero size 2024-04-22T23:46:12.996 INFO:tasks.workunit.client.1.smithi044.stdout:4/640: unlink d5/d31/d44/d20/f71 0 2024-04-22T23:46:12.998 INFO:tasks.workunit.client.1.smithi044.stdout:4/641: truncate d5/d42/fbe 921619 0 2024-04-22T23:46:12.998 INFO:tasks.workunit.client.1.smithi044.stdout:0/647: dread d0/d2/d4b/fa3 [0,4194304] 0 2024-04-22T23:46:13.002 INFO:tasks.workunit.client.1.smithi044.stdout:5/626: fsync d15/d5d/db5/dc0/f3b 0 2024-04-22T23:46:13.004 INFO:tasks.workunit.client.1.smithi044.stdout:0/648: dwrite d0/d2e/d33/f3a [0,4194304] 0 2024-04-22T23:46:13.009 INFO:tasks.workunit.client.1.smithi044.stdout:0/649: dread d0/d2/dd/d3e/f3f [0,4194304] 0 2024-04-22T23:46:13.011 INFO:tasks.workunit.client.1.smithi044.stdout:0/650: read d0/d2e/d2f/d77/fba [2258287,75498] 0 2024-04-22T23:46:13.014 INFO:tasks.workunit.client.1.smithi044.stdout:5/627: getdents d15/d16/d1d/d22/d81 0 2024-04-22T23:46:13.016 INFO:tasks.workunit.client.1.smithi044.stdout:0/651: mknod d0/d2/d4b/d6b/cef 0 2024-04-22T23:46:13.019 INFO:tasks.workunit.client.1.smithi044.stdout:6/479: write d5/d3f/d4c/d23/f26 [2302797,37009] 0 2024-04-22T23:46:13.020 INFO:tasks.workunit.client.1.smithi044.stdout:6/480: chown d5/d3f/d4c/f72 0 1 2024-04-22T23:46:13.021 INFO:tasks.workunit.client.1.smithi044.stdout:5/628: dwrite d15/d16/dbc/fc7 [0,4194304] 0 2024-04-22T23:46:13.023 INFO:tasks.workunit.client.1.smithi044.stdout:0/652: symlink d0/d2e/lf0 0 2024-04-22T23:46:13.027 INFO:tasks.workunit.client.1.smithi044.stdout:5/629: read d15/d16/f7b [402490,41402] 0 2024-04-22T23:46:13.029 INFO:tasks.workunit.client.1.smithi044.stdout:0/653: creat d0/d2e/d2f/ff1 x:0 0 0 2024-04-22T23:46:13.030 INFO:tasks.workunit.client.1.smithi044.stdout:5/630: write d15/d16/d1d/d22/d2a/d3a/d43/d80/f8c [3916458,123887] 0 2024-04-22T23:46:13.034 INFO:tasks.workunit.client.1.smithi044.stdout:0/654: creat d0/d60/ff2 x:0 0 0 2024-04-22T23:46:13.035 INFO:tasks.workunit.client.1.smithi044.stdout:6/481: creat d5/d3f/f99 x:0 0 0 2024-04-22T23:46:13.037 INFO:tasks.workunit.client.1.smithi044.stdout:5/631: mkdir d15/d5d/db5/dc0/d38/d9f/dca 0 2024-04-22T23:46:13.038 INFO:tasks.workunit.client.1.smithi044.stdout:5/632: dread - d15/d5d/d64/d6f/d7f/fbb zero size 2024-04-22T23:46:13.040 INFO:tasks.workunit.client.1.smithi044.stdout:5/633: write d15/d5d/d64/d6f/d7f/fbb [598948,76361] 0 2024-04-22T23:46:13.041 INFO:tasks.workunit.client.1.smithi044.stdout:0/655: creat d0/d2/dd/d3e/d40/d43/dde/ff3 x:0 0 0 2024-04-22T23:46:13.041 INFO:tasks.workunit.client.1.smithi044.stdout:6/482: dwrite d5/dd/d67/f5d [0,4194304] 0 2024-04-22T23:46:13.043 INFO:tasks.workunit.client.1.smithi044.stdout:6/483: chown d5/dd/d28/d75/d88/l8a 31024142 1 2024-04-22T23:46:13.052 INFO:tasks.workunit.client.1.smithi044.stdout:6/484: mkdir d5/d3f/d4c/d40/d53/d9a 0 2024-04-22T23:46:13.053 INFO:tasks.workunit.client.1.smithi044.stdout:6/485: write d5/dd/f10 [5118346,73135] 0 2024-04-22T23:46:13.054 INFO:tasks.workunit.client.1.smithi044.stdout:5/634: rmdir d15/d5d/db5/dc0/d38/d9f/dca 0 2024-04-22T23:46:13.060 INFO:tasks.workunit.client.1.smithi044.stdout:6/486: creat d5/d3f/d4c/d23/f9b x:0 0 0 2024-04-22T23:46:13.060 INFO:tasks.workunit.client.1.smithi044.stdout:0/656: write d0/d2/dd/d3e/fe9 [3699555,3608] 0 2024-04-22T23:46:13.063 INFO:tasks.workunit.client.1.smithi044.stdout:5/635: dwrite d15/d16/d1d/d22/d2a/d3a/f4b [4194304,4194304] 0 2024-04-22T23:46:13.069 INFO:tasks.workunit.client.1.smithi044.stdout:5/636: creat d15/d5d/db5/dc0/fcb x:0 0 0 2024-04-22T23:46:13.070 INFO:tasks.workunit.client.1.smithi044.stdout:0/657: dread d0/d2e/d2f/d77/f93 [0,4194304] 0 2024-04-22T23:46:13.072 INFO:tasks.workunit.client.1.smithi044.stdout:0/658: chown d0/d2/dd/d3e/d40/d43/d79 0 1 2024-04-22T23:46:13.072 INFO:tasks.workunit.client.1.smithi044.stdout:5/637: chown d15/d3c/l48 814824449 1 2024-04-22T23:46:13.073 INFO:tasks.workunit.client.1.smithi044.stdout:0/659: chown d0/d2e/d33/c38 23440208 1 2024-04-22T23:46:13.073 INFO:tasks.workunit.client.1.smithi044.stdout:5/638: stat d15/d5d/d64/d6f/f8a 0 2024-04-22T23:46:13.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/487: link d5/d3f/d4c/d23/f3b d5/dd/f9c 0 2024-04-22T23:46:13.078 INFO:tasks.workunit.client.1.smithi044.stdout:6/488: chown d5/d3f/d50/d79/f8c 24299 1 2024-04-22T23:46:13.083 INFO:tasks.workunit.client.1.smithi044.stdout:6/489: dwrite f1 [4194304,4194304] 0 2024-04-22T23:46:13.088 INFO:tasks.workunit.client.1.smithi044.stdout:6/490: dread d5/d3f/d4c/d23/d3d/d64/f77 [0,4194304] 0 2024-04-22T23:46:13.091 INFO:tasks.workunit.client.1.smithi044.stdout:0/660: write d0/d2/f6 [1361194,83447] 0 2024-04-22T23:46:13.092 INFO:tasks.workunit.client.1.smithi044.stdout:0/661: dread - d0/d2e/dbf/fec zero size 2024-04-22T23:46:13.093 INFO:tasks.workunit.client.1.smithi044.stdout:0/662: readlink d0/d2e/d2f/ld0 0 2024-04-22T23:46:13.095 INFO:tasks.workunit.client.1.smithi044.stdout:0/663: chown d0/d2/f6 61153105 1 2024-04-22T23:46:13.096 INFO:tasks.workunit.client.1.smithi044.stdout:5/639: dwrite f12 [0,4194304] 0 2024-04-22T23:46:13.097 INFO:tasks.workunit.client.1.smithi044.stdout:0/664: write d0/d2e/d2f/f51 [4645330,86235] 0 2024-04-22T23:46:13.098 INFO:tasks.workunit.client.1.smithi044.stdout:0/665: fdatasync d0/d2/d4b/f68 0 2024-04-22T23:46:13.101 INFO:tasks.workunit.client.1.smithi044.stdout:0/666: dread d0/d2/f49 [0,4194304] 0 2024-04-22T23:46:13.103 INFO:tasks.workunit.client.1.smithi044.stdout:0/667: readlink d0/d2/d4b/d6b/l9c 0 2024-04-22T23:46:13.105 INFO:tasks.workunit.client.1.smithi044.stdout:5/640: dwrite d15/d16/d1d/d22/d4f/d68/fc3 [0,4194304] 0 2024-04-22T23:46:13.109 INFO:tasks.workunit.client.1.smithi044.stdout:0/668: mkdir d0/dd6/df4 0 2024-04-22T23:46:13.110 INFO:tasks.workunit.client.1.smithi044.stdout:5/641: symlink d15/d5d/d64/lcc 0 2024-04-22T23:46:13.113 INFO:tasks.workunit.client.1.smithi044.stdout:0/669: creat d0/d2/dd/d3e/d40/d67/ff5 x:0 0 0 2024-04-22T23:46:13.115 INFO:tasks.workunit.client.1.smithi044.stdout:0/670: write d0/d2/dd/f36 [5036427,8251] 0 2024-04-22T23:46:13.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/642: symlink d15/d16/lcd 0 2024-04-22T23:46:13.117 INFO:tasks.workunit.client.1.smithi044.stdout:0/671: rmdir d0/d2e 39 2024-04-22T23:46:13.122 INFO:tasks.workunit.client.1.smithi044.stdout:5/643: creat d15/d5d/db5/dc0/d38/d9f/fce x:0 0 0 2024-04-22T23:46:13.123 INFO:tasks.workunit.client.1.smithi044.stdout:0/672: dread d0/d4c/d59/d5a/fae [0,4194304] 0 2024-04-22T23:46:13.123 INFO:tasks.workunit.client.1.smithi044.stdout:5/644: dread - d15/d5d/d64/d6f/fbe zero size 2024-04-22T23:46:13.125 INFO:tasks.workunit.client.1.smithi044.stdout:5/645: write d15/d5d/d64/d6f/d7f/fbb [949313,22117] 0 2024-04-22T23:46:13.130 INFO:tasks.workunit.client.1.smithi044.stdout:0/673: link d0/d2e/d2f/f4d d0/d2e/d2f/d77/ff6 0 2024-04-22T23:46:13.131 INFO:tasks.workunit.client.1.smithi044.stdout:5/646: creat d15/d16/d1d/d22/d2a/d3a/d43/d80/fcf x:0 0 0 2024-04-22T23:46:13.134 INFO:tasks.workunit.client.1.smithi044.stdout:0/674: read d0/d2/dd/d3e/d40/d43/f4a [3939662,36490] 0 2024-04-22T23:46:13.135 INFO:tasks.workunit.client.1.smithi044.stdout:5/647: chown d15/d16/f39 3302465 1 2024-04-22T23:46:13.135 INFO:tasks.workunit.client.1.smithi044.stdout:0/675: readlink d0/d2e/d2f/ld0 0 2024-04-22T23:46:13.135 INFO:tasks.workunit.client.1.smithi044.stdout:5/648: chown d15/d16/dbc 28246563 1 2024-04-22T23:46:13.137 INFO:tasks.workunit.client.1.smithi044.stdout:0/676: symlink d0/d60/lf7 0 2024-04-22T23:46:13.142 INFO:tasks.workunit.client.1.smithi044.stdout:5/649: dwrite d15/d5d/d64/d6f/fbe [0,4194304] 0 2024-04-22T23:46:13.143 INFO:tasks.workunit.client.1.smithi044.stdout:0/677: creat d0/d2/dd/d3e/d40/d43/ff8 x:0 0 0 2024-04-22T23:46:13.144 INFO:tasks.workunit.client.1.smithi044.stdout:0/678: dread - d0/d2e/d2f/fe3 zero size 2024-04-22T23:46:13.145 INFO:tasks.workunit.client.1.smithi044.stdout:5/650: creat d15/d5d/db5/dc0/fd0 x:0 0 0 2024-04-22T23:46:13.147 INFO:tasks.workunit.client.1.smithi044.stdout:0/679: fdatasync d0/d2/dd/f13 0 2024-04-22T23:46:13.155 INFO:tasks.workunit.client.1.smithi044.stdout:5/651: symlink d15/d16/d1d/d22/ld1 0 2024-04-22T23:46:13.155 INFO:tasks.workunit.client.1.smithi044.stdout:5/652: readlink d15/d16/lcd 0 2024-04-22T23:46:13.203 INFO:tasks.workunit.client.1.smithi044.stdout:5/653: mkdir d15/d16/d1d/d22/d2a/d3a/dd2 0 2024-04-22T23:46:13.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/680: dread d0/d4c/fbe [0,4194304] 0 2024-04-22T23:46:13.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/681: stat d0/d2/d4b/fa3 0 2024-04-22T23:46:13.206 INFO:tasks.workunit.client.1.smithi044.stdout:5/654: mknod d15/d16/d1d/d22/d4f/d99/cd3 0 2024-04-22T23:46:13.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/655: rename d15/d5d/db5/dc0/l72 to d15/d5d/db5/dc0/d38/d9f/ld4 0 2024-04-22T23:46:13.209 INFO:tasks.workunit.client.1.smithi044.stdout:5/656: fsync d15/f94 0 2024-04-22T23:46:13.210 INFO:tasks.workunit.client.1.smithi044.stdout:0/682: creat d0/d2/dd/d3e/d40/d43/ff9 x:0 0 0 2024-04-22T23:46:13.213 INFO:tasks.workunit.client.1.smithi044.stdout:5/657: fsync d15/d16/d1d/d22/d2a/d3a/d43/f58 0 2024-04-22T23:46:13.216 INFO:tasks.workunit.client.1.smithi044.stdout:0/683: dwrite d0/d2/dd/d3e/f6e [4194304,4194304] 0 2024-04-22T23:46:13.218 INFO:tasks.workunit.client.1.smithi044.stdout:5/658: mkdir d15/d16/d1d/da2/dd5 0 2024-04-22T23:46:13.220 INFO:tasks.workunit.client.1.smithi044.stdout:5/659: write d15/d16/d1d/d22/d4f/dc9/d8d/f98 [262242,114975] 0 2024-04-22T23:46:13.221 INFO:tasks.workunit.client.1.smithi044.stdout:5/660: chown d15/d5d/d64/l6e 368002 1 2024-04-22T23:46:13.224 INFO:tasks.workunit.client.1.smithi044.stdout:0/684: dread d0/d2e/d2f/d77/f9d [0,4194304] 0 2024-04-22T23:46:13.226 INFO:tasks.workunit.client.1.smithi044.stdout:5/661: symlink d15/d16/d1d/d22/d2a/d3a/d43/ld6 0 2024-04-22T23:46:13.228 INFO:tasks.workunit.client.1.smithi044.stdout:0/685: mknod d0/d2/dd/d3e/d40/d67/cfa 0 2024-04-22T23:46:13.232 INFO:tasks.workunit.client.1.smithi044.stdout:0/686: creat d0/d2e/d33/ffb x:0 0 0 2024-04-22T23:46:13.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/662: dwrite d15/d5d/d64/d6f/faa [0,4194304] 0 2024-04-22T23:46:13.235 INFO:tasks.workunit.client.1.smithi044.stdout:0/687: truncate d0/d4c/d59/d5a/fd3 8451569 0 2024-04-22T23:46:13.237 INFO:tasks.workunit.client.1.smithi044.stdout:0/688: truncate d0/d60/fb6 784482 0 2024-04-22T23:46:13.238 INFO:tasks.workunit.client.1.smithi044.stdout:5/663: creat d15/d5d/d64/dc5/fd7 x:0 0 0 2024-04-22T23:46:13.243 INFO:tasks.workunit.client.1.smithi044.stdout:0/689: dwrite d0/d2/dd/d3e/fe9 [0,4194304] 0 2024-04-22T23:46:13.245 INFO:tasks.workunit.client.1.smithi044.stdout:0/690: write d0/d60/fe2 [1120035,75450] 0 2024-04-22T23:46:13.247 INFO:tasks.workunit.client.1.smithi044.stdout:5/664: fsync d15/d5d/db5/dc0/f2e 0 2024-04-22T23:46:13.250 INFO:tasks.workunit.client.1.smithi044.stdout:0/691: creat d0/d2/dd/d3e/d40/d43/d7b/ffc x:0 0 0 2024-04-22T23:46:13.252 INFO:tasks.workunit.client.1.smithi044.stdout:5/665: creat d15/d5d/db5/dc0/d38/fd8 x:0 0 0 2024-04-22T23:46:13.253 INFO:tasks.workunit.client.1.smithi044.stdout:0/692: creat d0/d2e/d33/ffd x:0 0 0 2024-04-22T23:46:13.257 INFO:tasks.workunit.client.1.smithi044.stdout:0/693: chown d0/d2/dd/d3e/d40/d43/f66 248435 1 2024-04-22T23:46:13.257 INFO:tasks.workunit.client.1.smithi044.stdout:0/694: fdatasync d0/d2/f1e 0 2024-04-22T23:46:13.261 INFO:tasks.workunit.client.1.smithi044.stdout:5/666: rename d15/d5d/db5/dc0/l46 to d15/d5d/ld9 0 2024-04-22T23:46:13.261 INFO:tasks.workunit.client.1.smithi044.stdout:0/695: mknod d0/d2/dd/d3e/d40/d43/d79/dbb/cfe 0 2024-04-22T23:46:13.264 INFO:tasks.workunit.client.1.smithi044.stdout:5/667: creat d15/d16/d1d/d22/d2a/d3a/d43/d80/fda x:0 0 0 2024-04-22T23:46:13.269 INFO:tasks.workunit.client.1.smithi044.stdout:5/668: dread d15/d16/d1d/d22/d4f/d68/f6c [0,4194304] 0 2024-04-22T23:46:13.270 INFO:tasks.workunit.client.1.smithi044.stdout:0/696: dwrite d0/d4c/fbe [0,4194304] 0 2024-04-22T23:46:13.270 INFO:tasks.workunit.client.1.smithi044.stdout:5/669: chown d15/d3c/l48 3676424 1 2024-04-22T23:46:13.272 INFO:tasks.workunit.client.1.smithi044.stdout:5/670: write d15/d16/d1d/d22/d2a/d3a/f4b [7321042,128053] 0 2024-04-22T23:46:13.275 INFO:tasks.workunit.client.1.smithi044.stdout:5/671: dread d15/d5d/d64/d6f/d7f/f91 [0,4194304] 0 2024-04-22T23:46:13.282 INFO:tasks.workunit.client.1.smithi044.stdout:5/672: creat d15/d16/d1d/d22/d81/fdb x:0 0 0 2024-04-22T23:46:13.285 INFO:tasks.workunit.client.1.smithi044.stdout:5/673: dread d15/d5d/d82/f83 [0,4194304] 0 2024-04-22T23:46:13.294 INFO:tasks.workunit.client.1.smithi044.stdout:0/697: write d0/d2/dd/d3e/d40/d43/f66 [1025627,75089] 0 2024-04-22T23:46:13.308 INFO:tasks.workunit.client.1.smithi044.stdout:0/698: dwrite d0/d2e/d2f/d77/f9d [0,4194304] 0 2024-04-22T23:46:13.310 INFO:tasks.workunit.client.1.smithi044.stdout:0/699: read d0/d2/dd/f2b [2520526,62481] 0 2024-04-22T23:46:13.316 INFO:tasks.workunit.client.1.smithi044.stdout:0/700: rename d0/d2/dd/f11 to d0/d2/dd/d3e/d40/d67/fff 0 2024-04-22T23:46:13.317 INFO:tasks.workunit.client.1.smithi044.stdout:0/701: fsync d0/d2/dd/d3e/fa0 0 2024-04-22T23:46:13.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/702: mknod d0/d2/dd/d3e/d40/d43/c100 0 2024-04-22T23:46:13.330 INFO:tasks.workunit.client.1.smithi044.stdout:5/674: dread d15/d16/d1d/d22/f2d [0,4194304] 0 2024-04-22T23:46:13.332 INFO:tasks.workunit.client.1.smithi044.stdout:5/675: mkdir d15/d16/d1d/da2/ddc 0 2024-04-22T23:46:13.338 INFO:tasks.workunit.client.1.smithi044.stdout:5/676: symlink d15/d16/d1d/d22/d2a/d3a/d43/d80/ldd 0 2024-04-22T23:46:13.338 INFO:tasks.workunit.client.1.smithi044.stdout:5/677: symlink d15/d16/d1d/d22/d4f/lde 0 2024-04-22T23:46:13.340 INFO:tasks.workunit.client.1.smithi044.stdout:5/678: symlink d15/d5d/ldf 0 2024-04-22T23:46:13.341 INFO:tasks.workunit.client.1.smithi044.stdout:0/703: dwrite d0/d2/f64 [0,4194304] 0 2024-04-22T23:46:13.343 INFO:tasks.workunit.client.1.smithi044.stdout:3/587: sync 2024-04-22T23:46:13.343 INFO:tasks.workunit.client.1.smithi044.stdout:9/446: sync 2024-04-22T23:46:13.346 INFO:tasks.workunit.client.1.smithi044.stdout:0/704: creat d0/d2/dd/d3e/d40/d43/de4/f101 x:0 0 0 2024-04-22T23:46:13.347 INFO:tasks.workunit.client.1.smithi044.stdout:0/705: fsync d0/d2/dd/d3e/d40/f44 0 2024-04-22T23:46:13.350 INFO:tasks.workunit.client.1.smithi044.stdout:3/588: dwrite d9/db/d15/d54/f4b [0,4194304] 0 2024-04-22T23:46:13.352 INFO:tasks.workunit.client.1.smithi044.stdout:9/447: rename d2/d5 to d2/dc/d94 0 2024-04-22T23:46:13.354 INFO:tasks.workunit.client.1.smithi044.stdout:0/706: rmdir d0/d2/d4b/d6b 39 2024-04-22T23:46:13.357 INFO:tasks.workunit.client.1.smithi044.stdout:9/448: fdatasync d2/f4 0 2024-04-22T23:46:13.358 INFO:tasks.workunit.client.1.smithi044.stdout:0/707: unlink d0/d2/dd/c2d 0 2024-04-22T23:46:13.359 INFO:tasks.workunit.client.1.smithi044.stdout:9/449: truncate d2/dc/d94/f1f 4916952 0 2024-04-22T23:46:13.360 INFO:tasks.workunit.client.1.smithi044.stdout:9/450: write d2/f7f [1073122,15141] 0 2024-04-22T23:46:13.360 INFO:tasks.workunit.client.1.smithi044.stdout:0/708: chown d0/d2/dd/d3e/d40/d67/cdb 31 1 2024-04-22T23:46:13.361 INFO:tasks.workunit.client.1.smithi044.stdout:9/451: chown d2/dc/d1d/d3f/l47 1009 1 2024-04-22T23:46:13.507 INFO:tasks.workunit.client.1.smithi044.stdout:0/709: creat d0/d2/dd/d3e/d40/d43/d79/dbb/f102 x:0 0 0 2024-04-22T23:46:13.508 INFO:tasks.workunit.client.1.smithi044.stdout:9/452: symlink d2/dc/d51/l95 0 2024-04-22T23:46:13.509 INFO:tasks.workunit.client.1.smithi044.stdout:9/453: chown d2/d6/d42/c81 6 1 2024-04-22T23:46:13.515 INFO:tasks.workunit.client.1.smithi044.stdout:9/454: rename d2/dc/d1d/d34/d38/d5c/f83 to d2/dc/d1d/d34/d38/d5c/d69/f96 0 2024-04-22T23:46:13.519 INFO:tasks.workunit.client.1.smithi044.stdout:3/589: write d9/db/d15/d25/d4c/f4d [4492464,59667] 0 2024-04-22T23:46:13.522 INFO:tasks.workunit.client.1.smithi044.stdout:9/455: unlink d2/dc/d94/f7 0 2024-04-22T23:46:13.522 INFO:tasks.workunit.client.1.smithi044.stdout:0/710: link d0/d4c/lc3 d0/d2/d4b/l103 0 2024-04-22T23:46:13.528 INFO:tasks.workunit.client.1.smithi044.stdout:3/590: symlink d9/db/d33/d5d/d64/dbe/lbf 0 2024-04-22T23:46:13.529 INFO:tasks.workunit.client.1.smithi044.stdout:0/711: dwrite d0/d4c/d59/fb2 [0,4194304] 0 2024-04-22T23:46:13.533 INFO:tasks.workunit.client.1.smithi044.stdout:0/712: stat d0/d4c/d8b/cd2 0 2024-04-22T23:46:13.537 INFO:tasks.workunit.client.1.smithi044.stdout:3/591: mknod d9/db/d15/d25/d5b/d52/dbb/cc0 0 2024-04-22T23:46:13.538 INFO:tasks.workunit.client.1.smithi044.stdout:0/713: creat d0/d2e/ddf/f104 x:0 0 0 2024-04-22T23:46:13.542 INFO:tasks.workunit.client.1.smithi044.stdout:3/592: truncate d9/db/d15/d25/f6d 224406 0 2024-04-22T23:46:13.543 INFO:tasks.workunit.client.1.smithi044.stdout:3/593: fdatasync d9/db/d9f/d38/f4a 0 2024-04-22T23:46:13.544 INFO:tasks.workunit.client.1.smithi044.stdout:0/714: write d0/d2/dd/d3e/d40/d67/fff [5743708,7732] 0 2024-04-22T23:46:13.548 INFO:tasks.workunit.client.1.smithi044.stdout:7/581: sync 2024-04-22T23:46:13.552 INFO:tasks.workunit.client.1.smithi044.stdout:3/594: symlink d9/db/lc1 0 2024-04-22T23:46:13.552 INFO:tasks.workunit.client.1.smithi044.stdout:3/595: fdatasync d9/d12/f78 0 2024-04-22T23:46:13.554 INFO:tasks.workunit.client.1.smithi044.stdout:9/456: truncate d2/d6/f43 837105 0 2024-04-22T23:46:13.555 INFO:tasks.workunit.client.1.smithi044.stdout:0/715: mknod d0/d2/dd/d3e/c105 0 2024-04-22T23:46:13.557 INFO:tasks.workunit.client.1.smithi044.stdout:0/716: chown d0/d2/dd/d3e/f6e 128861 1 2024-04-22T23:46:13.557 INFO:tasks.workunit.client.1.smithi044.stdout:0/717: dread - d0/d60/fd7 zero size 2024-04-22T23:46:13.560 INFO:tasks.workunit.client.1.smithi044.stdout:9/457: rename d2/d9/c1a to d2/dc/d1d/d34/d38/d5c/d7b/c97 0 2024-04-22T23:46:13.561 INFO:tasks.workunit.client.1.smithi044.stdout:0/718: mknod d0/d2e/dbf/c106 0 2024-04-22T23:46:13.562 INFO:tasks.workunit.client.1.smithi044.stdout:7/582: dread dd/d27/f6d [0,4194304] 0 2024-04-22T23:46:13.563 INFO:tasks.workunit.client.1.smithi044.stdout:0/719: write d0/d2/dd/f1a [2913003,89168] 0 2024-04-22T23:46:13.565 INFO:tasks.workunit.client.1.smithi044.stdout:7/583: truncate f8 1514536 0 2024-04-22T23:46:13.565 INFO:tasks.workunit.client.1.smithi044.stdout:0/720: write d0/d2e/d33/ffd [148107,24090] 0 2024-04-22T23:46:13.569 INFO:tasks.workunit.client.1.smithi044.stdout:9/458: dwrite d2/f19 [0,4194304] 0 2024-04-22T23:46:13.572 INFO:tasks.workunit.client.1.smithi044.stdout:0/721: dread d0/d4c/f52 [0,4194304] 0 2024-04-22T23:46:13.572 INFO:tasks.workunit.client.1.smithi044.stdout:9/459: read d2/d6/f74 [2876806,87122] 0 2024-04-22T23:46:13.574 INFO:tasks.workunit.client.1.smithi044.stdout:9/460: chown d2/dc/d1d/c2e 6 1 2024-04-22T23:46:13.577 INFO:tasks.workunit.client.1.smithi044.stdout:9/461: read d2/dc/f27 [2798350,27013] 0 2024-04-22T23:46:13.581 INFO:tasks.workunit.client.1.smithi044.stdout:0/722: dwrite d0/d2e/ddf/fdc [0,4194304] 0 2024-04-22T23:46:13.582 INFO:tasks.workunit.client.1.smithi044.stdout:7/584: mkdir dd/d55/d5e/d60/d68/d94/dab/dd5 0 2024-04-22T23:46:13.583 INFO:tasks.workunit.client.1.smithi044.stdout:0/723: write d0/d2e/d33/ffb [535069,14543] 0 2024-04-22T23:46:13.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/462: dwrite d2/f2c [4194304,4194304] 0 2024-04-22T23:46:13.588 INFO:tasks.workunit.client.1.smithi044.stdout:9/463: write d2/dc/d1d/d34/f92 [239259,114474] 0 2024-04-22T23:46:13.598 INFO:tasks.workunit.client.1.smithi044.stdout:7/585: symlink dd/d55/d5e/ld6 0 2024-04-22T23:46:13.598 INFO:tasks.workunit.client.1.smithi044.stdout:0/724: rmdir d0/dd6 39 2024-04-22T23:46:13.600 INFO:tasks.workunit.client.1.smithi044.stdout:9/464: unlink d2/dc/c17 0 2024-04-22T23:46:13.600 INFO:tasks.workunit.client.1.smithi044.stdout:0/725: fdatasync d0/d2/dd/d3e/d40/d43/d79/dbb/fe8 0 2024-04-22T23:46:13.601 INFO:tasks.workunit.client.1.smithi044.stdout:0/726: chown d0/d2/dd/d3e/d40/d43/d79/dbb/f102 11835 1 2024-04-22T23:46:13.606 INFO:tasks.workunit.client.1.smithi044.stdout:0/727: dread d0/d2e/f7a [0,4194304] 0 2024-04-22T23:46:13.616 INFO:tasks.workunit.client.1.smithi044.stdout:9/465: dread d2/d9/f5a [0,4194304] 0 2024-04-22T23:46:13.617 INFO:tasks.workunit.client.1.smithi044.stdout:9/466: write d2/d6/f33 [8975484,47896] 0 2024-04-22T23:46:13.618 INFO:tasks.workunit.client.1.smithi044.stdout:9/467: write d2/dc/d94/f50 [3320121,91193] 0 2024-04-22T23:46:13.623 INFO:tasks.workunit.client.1.smithi044.stdout:9/468: dread d2/f2c [4194304,4194304] 0 2024-04-22T23:46:13.627 INFO:tasks.workunit.client.1.smithi044.stdout:9/469: dwrite d2/dc/f18 [0,4194304] 0 2024-04-22T23:46:13.660 INFO:tasks.workunit.client.1.smithi044.stdout:2/469: sync 2024-04-22T23:46:13.660 INFO:tasks.workunit.client.1.smithi044.stdout:4/642: sync 2024-04-22T23:46:13.660 INFO:tasks.workunit.client.1.smithi044.stdout:1/596: sync 2024-04-22T23:46:13.678 INFO:tasks.workunit.client.1.smithi044.stdout:9/470: write d2/d9/f6a [102056,106910] 0 2024-04-22T23:46:13.679 INFO:tasks.workunit.client.1.smithi044.stdout:9/471: chown d2/dc/d1d/d34/d38/d4a/d72/f8f 2309 1 2024-04-22T23:46:13.680 INFO:tasks.workunit.client.1.smithi044.stdout:4/643: symlink d5/d42/ld0 0 2024-04-22T23:46:13.681 INFO:tasks.workunit.client.1.smithi044.stdout:2/470: symlink d8/d19/d36/la1 0 2024-04-22T23:46:13.682 INFO:tasks.workunit.client.1.smithi044.stdout:1/597: symlink d1/d8/d34/ld6 0 2024-04-22T23:46:13.684 INFO:tasks.workunit.client.1.smithi044.stdout:0/728: symlink d0/d2e/l107 0 2024-04-22T23:46:13.685 INFO:tasks.workunit.client.1.smithi044.stdout:0/729: stat d0/d2e/f32 0 2024-04-22T23:46:13.685 INFO:tasks.workunit.client.1.smithi044.stdout:0/730: dread - d0/d4c/dbd/fcd zero size 2024-04-22T23:46:13.687 INFO:tasks.workunit.client.1.smithi044.stdout:9/472: unlink d2/dc/f15 0 2024-04-22T23:46:13.692 INFO:tasks.workunit.client.1.smithi044.stdout:0/731: dwrite d0/d2/dd/d3e/fe9 [0,4194304] 0 2024-04-22T23:46:13.692 INFO:tasks.workunit.client.1.smithi044.stdout:2/471: mknod d8/d19/d53/ca2 0 2024-04-22T23:46:13.696 INFO:tasks.workunit.client.1.smithi044.stdout:1/598: rename d1/d8/d1c/d23/d3e/db5 to d1/d8/d1c/d7b/d80/d4b/dd7 0 2024-04-22T23:46:13.697 INFO:tasks.workunit.client.1.smithi044.stdout:1/599: read - d1/d14/fd2 zero size 2024-04-22T23:46:13.698 INFO:tasks.workunit.client.1.smithi044.stdout:1/600: stat d1/d14/fc7 0 2024-04-22T23:46:13.698 INFO:tasks.workunit.client.1.smithi044.stdout:0/732: dread d0/d2/dd/d3e/d40/d43/d7b/f26 [0,4194304] 0 2024-04-22T23:46:13.700 INFO:tasks.workunit.client.1.smithi044.stdout:4/644: symlink d5/d31/d44/d7e/daa/ld1 0 2024-04-22T23:46:13.704 INFO:tasks.workunit.client.1.smithi044.stdout:2/472: creat d8/d27/d5b/fa3 x:0 0 0 2024-04-22T23:46:13.706 INFO:tasks.workunit.client.1.smithi044.stdout:1/601: mknod d1/d8/d34/cd8 0 2024-04-22T23:46:13.708 INFO:tasks.workunit.client.1.smithi044.stdout:1/602: write d1/d8/d1c/d7b/fb6 [924391,49024] 0 2024-04-22T23:46:13.708 INFO:tasks.workunit.client.1.smithi044.stdout:4/645: write d5/d31/d44/d20/d22/f80 [906646,14700] 0 2024-04-22T23:46:13.709 INFO:tasks.workunit.client.1.smithi044.stdout:4/646: readlink d5/d42/ld0 0 2024-04-22T23:46:13.709 INFO:tasks.workunit.client.1.smithi044.stdout:1/603: dread - d1/d14/fd2 zero size 2024-04-22T23:46:13.710 INFO:tasks.workunit.client.1.smithi044.stdout:0/733: read d0/d2e/d2f/d77/fba [457349,66212] 0 2024-04-22T23:46:13.718 INFO:tasks.workunit.client.1.smithi044.stdout:1/604: dwrite d1/d8/d1c/d23/d48/fc0 [0,4194304] 0 2024-04-22T23:46:13.720 INFO:tasks.workunit.client.1.smithi044.stdout:4/647: mkdir d5/d31/d44/dd2 0 2024-04-22T23:46:13.722 INFO:tasks.workunit.client.1.smithi044.stdout:4/648: dread d5/d31/d24/d38/f40 [0,4194304] 0 2024-04-22T23:46:13.727 INFO:tasks.workunit.client.1.smithi044.stdout:2/473: mknod d8/d19/d36/d93/ca4 0 2024-04-22T23:46:13.729 INFO:tasks.workunit.client.1.smithi044.stdout:1/605: fdatasync d1/d8/d20/d45/f3f 0 2024-04-22T23:46:13.737 INFO:tasks.workunit.client.1.smithi044.stdout:0/734: link d0/d2e/d2f/d77/f9d d0/d2/dd/d3e/d40/d43/de4/f108 0 2024-04-22T23:46:13.738 INFO:tasks.workunit.client.1.smithi044.stdout:0/735: readlink d0/d60/la4 0 2024-04-22T23:46:13.740 INFO:tasks.workunit.client.1.smithi044.stdout:6/491: sync 2024-04-22T23:46:13.740 INFO:tasks.workunit.client.1.smithi044.stdout:5/679: sync 2024-04-22T23:46:13.740 INFO:tasks.workunit.client.1.smithi044.stdout:3/596: sync 2024-04-22T23:46:13.740 INFO:tasks.workunit.client.1.smithi044.stdout:8/668: sync 2024-04-22T23:46:13.742 INFO:tasks.workunit.client.1.smithi044.stdout:2/474: creat d8/d19/d36/fa5 x:0 0 0 2024-04-22T23:46:13.746 INFO:tasks.workunit.client.1.smithi044.stdout:2/475: read - d8/d19/d36/d46/f89 zero size 2024-04-22T23:46:13.746 INFO:tasks.workunit.client.1.smithi044.stdout:1/606: creat d1/d8/d1c/d7b/fd9 x:0 0 0 2024-04-22T23:46:13.754 INFO:tasks.workunit.client.1.smithi044.stdout:0/736: write d0/d2/dd/f2b [977172,120139] 0 2024-04-22T23:46:13.755 INFO:tasks.workunit.client.1.smithi044.stdout:6/492: dread d5/dd/d67/f5d [0,4194304] 0 2024-04-22T23:46:13.759 INFO:tasks.workunit.client.1.smithi044.stdout:3/597: dwrite d9/db/d15/d54/f58 [4194304,4194304] 0 2024-04-22T23:46:13.759 INFO:tasks.workunit.client.1.smithi044.stdout:5/680: dwrite d15/d16/dbc/fc7 [0,4194304] 0 2024-04-22T23:46:13.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/669: dwrite db/d68/d8b/da7/fce [0,4194304] 0 2024-04-22T23:46:13.764 INFO:tasks.workunit.client.1.smithi044.stdout:3/598: chown d9/db/fa2 11386 1 2024-04-22T23:46:13.764 INFO:tasks.workunit.client.1.smithi044.stdout:5/681: stat d15/d16/lcd 0 2024-04-22T23:46:13.767 INFO:tasks.workunit.client.1.smithi044.stdout:4/649: link d5/d52/fc1 d5/fd3 0 2024-04-22T23:46:13.768 INFO:tasks.workunit.client.1.smithi044.stdout:5/682: write d15/d5d/d64/d6f/f74 [2291604,123429] 0 2024-04-22T23:46:13.769 INFO:tasks.workunit.client.1.smithi044.stdout:4/650: chown d5/d31/d44/d20/d93 285 1 2024-04-22T23:46:13.769 INFO:tasks.workunit.client.1.smithi044.stdout:2/476: creat d8/d19/d9e/fa6 x:0 0 0 2024-04-22T23:46:13.770 INFO:tasks.workunit.client.1.smithi044.stdout:2/477: chown d8/d19/c31 10 1 2024-04-22T23:46:13.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/607: symlink d1/d8/d1c/d7b/d80/d4b/d9c/lda 0 2024-04-22T23:46:13.775 INFO:tasks.workunit.client.1.smithi044.stdout:0/737: rmdir d0/d60/de6 39 2024-04-22T23:46:13.776 INFO:tasks.workunit.client.1.smithi044.stdout:9/473: truncate d2/d6/f53 2374463 0 2024-04-22T23:46:13.777 INFO:tasks.workunit.client.1.smithi044.stdout:9/474: chown d2/d9/l23 4661612 1 2024-04-22T23:46:13.779 INFO:tasks.workunit.client.1.smithi044.stdout:8/670: truncate db/f12 5172404 0 2024-04-22T23:46:13.783 INFO:tasks.workunit.client.1.smithi044.stdout:1/608: mkdir d1/d8/d20/d45/d75/d77/dbf/ddb 0 2024-04-22T23:46:13.788 INFO:tasks.workunit.client.1.smithi044.stdout:2/478: creat d8/d19/d36/d46/d63/d65/d94/fa7 x:0 0 0 2024-04-22T23:46:13.789 INFO:tasks.workunit.client.1.smithi044.stdout:2/479: chown d8/d19/d36/d93 393 1 2024-04-22T23:46:13.792 INFO:tasks.workunit.client.1.smithi044.stdout:0/738: unlink d0/d2/d4b/d6b/fd4 0 2024-04-22T23:46:13.793 INFO:tasks.workunit.client.1.smithi044.stdout:1/609: mknod d1/d8/d1c/d7b/d80/d4b/d9c/cdc 0 2024-04-22T23:46:13.795 INFO:tasks.workunit.client.1.smithi044.stdout:6/493: rename d5/d3f/d50/f60 to d5/dd/f9d 0 2024-04-22T23:46:13.797 INFO:tasks.workunit.client.1.smithi044.stdout:9/475: creat d2/d6/d42/d8b/f98 x:0 0 0 2024-04-22T23:46:13.797 INFO:tasks.workunit.client.1.smithi044.stdout:6/494: write d5/d3f/d4c/d23/d3d/d64/f94 [1003935,33024] 0 2024-04-22T23:46:13.799 INFO:tasks.workunit.client.1.smithi044.stdout:0/739: dread d0/d2/f6 [0,4194304] 0 2024-04-22T23:46:13.800 INFO:tasks.workunit.client.1.smithi044.stdout:0/740: truncate d0/d2e/d33/fe1 436783 0 2024-04-22T23:46:13.801 INFO:tasks.workunit.client.1.smithi044.stdout:8/671: link db/d14/d84/dbb/fbd db/d14/d9e/ff4 0 2024-04-22T23:46:13.804 INFO:tasks.workunit.client.1.smithi044.stdout:2/480: creat d8/d19/d36/d46/d63/d65/d94/fa8 x:0 0 0 2024-04-22T23:46:13.807 INFO:tasks.workunit.client.1.smithi044.stdout:1/610: creat d1/d8/d20/fdd x:0 0 0 2024-04-22T23:46:13.812 INFO:tasks.workunit.client.1.smithi044.stdout:3/599: link d9/db/d15/d25/d5b/lba d9/db/d33/d5d/d64/dbe/lc2 0 2024-04-22T23:46:13.816 INFO:tasks.workunit.client.1.smithi044.stdout:8/672: dread - db/d14/f41 zero size 2024-04-22T23:46:13.818 INFO:tasks.workunit.client.1.smithi044.stdout:2/481: creat d8/d19/d5d/fa9 x:0 0 0 2024-04-22T23:46:13.821 INFO:tasks.workunit.client.1.smithi044.stdout:1/611: mkdir d1/d8/d1c/d23/d3e/dde 0 2024-04-22T23:46:13.824 INFO:tasks.workunit.client.1.smithi044.stdout:9/476: readlink d2/dc/d1d/d3f/l62 0 2024-04-22T23:46:13.827 INFO:tasks.workunit.client.1.smithi044.stdout:3/600: symlink d9/db/d15/d25/d67/d6b/lc3 0 2024-04-22T23:46:13.829 INFO:tasks.workunit.client.1.smithi044.stdout:4/651: write d5/db/d16/fc9 [507415,3308] 0 2024-04-22T23:46:13.831 INFO:tasks.workunit.client.1.smithi044.stdout:8/673: unlink db/d14/d24/l33 0 2024-04-22T23:46:13.832 INFO:tasks.workunit.client.1.smithi044.stdout:4/652: read d5/d31/d24/d38/d6a/f89 [904803,98614] 0 2024-04-22T23:46:13.836 INFO:tasks.workunit.client.1.smithi044.stdout:9/477: truncate d2/dc/d1d/d34/d38/d5c/f65 822984 0 2024-04-22T23:46:13.837 INFO:tasks.workunit.client.1.smithi044.stdout:3/601: readlink d9/le 0 2024-04-22T23:46:13.843 INFO:tasks.workunit.client.1.smithi044.stdout:2/482: mknod d8/caa 0 2024-04-22T23:46:13.844 INFO:tasks.workunit.client.1.smithi044.stdout:4/653: mknod d5/d31/d44/cd4 0 2024-04-22T23:46:13.845 INFO:tasks.workunit.client.1.smithi044.stdout:9/478: symlink d2/d6/d42/d8b/l99 0 2024-04-22T23:46:13.846 INFO:tasks.workunit.client.1.smithi044.stdout:3/602: dread - d9/db/d15/d2e/d48/f73 zero size 2024-04-22T23:46:13.848 INFO:tasks.workunit.client.1.smithi044.stdout:3/603: read - d9/db/d15/d2e/d48/f73 zero size 2024-04-22T23:46:13.851 INFO:tasks.workunit.client.1.smithi044.stdout:6/495: dwrite d5/dd/f1e [0,4194304] 0 2024-04-22T23:46:13.851 INFO:tasks.workunit.client.1.smithi044.stdout:4/654: dread d5/db/f86 [0,4194304] 0 2024-04-22T23:46:13.857 INFO:tasks.workunit.client.1.smithi044.stdout:8/674: rmdir db/ddd/df2/deb 0 2024-04-22T23:46:13.861 INFO:tasks.workunit.client.1.smithi044.stdout:4/655: dread d5/db/d34/fb9 [0,4194304] 0 2024-04-22T23:46:13.865 INFO:tasks.workunit.client.1.smithi044.stdout:3/604: fdatasync d9/db/d33/f96 0 2024-04-22T23:46:13.866 INFO:tasks.workunit.client.1.smithi044.stdout:9/479: dwrite d2/d9/f5a [0,4194304] 0 2024-04-22T23:46:13.875 INFO:tasks.workunit.client.1.smithi044.stdout:0/741: dwrite d0/d2/d4b/f86 [0,4194304] 0 2024-04-22T23:46:13.878 INFO:tasks.workunit.client.1.smithi044.stdout:8/675: dwrite db/ddd/fe3 [0,4194304] 0 2024-04-22T23:46:13.881 INFO:tasks.workunit.client.1.smithi044.stdout:1/612: dwrite d1/d8/d20/d47/d4a/f54 [4194304,4194304] 0 2024-04-22T23:46:13.881 INFO:tasks.workunit.client.1.smithi044.stdout:2/483: getdents d8/d19/d9e 0 2024-04-22T23:46:13.882 INFO:tasks.workunit.client.1.smithi044.stdout:2/484: stat d8/d19/d96/f73 0 2024-04-22T23:46:13.885 INFO:tasks.workunit.client.1.smithi044.stdout:6/496: dread d5/d3f/d4c/f3a [0,4194304] 0 2024-04-22T23:46:13.887 INFO:tasks.workunit.client.1.smithi044.stdout:4/656: rename d5/db/l5b to d5/d31/d44/d20/d22/d8d/ld5 0 2024-04-22T23:46:13.888 INFO:tasks.workunit.client.1.smithi044.stdout:9/480: mknod d2/d6/d42/c9a 0 2024-04-22T23:46:13.893 INFO:tasks.workunit.client.1.smithi044.stdout:3/605: creat d9/db/d9f/d38/d74/fc4 x:0 0 0 2024-04-22T23:46:13.895 INFO:tasks.workunit.client.1.smithi044.stdout:7/586: sync 2024-04-22T23:46:13.896 INFO:tasks.workunit.client.1.smithi044.stdout:7/587: readlink dd/l43 0 2024-04-22T23:46:13.897 INFO:tasks.workunit.client.1.smithi044.stdout:7/588: chown dd/d79/d8c/dc0 23 1 2024-04-22T23:46:13.899 INFO:tasks.workunit.client.1.smithi044.stdout:1/613: chown d1/d8/d1c/d7b/f8b 6651872 1 2024-04-22T23:46:13.900 INFO:tasks.workunit.client.1.smithi044.stdout:2/485: creat d8/d27/d5b/d74/fab x:0 0 0 2024-04-22T23:46:13.927 INFO:tasks.workunit.client.1.smithi044.stdout:0/742: truncate d0/d2/d4b/f68 3476509 0 2024-04-22T23:46:13.928 INFO:tasks.workunit.client.1.smithi044.stdout:7/589: unlink dd/d27/f3e 0 2024-04-22T23:46:13.928 INFO:tasks.workunit.client.1.smithi044.stdout:2/486: chown d8/c13 5323797 1 2024-04-22T23:46:13.929 INFO:tasks.workunit.client.1.smithi044.stdout:3/606: dwrite d9/db/d15/d2e/d48/d8f/fa3 [0,4194304] 0 2024-04-22T23:46:13.931 INFO:tasks.workunit.client.1.smithi044.stdout:0/743: read d0/d2/f64 [3911710,38029] 0 2024-04-22T23:46:13.938 INFO:tasks.workunit.client.1.smithi044.stdout:1/614: dwrite d1/d8/f1a [0,4194304] 0 2024-04-22T23:46:13.941 INFO:tasks.workunit.client.1.smithi044.stdout:0/744: dwrite d0/d2/f8e [0,4194304] 0 2024-04-22T23:46:13.947 INFO:tasks.workunit.client.1.smithi044.stdout:7/590: creat dd/d98/fd7 x:0 0 0 2024-04-22T23:46:13.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/591: chown dd/d79/d8c 5292730 1 2024-04-22T23:46:13.950 INFO:tasks.workunit.client.1.smithi044.stdout:6/497: creat d5/d3f/d4c/f9e x:0 0 0 2024-04-22T23:46:13.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/592: chown dd/d1f/d24/f39 6 1 2024-04-22T23:46:13.950 INFO:tasks.workunit.client.1.smithi044.stdout:3/607: mkdir d9/d14/d7d/dc5 0 2024-04-22T23:46:13.954 INFO:tasks.workunit.client.1.smithi044.stdout:8/676: getdents db 0 2024-04-22T23:46:13.955 INFO:tasks.workunit.client.1.smithi044.stdout:6/498: chown d5/d3f/d4c/d40/d53/f8e 17807082 1 2024-04-22T23:46:13.957 INFO:tasks.workunit.client.1.smithi044.stdout:9/481: dwrite d2/d6/f43 [0,4194304] 0 2024-04-22T23:46:13.958 INFO:tasks.workunit.client.1.smithi044.stdout:1/615: creat d1/d8/d20/d45/d5d/fdf x:0 0 0 2024-04-22T23:46:13.960 INFO:tasks.workunit.client.1.smithi044.stdout:1/616: fsync d1/d8/d20/f2f 0 2024-04-22T23:46:13.961 INFO:tasks.workunit.client.1.smithi044.stdout:2/487: creat d8/d27/d5b/d74/d81/fac x:0 0 0 2024-04-22T23:46:13.963 INFO:tasks.workunit.client.1.smithi044.stdout:3/608: dread d9/db/d15/d2e/d48/d8f/fa3 [0,4194304] 0 2024-04-22T23:46:13.966 INFO:tasks.workunit.client.1.smithi044.stdout:6/499: creat d5/d30/d73/f9f x:0 0 0 2024-04-22T23:46:13.972 INFO:tasks.workunit.client.1.smithi044.stdout:8/677: rename db/d6c/fa5 to db/d14/d84/dc0/ff5 0 2024-04-22T23:46:13.973 INFO:tasks.workunit.client.1.smithi044.stdout:9/482: creat d2/d9/f9b x:0 0 0 2024-04-22T23:46:13.974 INFO:tasks.workunit.client.1.smithi044.stdout:8/678: stat db/d52/d9c/lbe 0 2024-04-22T23:46:13.975 INFO:tasks.workunit.client.1.smithi044.stdout:8/679: readlink db/d14/d24/lb6 0 2024-04-22T23:46:13.985 INFO:tasks.workunit.client.1.smithi044.stdout:7/593: dread dd/d1a/f7e [0,4194304] 0 2024-04-22T23:46:13.986 INFO:tasks.workunit.client.1.smithi044.stdout:7/594: chown dd/d1f/d24/f30 1348716 1 2024-04-22T23:46:13.989 INFO:tasks.workunit.client.1.smithi044.stdout:0/745: truncate d0/d4c/d59/d5a/fae 5497748 0 2024-04-22T23:46:13.990 INFO:tasks.workunit.client.1.smithi044.stdout:0/746: write d0/f17 [3078566,51530] 0 2024-04-22T23:46:13.996 INFO:tasks.workunit.client.1.smithi044.stdout:2/488: dwrite d8/f43 [0,4194304] 0 2024-04-22T23:46:13.998 INFO:tasks.workunit.client.1.smithi044.stdout:3/609: dwrite d9/db/d15/d25/d5b/f7e [0,4194304] 0 2024-04-22T23:46:13.998 INFO:tasks.workunit.client.1.smithi044.stdout:2/489: readlink d8/d19/d36/d46/l66 0 2024-04-22T23:46:14.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/610: write d9/db/d15/d54/f58 [8711844,42925] 0 2024-04-22T23:46:14.002 INFO:tasks.workunit.client.1.smithi044.stdout:3/611: dread - d9/db/d9f/d38/d74/fab zero size 2024-04-22T23:46:14.010 INFO:tasks.workunit.client.1.smithi044.stdout:3/612: dwrite d9/db/d15/f85 [0,4194304] 0 2024-04-22T23:46:14.011 INFO:tasks.workunit.client.1.smithi044.stdout:3/613: write d9/db/d15/d25/d67/f91 [851844,343] 0 2024-04-22T23:46:14.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/617: rename d1/c62 to d1/d8/d20/d9e/db9/ce0 0 2024-04-22T23:46:14.028 INFO:tasks.workunit.client.1.smithi044.stdout:7/595: rename dd/d55/d5e/d89 to dd/d55/d5e/d60/d68/d94/dbd/dd8 0 2024-04-22T23:46:14.030 INFO:tasks.workunit.client.1.smithi044.stdout:6/500: rmdir d5/d3f/d4c/d40/d53/d9a 0 2024-04-22T23:46:14.032 INFO:tasks.workunit.client.1.smithi044.stdout:3/614: rename d9/db/d15/d25/d67/f91 to d9/db/d15/d25/d4c/d86/fc6 0 2024-04-22T23:46:14.032 INFO:tasks.workunit.client.1.smithi044.stdout:9/483: creat d2/dc/d1d/d34/f9c x:0 0 0 2024-04-22T23:46:14.033 INFO:tasks.workunit.client.1.smithi044.stdout:7/596: mknod dd/d55/d5e/d60/d68/d94/cd9 0 2024-04-22T23:46:14.033 INFO:tasks.workunit.client.1.smithi044.stdout:3/615: chown d9/db/d15/d25/d5b/f7e 175194 1 2024-04-22T23:46:14.034 INFO:tasks.workunit.client.1.smithi044.stdout:3/616: readlink d9/db/d15/d25/d67/d6b/lc3 0 2024-04-22T23:46:14.036 INFO:tasks.workunit.client.1.smithi044.stdout:0/747: rename d0/d2e/ddf to d0/d60/de6/d109 0 2024-04-22T23:46:14.037 INFO:tasks.workunit.client.1.smithi044.stdout:9/484: mkdir d2/dc/d1d/d3f/d9d 0 2024-04-22T23:46:14.038 INFO:tasks.workunit.client.1.smithi044.stdout:5/683: sync 2024-04-22T23:46:14.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/597: creat dd/d1a/db7/fda x:0 0 0 2024-04-22T23:46:14.039 INFO:tasks.workunit.client.1.smithi044.stdout:6/501: symlink d5/d3f/d4c/d83/la0 0 2024-04-22T23:46:14.040 INFO:tasks.workunit.client.1.smithi044.stdout:3/617: unlink d9/d14/c61 0 2024-04-22T23:46:14.042 INFO:tasks.workunit.client.1.smithi044.stdout:0/748: rename d0/d2/dd/d3e/d40/d43/fe5 to d0/d2/d4b/dad/f10a 0 2024-04-22T23:46:14.043 INFO:tasks.workunit.client.1.smithi044.stdout:7/598: symlink dd/d1a/db7/ldb 0 2024-04-22T23:46:14.044 INFO:tasks.workunit.client.1.smithi044.stdout:5/684: mkdir d15/d5d/d64/de0 0 2024-04-22T23:46:14.045 INFO:tasks.workunit.client.1.smithi044.stdout:3/618: read d9/db/d15/d54/d3c/f81 [1720169,105392] 0 2024-04-22T23:46:14.046 INFO:tasks.workunit.client.1.smithi044.stdout:9/485: write d2/dc/d1d/d34/f49 [2072361,41806] 0 2024-04-22T23:46:14.047 INFO:tasks.workunit.client.1.smithi044.stdout:3/619: chown d9/db/d15/d25/d5b/d52/dbb 859 1 2024-04-22T23:46:14.047 INFO:tasks.workunit.client.1.smithi044.stdout:0/749: dread - d0/d2/dd/d3e/d40/f98 zero size 2024-04-22T23:46:14.048 INFO:tasks.workunit.client.1.smithi044.stdout:9/486: write d2/f3 [8858831,68851] 0 2024-04-22T23:46:14.049 INFO:tasks.workunit.client.1.smithi044.stdout:6/502: creat d5/d3f/d50/d79/d96/fa1 x:0 0 0 2024-04-22T23:46:14.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/599: rename dd/d55/d5e/d60/d68/l82 to dd/d55/d5e/d60/d68/ldc 0 2024-04-22T23:46:14.053 INFO:tasks.workunit.client.1.smithi044.stdout:5/685: creat d15/d16/d1d/d22/d4f/fe1 x:0 0 0 2024-04-22T23:46:14.056 INFO:tasks.workunit.client.1.smithi044.stdout:0/750: creat d0/d2e/dbf/f10b x:0 0 0 2024-04-22T23:46:14.059 INFO:tasks.workunit.client.1.smithi044.stdout:6/503: creat d5/d3f/d50/d79/d96/fa2 x:0 0 0 2024-04-22T23:46:14.062 INFO:tasks.workunit.client.1.smithi044.stdout:0/751: mknod d0/d2e/d33/c10c 0 2024-04-22T23:46:14.063 INFO:tasks.workunit.client.1.smithi044.stdout:3/620: symlink d9/db/d15/d54/d3c/d95/lc7 0 2024-04-22T23:46:14.065 INFO:tasks.workunit.client.1.smithi044.stdout:7/600: dread dd/d1f/d24/fa2 [0,4194304] 0 2024-04-22T23:46:14.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/504: getdents d5/dd/d28/d75/d88 0 2024-04-22T23:46:14.070 INFO:tasks.workunit.client.1.smithi044.stdout:6/505: stat d5/dd/d28/f93 0 2024-04-22T23:46:14.071 INFO:tasks.workunit.client.1.smithi044.stdout:3/621: mkdir d9/dc8 0 2024-04-22T23:46:14.071 INFO:tasks.workunit.client.1.smithi044.stdout:7/601: dwrite dd/d27/f6d [0,4194304] 0 2024-04-22T23:46:14.071 INFO:tasks.workunit.client.1.smithi044.stdout:6/506: chown d5/d3f/d4c/d23 162934 1 2024-04-22T23:46:14.073 INFO:tasks.workunit.client.1.smithi044.stdout:7/602: chown dd/d1f/d24/f19 11 1 2024-04-22T23:46:14.073 INFO:tasks.workunit.client.1.smithi044.stdout:9/487: link d2/d6/c80 d2/dc/c9e 0 2024-04-22T23:46:14.074 INFO:tasks.workunit.client.1.smithi044.stdout:6/507: read - d5/f6b zero size 2024-04-22T23:46:14.077 INFO:tasks.workunit.client.1.smithi044.stdout:6/508: truncate d5/d3f/d50/f6d 721341 0 2024-04-22T23:46:14.078 INFO:tasks.workunit.client.1.smithi044.stdout:4/657: sync 2024-04-22T23:46:14.081 INFO:tasks.workunit.client.1.smithi044.stdout:3/622: rename d9/db/d33/d5d/c47 to d9/db/d15/d54/d3c/d95/cc9 0 2024-04-22T23:46:14.082 INFO:tasks.workunit.client.1.smithi044.stdout:0/752: link d0/d2e/d2f/c42 d0/d2/dd/d3e/d40/d43/dde/c10d 0 2024-04-22T23:46:14.085 INFO:tasks.workunit.client.1.smithi044.stdout:7/603: dwrite dd/d99/f86 [0,4194304] 0 2024-04-22T23:46:14.087 INFO:tasks.workunit.client.1.smithi044.stdout:6/509: creat d5/d3f/d4c/d23/d3d/d64/fa3 x:0 0 0 2024-04-22T23:46:14.088 INFO:tasks.workunit.client.1.smithi044.stdout:4/658: fdatasync d5/d31/d44/f29 0 2024-04-22T23:46:14.090 INFO:tasks.workunit.client.1.smithi044.stdout:7/604: truncate dd/d1a/db7/dc8/fa8 982354 0 2024-04-22T23:46:14.095 INFO:tasks.workunit.client.1.smithi044.stdout:2/490: write d8/d19/f4c [1687031,9991] 0 2024-04-22T23:46:14.097 INFO:tasks.workunit.client.1.smithi044.stdout:4/659: dread d5/db/d16/fc9 [0,4194304] 0 2024-04-22T23:46:14.097 INFO:tasks.workunit.client.1.smithi044.stdout:8/680: dwrite db/d14/f41 [0,4194304] 0 2024-04-22T23:46:14.106 INFO:tasks.workunit.client.1.smithi044.stdout:7/605: dread - dd/d1f/d24/f3f zero size 2024-04-22T23:46:14.109 INFO:tasks.workunit.client.1.smithi044.stdout:3/623: dread d9/db/d15/d2e/f3f [0,4194304] 0 2024-04-22T23:46:14.110 INFO:tasks.workunit.client.1.smithi044.stdout:1/618: dwrite d1/d8/d20/d45/f97 [0,4194304] 0 2024-04-22T23:46:14.112 INFO:tasks.workunit.client.1.smithi044.stdout:1/619: write d1/d8/d1c/d7b/d80/d4b/f9a [1009647,73794] 0 2024-04-22T23:46:14.114 INFO:tasks.workunit.client.1.smithi044.stdout:6/510: rename d5/d3f/d4c/d40/d53/d56/f58 to d5/d30/fa4 0 2024-04-22T23:46:14.115 INFO:tasks.workunit.client.1.smithi044.stdout:2/491: creat d8/d19/d96/fad x:0 0 0 2024-04-22T23:46:14.116 INFO:tasks.workunit.client.1.smithi044.stdout:6/511: write d5/d3f/d4c/d40/d53/f8e [135411,81720] 0 2024-04-22T23:46:14.161 INFO:tasks.workunit.client.1.smithi044.stdout:3/624: fdatasync d9/db/d15/d2e/d48/f73 0 2024-04-22T23:46:14.161 INFO:tasks.workunit.client.1.smithi044.stdout:4/660: write d5/d31/d44/f94 [398799,107612] 0 2024-04-22T23:46:14.166 INFO:tasks.workunit.client.1.smithi044.stdout:8/681: write db/ff [3945429,35073] 0 2024-04-22T23:46:14.167 INFO:tasks.workunit.client.1.smithi044.stdout:1/620: write d1/d8/d1c/d7b/f66 [2717618,84334] 0 2024-04-22T23:46:14.167 INFO:tasks.workunit.client.1.smithi044.stdout:2/492: mkdir d8/d19/d36/d93/dae 0 2024-04-22T23:46:14.171 INFO:tasks.workunit.client.1.smithi044.stdout:5/686: dwrite d15/d5d/d64/d6f/d7f/f91 [0,4194304] 0 2024-04-22T23:46:14.174 INFO:tasks.workunit.client.1.smithi044.stdout:9/488: mkdir d2/dc/d1d/d34/d38/d5c/d69/d9f 0 2024-04-22T23:46:14.176 INFO:tasks.workunit.client.1.smithi044.stdout:9/489: chown d2/dc/d1d/d34/d38/d5c/d69/d7a/c8c 2 1 2024-04-22T23:46:14.184 INFO:tasks.workunit.client.1.smithi044.stdout:6/512: unlink d5/d3f/c36 0 2024-04-22T23:46:14.184 INFO:tasks.workunit.client.1.smithi044.stdout:3/625: write d9/db/d33/f96 [1161644,66566] 0 2024-04-22T23:46:14.191 INFO:tasks.workunit.client.1.smithi044.stdout:4/661: dwrite d5/db/d16/d4d/f8a [0,4194304] 0 2024-04-22T23:46:14.192 INFO:tasks.workunit.client.1.smithi044.stdout:2/493: chown d8/ff 34771 1 2024-04-22T23:46:14.192 INFO:tasks.workunit.client.1.smithi044.stdout:8/682: symlink db/d68/d74/lf6 0 2024-04-22T23:46:14.196 INFO:tasks.workunit.client.1.smithi044.stdout:1/621: symlink d1/d8/d20/d45/d75/d77/dbf/le1 0 2024-04-22T23:46:14.198 INFO:tasks.workunit.client.1.smithi044.stdout:0/753: dwrite d0/d2e/d33/f3a [0,4194304] 0 2024-04-22T23:46:14.200 INFO:tasks.workunit.client.1.smithi044.stdout:0/754: fdatasync d0/d2e/d33/fe1 0 2024-04-22T23:46:14.201 INFO:tasks.workunit.client.1.smithi044.stdout:0/755: chown d0/d2/dd/f10 0 1 2024-04-22T23:46:14.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/756: write d0/d2/dd/d3e/d40/d67/ff5 [527522,108480] 0 2024-04-22T23:46:14.204 INFO:tasks.workunit.client.1.smithi044.stdout:3/626: chown d9/db/d15/d2e/c5a 4090 1 2024-04-22T23:46:14.206 INFO:tasks.workunit.client.1.smithi044.stdout:0/757: chown d0/d2/dd/d3e/d40/d43/l95 7936823 1 2024-04-22T23:46:14.206 INFO:tasks.workunit.client.1.smithi044.stdout:4/662: fsync d5/d31/d24/d38/d6a/f89 0 2024-04-22T23:46:14.208 INFO:tasks.workunit.client.1.smithi044.stdout:2/494: mkdir d8/d19/d1c/daf 0 2024-04-22T23:46:14.208 INFO:tasks.workunit.client.1.smithi044.stdout:0/758: chown d0/d2/dd/d3e/d40/d43/d7b/f55 61679743 1 2024-04-22T23:46:14.216 INFO:tasks.workunit.client.1.smithi044.stdout:1/622: creat d1/d8/d20/dae/fe2 x:0 0 0 2024-04-22T23:46:14.218 INFO:tasks.workunit.client.1.smithi044.stdout:6/513: dwrite d5/d3f/d4c/f57 [0,4194304] 0 2024-04-22T23:46:14.218 INFO:tasks.workunit.client.1.smithi044.stdout:1/623: write d1/f7 [3887662,126134] 0 2024-04-22T23:46:14.219 INFO:tasks.workunit.client.1.smithi044.stdout:9/490: creat d2/dc/d1d/d34/d38/d5c/d69/d9f/fa0 x:0 0 0 2024-04-22T23:46:14.220 INFO:tasks.workunit.client.1.smithi044.stdout:8/683: dwrite db/d14/dc7/dc9/fe9 [0,4194304] 0 2024-04-22T23:46:14.224 INFO:tasks.workunit.client.1.smithi044.stdout:9/491: dread - d2/dc/d1d/d34/d38/d5c/d69/d9f/fa0 zero size 2024-04-22T23:46:14.224 INFO:tasks.workunit.client.1.smithi044.stdout:3/627: creat d9/db/d15/d25/d4c/d86/fca x:0 0 0 2024-04-22T23:46:14.228 INFO:tasks.workunit.client.1.smithi044.stdout:4/663: fsync d5/d52/f95 0 2024-04-22T23:46:14.230 INFO:tasks.workunit.client.1.smithi044.stdout:2/495: read - d8/d19/d96/f7a zero size 2024-04-22T23:46:14.231 INFO:tasks.workunit.client.1.smithi044.stdout:0/759: unlink d0/d4c/fbe 0 2024-04-22T23:46:14.234 INFO:tasks.workunit.client.1.smithi044.stdout:5/687: creat d15/d16/d1d/d22/fe2 x:0 0 0 2024-04-22T23:46:14.235 INFO:tasks.workunit.client.1.smithi044.stdout:6/514: dwrite d5/d3f/f63 [4194304,4194304] 0 2024-04-22T23:46:14.236 INFO:tasks.workunit.client.1.smithi044.stdout:5/688: chown d15/d16/d1d/da2/ddc 24 1 2024-04-22T23:46:14.242 INFO:tasks.workunit.client.1.smithi044.stdout:9/492: readlink d2/dc/l5f 0 2024-04-22T23:46:14.243 INFO:tasks.workunit.client.1.smithi044.stdout:9/493: chown d2/f7d 1 1 2024-04-22T23:46:14.243 INFO:tasks.workunit.client.1.smithi044.stdout:9/494: read - d2/d6/d42/d8b/f98 zero size 2024-04-22T23:46:14.245 INFO:tasks.workunit.client.1.smithi044.stdout:8/684: dread db/f3c [0,4194304] 0 2024-04-22T23:46:14.249 INFO:tasks.workunit.client.1.smithi044.stdout:8/685: dread db/ddd/fe3 [0,4194304] 0 2024-04-22T23:46:14.251 INFO:tasks.workunit.client.1.smithi044.stdout:3/628: rename d9/d14/d7d/dc5 to d9/db/d15/d54/d3c/d95/dcb 0 2024-04-22T23:46:14.251 INFO:tasks.workunit.client.1.smithi044.stdout:3/629: stat f7 0 2024-04-22T23:46:14.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/664: unlink d5/d31/d44/d20/d22/f35 0 2024-04-22T23:46:14.255 INFO:tasks.workunit.client.1.smithi044.stdout:2/496: fdatasync d8/d27/f28 0 2024-04-22T23:46:14.257 INFO:tasks.workunit.client.1.smithi044.stdout:0/760: readlink d0/d2/d4b/d6b/l91 0 2024-04-22T23:46:14.258 INFO:tasks.workunit.client.1.smithi044.stdout:0/761: write d0/d2/f8e [2840776,100901] 0 2024-04-22T23:46:14.262 INFO:tasks.workunit.client.1.smithi044.stdout:5/689: creat d15/d16/d1d/da2/fe3 x:0 0 0 2024-04-22T23:46:14.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/630: dread d9/db/f10 [0,4194304] 0 2024-04-22T23:46:14.270 INFO:tasks.workunit.client.1.smithi044.stdout:8/686: unlink db/d14/c94 0 2024-04-22T23:46:14.274 INFO:tasks.workunit.client.1.smithi044.stdout:4/665: creat d5/d31/d44/d20/d22/d8d/fd6 x:0 0 0 2024-04-22T23:46:14.276 INFO:tasks.workunit.client.1.smithi044.stdout:2/497: mkdir d8/d19/d5d/db0 0 2024-04-22T23:46:14.279 INFO:tasks.workunit.client.1.smithi044.stdout:5/690: dread d15/d16/d1d/d22/d2a/d3a/d43/f61 [0,4194304] 0 2024-04-22T23:46:14.280 INFO:tasks.workunit.client.1.smithi044.stdout:0/762: mkdir d0/d2/d4b/dad/dc8/d10e 0 2024-04-22T23:46:14.282 INFO:tasks.workunit.client.1.smithi044.stdout:5/691: fsync d15/d16/d1d/d22/d4f/dc9/fbd 0 2024-04-22T23:46:14.283 INFO:tasks.workunit.client.1.smithi044.stdout:1/624: dwrite d1/d8/d1c/d23/f5f [0,4194304] 0 2024-04-22T23:46:14.289 INFO:tasks.workunit.client.1.smithi044.stdout:8/687: unlink db/f1c 0 2024-04-22T23:46:14.292 INFO:tasks.workunit.client.1.smithi044.stdout:9/495: rename d2/dc/f89 to d2/d6/fa1 0 2024-04-22T23:46:14.294 INFO:tasks.workunit.client.1.smithi044.stdout:6/515: write d5/d3f/d4c/d23/f3b [744663,34296] 0 2024-04-22T23:46:14.296 INFO:tasks.workunit.client.1.smithi044.stdout:6/516: fdatasync d5/d3f/d4c/d23/d3d/d64/f94 0 2024-04-22T23:46:14.300 INFO:tasks.workunit.client.1.smithi044.stdout:2/498: dread d8/d19/f29 [0,4194304] 0 2024-04-22T23:46:14.302 INFO:tasks.workunit.client.1.smithi044.stdout:0/763: unlink d0/d2/dd/f1a 0 2024-04-22T23:46:14.304 INFO:tasks.workunit.client.1.smithi044.stdout:4/666: write d5/d52/f95 [599234,127225] 0 2024-04-22T23:46:14.307 INFO:tasks.workunit.client.1.smithi044.stdout:3/631: link d9/db/fa2 d9/d12/fcc 0 2024-04-22T23:46:14.308 INFO:tasks.workunit.client.1.smithi044.stdout:3/632: fdatasync d9/db/d9f/d38/d74/fab 0 2024-04-22T23:46:14.309 INFO:tasks.workunit.client.1.smithi044.stdout:3/633: dread - d9/db/d15/d25/d4c/d86/fca zero size 2024-04-22T23:46:14.310 INFO:tasks.workunit.client.1.smithi044.stdout:3/634: dread - d9/d12/db1/fbd zero size 2024-04-22T23:46:14.310 INFO:tasks.workunit.client.1.smithi044.stdout:3/635: readlink d9/db/d15/d54/l2c 0 2024-04-22T23:46:14.314 INFO:tasks.workunit.client.1.smithi044.stdout:9/496: chown d2/dc/d1d/d34/d38/d5c/d69/d7a/l82 469217061 1 2024-04-22T23:46:14.316 INFO:tasks.workunit.client.1.smithi044.stdout:6/517: fdatasync d5/d3f/d4c/d23/d3d/d64/f77 0 2024-04-22T23:46:14.320 INFO:tasks.workunit.client.1.smithi044.stdout:7/606: sync 2024-04-22T23:46:14.323 INFO:tasks.workunit.client.1.smithi044.stdout:6/518: dwrite d5/d3f/d50/f5a [4194304,4194304] 0 2024-04-22T23:46:14.330 INFO:tasks.workunit.client.1.smithi044.stdout:4/667: rename d5/d31/d24/d38/f9f to d5/d31/d44/d20/d22/fd7 0 2024-04-22T23:46:14.334 INFO:tasks.workunit.client.1.smithi044.stdout:1/625: mkdir d1/d8/d1c/d7b/d80/de3 0 2024-04-22T23:46:14.334 INFO:tasks.workunit.client.1.smithi044.stdout:6/519: dwrite d5/d30/f80 [0,4194304] 0 2024-04-22T23:46:14.336 INFO:tasks.workunit.client.1.smithi044.stdout:2/499: dwrite d8/d19/f20 [0,4194304] 0 2024-04-22T23:46:14.339 INFO:tasks.workunit.client.1.smithi044.stdout:2/500: chown d8/d27/f4d 963549 1 2024-04-22T23:46:14.340 INFO:tasks.workunit.client.1.smithi044.stdout:7/607: unlink dd/d79/d8c/fbc 0 2024-04-22T23:46:14.346 INFO:tasks.workunit.client.1.smithi044.stdout:5/692: creat d15/d16/d1d/d22/fe4 x:0 0 0 2024-04-22T23:46:14.349 INFO:tasks.workunit.client.1.smithi044.stdout:6/520: creat d5/d3f/d4c/d92/fa5 x:0 0 0 2024-04-22T23:46:14.349 INFO:tasks.workunit.client.1.smithi044.stdout:6/521: readlink d5/d3f/lb 0 2024-04-22T23:46:14.351 INFO:tasks.workunit.client.1.smithi044.stdout:6/522: write d5/d3f/d4c/d23/d3d/d64/f77 [1426524,122136] 0 2024-04-22T23:46:14.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/636: mkdir d9/db/d9f/dcd 0 2024-04-22T23:46:14.354 INFO:tasks.workunit.client.1.smithi044.stdout:8/688: mkdir db/d52/d9c/dad/db8/df7 0 2024-04-22T23:46:14.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/497: truncate d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f 2998073 0 2024-04-22T23:46:14.358 INFO:tasks.workunit.client.1.smithi044.stdout:3/637: dread d9/db/f10 [4194304,4194304] 0 2024-04-22T23:46:14.359 INFO:tasks.workunit.client.1.smithi044.stdout:2/501: chown d8/d19/d36/d46/c50 450096329 1 2024-04-22T23:46:14.361 INFO:tasks.workunit.client.1.smithi044.stdout:2/502: readlink d8/d60/l8a 0 2024-04-22T23:46:14.361 INFO:tasks.workunit.client.1.smithi044.stdout:9/498: chown d2/dc/d1d/d34/d38/d5c/d69/d9f 54044486 1 2024-04-22T23:46:14.361 INFO:tasks.workunit.client.1.smithi044.stdout:3/638: chown d9/db/d15/d54/d3c/fbc 96289625 1 2024-04-22T23:46:14.362 INFO:tasks.workunit.client.1.smithi044.stdout:2/503: chown d8/d19/d36/d44/f52 1468698 1 2024-04-22T23:46:14.364 INFO:tasks.workunit.client.1.smithi044.stdout:2/504: fsync d8/d19/d36/d46/d63/d65/f84 0 2024-04-22T23:46:14.365 INFO:tasks.workunit.client.1.smithi044.stdout:6/523: mknod d5/d3f/d50/ca6 0 2024-04-22T23:46:14.368 INFO:tasks.workunit.client.1.smithi044.stdout:0/764: truncate d0/d2/dd/f36 4409425 0 2024-04-22T23:46:14.369 INFO:tasks.workunit.client.1.smithi044.stdout:7/608: dread dd/d1a/db7/dc8/f49 [0,4194304] 0 2024-04-22T23:46:14.370 INFO:tasks.workunit.client.1.smithi044.stdout:9/499: symlink d2/d6/d42/la2 0 2024-04-22T23:46:14.370 INFO:tasks.workunit.client.1.smithi044.stdout:5/693: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f4c [0,4194304] 0 2024-04-22T23:46:14.371 INFO:tasks.workunit.client.1.smithi044.stdout:7/609: chown dd/d27/f46 0 1 2024-04-22T23:46:14.372 INFO:tasks.workunit.client.1.smithi044.stdout:0/765: fsync d0/d2/dd/d3e/d40/d43/de4/f101 0 2024-04-22T23:46:14.373 INFO:tasks.workunit.client.1.smithi044.stdout:5/694: write f5 [1436795,21282] 0 2024-04-22T23:46:14.374 INFO:tasks.workunit.client.1.smithi044.stdout:4/668: write d5/d31/d24/d38/f63 [691081,61517] 0 2024-04-22T23:46:14.374 INFO:tasks.workunit.client.1.smithi044.stdout:2/505: rename d8/d19/d1c/l2e to d8/d19/d36/d46/lb1 0 2024-04-22T23:46:14.377 INFO:tasks.workunit.client.1.smithi044.stdout:4/669: truncate d5/d31/fce 668273 0 2024-04-22T23:46:14.384 INFO:tasks.workunit.client.1.smithi044.stdout:6/524: mkdir d5/d3f/da7 0 2024-04-22T23:46:14.386 INFO:tasks.workunit.client.1.smithi044.stdout:6/525: write d5/d3f/d4c/d23/f9b [911112,17866] 0 2024-04-22T23:46:14.386 INFO:tasks.workunit.client.1.smithi044.stdout:7/610: chown dd/d1a/db7/dc8/l91 30 1 2024-04-22T23:46:14.387 INFO:tasks.workunit.client.1.smithi044.stdout:6/526: chown d5/d3f/d4c/d4f/f8f 205932 1 2024-04-22T23:46:14.388 INFO:tasks.workunit.client.1.smithi044.stdout:6/527: truncate d5/dd/d28/f2e 590628 0 2024-04-22T23:46:14.390 INFO:tasks.workunit.client.1.smithi044.stdout:8/689: creat db/d68/d8b/ded/ff8 x:0 0 0 2024-04-22T23:46:14.392 INFO:tasks.workunit.client.1.smithi044.stdout:3/639: write d9/db/d15/d25/d5b/d52/f6f [214476,15339] 0 2024-04-22T23:46:14.394 INFO:tasks.workunit.client.1.smithi044.stdout:0/766: mknod d0/d2/d4b/dad/dc8/c10f 0 2024-04-22T23:46:14.397 INFO:tasks.workunit.client.1.smithi044.stdout:4/670: creat d5/db/d16/d4d/fd8 x:0 0 0 2024-04-22T23:46:14.399 INFO:tasks.workunit.client.1.smithi044.stdout:5/695: rename d15/d16/d1d/d22/d2a/d3a/d43/c93 to d15/d16/d1d/da2/dd5/ce5 0 2024-04-22T23:46:14.402 INFO:tasks.workunit.client.1.smithi044.stdout:7/611: mknod dd/d1a/cdd 0 2024-04-22T23:46:14.403 INFO:tasks.workunit.client.1.smithi044.stdout:2/506: creat d8/d27/d5b/d74/d80/fb2 x:0 0 0 2024-04-22T23:46:14.405 INFO:tasks.workunit.client.1.smithi044.stdout:3/640: write d9/db/d15/d2e/f39 [773353,110010] 0 2024-04-22T23:46:14.406 INFO:tasks.workunit.client.1.smithi044.stdout:9/500: creat d2/fa3 x:0 0 0 2024-04-22T23:46:14.411 INFO:tasks.workunit.client.1.smithi044.stdout:3/641: dwrite d9/db/d33/f96 [0,4194304] 0 2024-04-22T23:46:14.411 INFO:tasks.workunit.client.1.smithi044.stdout:7/612: mkdir dd/d1a/dde 0 2024-04-22T23:46:14.412 INFO:tasks.workunit.client.1.smithi044.stdout:8/690: rmdir db/d14/d84/dbb 39 2024-04-22T23:46:14.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/696: creat d15/d16/d1d/daf/fe6 x:0 0 0 2024-04-22T23:46:14.423 INFO:tasks.workunit.client.1.smithi044.stdout:7/613: symlink dd/d55/d5e/d60/d75/ldf 0 2024-04-22T23:46:14.426 INFO:tasks.workunit.client.1.smithi044.stdout:2/507: write d8/d19/d96/f7a [723118,83776] 0 2024-04-22T23:46:14.427 INFO:tasks.workunit.client.1.smithi044.stdout:7/614: dread dd/d1a/db7/dc8/f8f [0,4194304] 0 2024-04-22T23:46:14.428 INFO:tasks.workunit.client.1.smithi044.stdout:0/767: creat d0/d2/dd/d3e/d40/d43/f110 x:0 0 0 2024-04-22T23:46:14.428 INFO:tasks.workunit.client.1.smithi044.stdout:4/671: truncate d5/d31/d24/d38/d84/fa4 3976787 0 2024-04-22T23:46:14.430 INFO:tasks.workunit.client.1.smithi044.stdout:8/691: dwrite db/f22 [8388608,4194304] 0 2024-04-22T23:46:14.430 INFO:tasks.workunit.client.1.smithi044.stdout:3/642: mkdir d9/dce 0 2024-04-22T23:46:14.432 INFO:tasks.workunit.client.1.smithi044.stdout:5/697: mknod d15/d5d/d64/d6f/ce7 0 2024-04-22T23:46:14.434 INFO:tasks.workunit.client.1.smithi044.stdout:8/692: chown db/d68/f77 249 1 2024-04-22T23:46:14.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:13 smithi031 ceph-mon[26721]: pgmap v10: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 30 MiB/s rd, 107 MiB/s wr, 250 op/s 2024-04-22T23:46:14.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:13 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:14.437 INFO:tasks.workunit.client.1.smithi044.stdout:4/672: mkdir d5/db/d16/d4d/dad/dc5/dd9 0 2024-04-22T23:46:14.440 INFO:tasks.workunit.client.1.smithi044.stdout:2/508: dwrite d8/d19/f4c [0,4194304] 0 2024-04-22T23:46:14.441 INFO:tasks.workunit.client.1.smithi044.stdout:0/768: symlink d0/d2e/l111 0 2024-04-22T23:46:14.441 INFO:tasks.workunit.client.1.smithi044.stdout:3/643: dread - d9/db/d15/d2e/d48/f73 zero size 2024-04-22T23:46:14.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/673: read d5/d31/d44/d20/d22/fbf [1149426,9346] 0 2024-04-22T23:46:14.445 INFO:tasks.workunit.client.1.smithi044.stdout:9/501: link d2/d6/c80 d2/dc/d51/ca4 0 2024-04-22T23:46:14.446 INFO:tasks.workunit.client.1.smithi044.stdout:2/509: symlink d8/d19/d9e/lb3 0 2024-04-22T23:46:14.449 INFO:tasks.workunit.client.1.smithi044.stdout:5/698: mknod d15/d5d/d64/de0/ce8 0 2024-04-22T23:46:14.451 INFO:tasks.workunit.client.1.smithi044.stdout:3/644: dwrite d9/db/d33/f96 [0,4194304] 0 2024-04-22T23:46:14.453 INFO:tasks.workunit.client.1.smithi044.stdout:8/693: dread db/f48 [0,4194304] 0 2024-04-22T23:46:14.453 INFO:tasks.workunit.client.1.smithi044.stdout:7/615: rename dd/d1a/l8b to dd/d1f/le0 0 2024-04-22T23:46:14.455 INFO:tasks.workunit.client.1.smithi044.stdout:4/674: mknod d5/d31/d24/d69/d90/cda 0 2024-04-22T23:46:14.458 INFO:tasks.workunit.client.1.smithi044.stdout:9/502: creat d2/d9/fa5 x:0 0 0 2024-04-22T23:46:14.460 INFO:tasks.workunit.client.1.smithi044.stdout:9/503: truncate d2/d6/d42/d8b/f98 182434 0 2024-04-22T23:46:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:13 smithi044 ceph-mon[34463]: pgmap v10: 65 pgs: 65 active+clean; 2.6 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 30 MiB/s rd, 107 MiB/s wr, 250 op/s 2024-04-22T23:46:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:13 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:14.461 INFO:tasks.workunit.client.1.smithi044.stdout:2/510: symlink d8/d19/d36/d46/d63/d65/d6c/lb4 0 2024-04-22T23:46:14.462 INFO:tasks.workunit.client.1.smithi044.stdout:5/699: symlink d15/d5d/d64/le9 0 2024-04-22T23:46:14.463 INFO:tasks.workunit.client.1.smithi044.stdout:5/700: chown f5 98271835 1 2024-04-22T23:46:14.466 INFO:tasks.workunit.client.1.smithi044.stdout:7/616: dread - dd/d1f/d24/f63 zero size 2024-04-22T23:46:14.467 INFO:tasks.workunit.client.1.smithi044.stdout:7/617: write dd/d79/d8c/fa9 [253232,61300] 0 2024-04-22T23:46:14.468 INFO:tasks.workunit.client.1.smithi044.stdout:0/769: creat d0/d2e/f112 x:0 0 0 2024-04-22T23:46:14.469 INFO:tasks.workunit.client.1.smithi044.stdout:0/770: fsync d0/d60/de6/d109/fdc 0 2024-04-22T23:46:14.470 INFO:tasks.workunit.client.1.smithi044.stdout:9/504: mkdir d2/dc/d1d/d34/d38/d4a/da6 0 2024-04-22T23:46:14.472 INFO:tasks.workunit.client.1.smithi044.stdout:2/511: creat d8/d27/d5b/fb5 x:0 0 0 2024-04-22T23:46:14.475 INFO:tasks.workunit.client.1.smithi044.stdout:8/694: symlink db/d14/dc7/dc9/lf9 0 2024-04-22T23:46:14.478 INFO:tasks.workunit.client.1.smithi044.stdout:3/645: rename d9/db/d33/d5d/d70/l79 to d9/db/d15/d54/d3c/d95/lcf 0 2024-04-22T23:46:14.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/646: fdatasync d9/db/d15/d54/f4b 0 2024-04-22T23:46:14.479 INFO:tasks.workunit.client.1.smithi044.stdout:7/618: creat dd/d79/d8c/d9c/fe1 x:0 0 0 2024-04-22T23:46:14.481 INFO:tasks.workunit.client.1.smithi044.stdout:0/771: rmdir d0/d2/d4b 39 2024-04-22T23:46:14.483 INFO:tasks.workunit.client.1.smithi044.stdout:9/505: mkdir d2/dc/d1d/d34/d38/d5c/d69/d7a/da7 0 2024-04-22T23:46:14.486 INFO:tasks.workunit.client.1.smithi044.stdout:8/695: creat db/d14/d9e/ffa x:0 0 0 2024-04-22T23:46:14.487 INFO:tasks.workunit.client.1.smithi044.stdout:0/772: dwrite d0/d2/dd/fa1 [4194304,4194304] 0 2024-04-22T23:46:14.490 INFO:tasks.workunit.client.1.smithi044.stdout:2/512: dread d8/d27/f4d [0,4194304] 0 2024-04-22T23:46:14.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/647: mknod d9/db/d33/cd0 0 2024-04-22T23:46:14.495 INFO:tasks.workunit.client.1.smithi044.stdout:3/648: stat d9/db/d9f/d38/c3e 0 2024-04-22T23:46:14.498 INFO:tasks.workunit.client.1.smithi044.stdout:0/773: dwrite d0/d4c/dbd/fcd [0,4194304] 0 2024-04-22T23:46:14.500 INFO:tasks.workunit.client.1.smithi044.stdout:2/513: mkdir d8/d19/d53/db6 0 2024-04-22T23:46:14.502 INFO:tasks.workunit.client.1.smithi044.stdout:3/649: readlink d9/db/d15/d25/d5b/lba 0 2024-04-22T23:46:14.504 INFO:tasks.workunit.client.1.smithi044.stdout:5/701: link d15/d5d/db5/dc0/f3b d15/fea 0 2024-04-22T23:46:14.504 INFO:tasks.workunit.client.1.smithi044.stdout:3/650: fsync d9/db/d15/d25/fb6 0 2024-04-22T23:46:14.508 INFO:tasks.workunit.client.1.smithi044.stdout:0/774: creat d0/d60/de6/d109/f113 x:0 0 0 2024-04-22T23:46:14.511 INFO:tasks.workunit.client.1.smithi044.stdout:2/514: creat d8/d19/d36/d46/d63/d65/d94/fb7 x:0 0 0 2024-04-22T23:46:14.511 INFO:tasks.workunit.client.1.smithi044.stdout:7/619: write dd/d1f/f38 [667858,63759] 0 2024-04-22T23:46:14.512 INFO:tasks.workunit.client.1.smithi044.stdout:5/702: readlink l3 0 2024-04-22T23:46:14.518 INFO:tasks.workunit.client.1.smithi044.stdout:8/696: unlink db/d14/d24/f43 0 2024-04-22T23:46:14.523 INFO:tasks.workunit.client.1.smithi044.stdout:5/703: creat d15/d16/d1d/d22/d81/feb x:0 0 0 2024-04-22T23:46:14.523 INFO:tasks.workunit.client.1.smithi044.stdout:2/515: dread d8/f41 [0,4194304] 0 2024-04-22T23:46:14.526 INFO:tasks.workunit.client.1.smithi044.stdout:5/704: dread - d15/d16/d1d/d22/d81/fdb zero size 2024-04-22T23:46:14.526 INFO:tasks.workunit.client.1.smithi044.stdout:2/516: write d8/d19/f37 [901302,123810] 0 2024-04-22T23:46:14.527 INFO:tasks.workunit.client.1.smithi044.stdout:7/620: rename dd/d55/d5e/d60/d68/ldc to dd/d99/dbb/le2 0 2024-04-22T23:46:14.528 INFO:tasks.workunit.client.1.smithi044.stdout:9/506: dwrite d2/dc/d94/f3b [0,4194304] 0 2024-04-22T23:46:14.529 INFO:tasks.workunit.client.1.smithi044.stdout:5/705: dread - d15/d5d/d64/dc5/fd7 zero size 2024-04-22T23:46:14.534 INFO:tasks.workunit.client.1.smithi044.stdout:3/651: write d9/db/d15/d25/f6d [642070,69991] 0 2024-04-22T23:46:14.535 INFO:tasks.workunit.client.1.smithi044.stdout:3/652: chown d9/d14/f2b 28 1 2024-04-22T23:46:14.538 INFO:tasks.workunit.client.1.smithi044.stdout:9/507: rmdir d2/d9 39 2024-04-22T23:46:14.539 INFO:tasks.workunit.client.1.smithi044.stdout:5/706: rmdir d15/d16/d1d/d22/d2a/d3a 39 2024-04-22T23:46:14.540 INFO:tasks.workunit.client.1.smithi044.stdout:5/707: fdatasync d15/f94 0 2024-04-22T23:46:14.540 INFO:tasks.workunit.client.1.smithi044.stdout:5/708: stat d15/d5d/db5/dc0/f2e 0 2024-04-22T23:46:14.602 INFO:tasks.workunit.client.1.smithi044.stdout:8/697: rename db/ff to db/d68/d74/ffb 0 2024-04-22T23:46:14.603 INFO:tasks.workunit.client.1.smithi044.stdout:2/517: mknod d8/d19/d36/d93/dae/cb8 0 2024-04-22T23:46:14.605 INFO:tasks.workunit.client.1.smithi044.stdout:3/653: mkdir d9/db/d15/d54/d3c/dd1 0 2024-04-22T23:46:14.605 INFO:tasks.workunit.client.1.smithi044.stdout:9/508: fsync d2/d6/fa1 0 2024-04-22T23:46:14.609 INFO:tasks.workunit.client.1.smithi044.stdout:2/518: mknod d8/d19/d36/d93/dae/cb9 0 2024-04-22T23:46:14.611 INFO:tasks.workunit.client.1.smithi044.stdout:3/654: mkdir d9/d14/dd2 0 2024-04-22T23:46:14.612 INFO:tasks.workunit.client.1.smithi044.stdout:3/655: chown d9/db/d15/d54/d3c/c6e 31697730 1 2024-04-22T23:46:14.618 INFO:tasks.workunit.client.1.smithi044.stdout:5/709: dread d15/d16/d1d/d22/d4f/dc9/d8d/f8f [0,4194304] 0 2024-04-22T23:46:14.620 INFO:tasks.workunit.client.1.smithi044.stdout:7/621: rename dd/d98/fd7 to dd/fe3 0 2024-04-22T23:46:14.620 INFO:tasks.workunit.client.1.smithi044.stdout:7/622: fdatasync fc 0 2024-04-22T23:46:14.622 INFO:tasks.workunit.client.1.smithi044.stdout:8/698: unlink db/f3f 0 2024-04-22T23:46:14.623 INFO:tasks.workunit.client.1.smithi044.stdout:3/656: symlink d9/db/d9f/dcd/ld3 0 2024-04-22T23:46:14.625 INFO:tasks.workunit.client.1.smithi044.stdout:2/519: rmdir d8/d19/d53/db6 0 2024-04-22T23:46:14.627 INFO:tasks.workunit.client.1.smithi044.stdout:7/623: creat dd/d55/d5e/fe4 x:0 0 0 2024-04-22T23:46:14.629 INFO:tasks.workunit.client.1.smithi044.stdout:8/699: truncate db/d14/f41 4358779 0 2024-04-22T23:46:14.629 INFO:tasks.workunit.client.1.smithi044.stdout:3/657: mknod d9/db/d15/d25/d67/cd4 0 2024-04-22T23:46:14.636 INFO:tasks.workunit.client.1.smithi044.stdout:3/658: mkdir d9/db/d9f/d38/d74/dd5 0 2024-04-22T23:46:14.640 INFO:tasks.workunit.client.1.smithi044.stdout:2/520: link d8/f43 d8/d27/d5b/d74/d81/fba 0 2024-04-22T23:46:14.644 INFO:tasks.workunit.client.1.smithi044.stdout:7/624: creat dd/d1f/fe5 x:0 0 0 2024-04-22T23:46:14.649 INFO:tasks.workunit.client.1.smithi044.stdout:2/521: dwrite d8/d19/d36/d46/d63/d65/d72/f8e [0,4194304] 0 2024-04-22T23:46:14.650 INFO:tasks.workunit.client.1.smithi044.stdout:8/700: dread db/d14/d9e/ff4 [0,4194304] 0 2024-04-22T23:46:14.658 INFO:tasks.workunit.client.1.smithi044.stdout:3/659: dwrite d9/d12/fcc [0,4194304] 0 2024-04-22T23:46:14.663 INFO:tasks.workunit.client.1.smithi044.stdout:3/660: symlink d9/db/d15/d54/ld6 0 2024-04-22T23:46:14.665 INFO:tasks.workunit.client.1.smithi044.stdout:3/661: readlink d9/db/d15/d25/d67/d6b/lc3 0 2024-04-22T23:46:14.668 INFO:tasks.workunit.client.1.smithi044.stdout:7/625: dread dd/d1a/db7/dc8/fa1 [0,4194304] 0 2024-04-22T23:46:14.668 INFO:tasks.workunit.client.1.smithi044.stdout:3/662: write d9/db/f10 [9943511,33490] 0 2024-04-22T23:46:14.672 INFO:tasks.workunit.client.1.smithi044.stdout:7/626: creat dd/d1a/db7/dc8/d97/fe6 x:0 0 0 2024-04-22T23:46:14.675 INFO:tasks.workunit.client.1.smithi044.stdout:2/522: dwrite d8/d19/f25 [0,4194304] 0 2024-04-22T23:46:14.677 INFO:tasks.workunit.client.1.smithi044.stdout:2/523: write d8/d19/d96/f7a [1728366,60374] 0 2024-04-22T23:46:14.679 INFO:tasks.workunit.client.1.smithi044.stdout:3/663: getdents d9/d14/dd2 0 2024-04-22T23:46:14.684 INFO:tasks.workunit.client.1.smithi044.stdout:8/701: dread db/d14/d24/f30 [0,4194304] 0 2024-04-22T23:46:14.691 INFO:tasks.workunit.client.1.smithi044.stdout:8/702: readlink db/d14/d24/l2f 0 2024-04-22T23:46:14.693 INFO:tasks.workunit.client.1.smithi044.stdout:2/524: link d8/d60/f69 d8/d27/d5b/d74/d81/fbb 0 2024-04-22T23:46:14.698 INFO:tasks.workunit.client.1.smithi044.stdout:7/627: dwrite dd/d1f/d24/f1e [0,4194304] 0 2024-04-22T23:46:14.699 INFO:tasks.workunit.client.1.smithi044.stdout:7/628: readlink dd/d1f/lb4 0 2024-04-22T23:46:14.699 INFO:tasks.workunit.client.1.smithi044.stdout:2/525: dread d8/f43 [0,4194304] 0 2024-04-22T23:46:14.702 INFO:tasks.workunit.client.1.smithi044.stdout:3/664: creat d9/db/d9f/fd7 x:0 0 0 2024-04-22T23:46:14.702 INFO:tasks.workunit.client.1.smithi044.stdout:8/703: symlink db/d14/dc7/dc9/lfc 0 2024-04-22T23:46:14.707 INFO:tasks.workunit.client.1.smithi044.stdout:7/629: dwrite dd/d1a/f8e [0,4194304] 0 2024-04-22T23:46:14.710 INFO:tasks.workunit.client.1.smithi044.stdout:7/630: write dd/d99/f86 [990529,130576] 0 2024-04-22T23:46:14.719 INFO:tasks.workunit.client.1.smithi044.stdout:7/631: mkdir dd/d27/de7 0 2024-04-22T23:46:14.719 INFO:tasks.workunit.client.1.smithi044.stdout:7/632: write fc [1620956,7425] 0 2024-04-22T23:46:14.720 INFO:tasks.workunit.client.1.smithi044.stdout:7/633: stat dd/d27/f46 0 2024-04-22T23:46:14.721 INFO:tasks.workunit.client.1.smithi044.stdout:3/665: getdents d9/db/d15/d54/d3c/d95/dcb 0 2024-04-22T23:46:14.724 INFO:tasks.workunit.client.1.smithi044.stdout:7/634: mkdir dd/d98/de8 0 2024-04-22T23:46:14.727 INFO:tasks.workunit.client.1.smithi044.stdout:8/704: dread db/d68/f77 [0,4194304] 0 2024-04-22T23:46:14.728 INFO:tasks.workunit.client.1.smithi044.stdout:3/666: unlink d9/db/d15/d54/d3c/d95/lcf 0 2024-04-22T23:46:14.729 INFO:tasks.workunit.client.1.smithi044.stdout:7/635: symlink dd/d79/d8c/db8/le9 0 2024-04-22T23:46:14.733 INFO:tasks.workunit.client.1.smithi044.stdout:3/667: mkdir d9/db/d33/d5d/d70/dd8 0 2024-04-22T23:46:14.738 INFO:tasks.workunit.client.1.smithi044.stdout:7/636: creat dd/d55/d5e/d60/d68/d94/dbd/fea x:0 0 0 2024-04-22T23:46:14.743 INFO:tasks.workunit.client.1.smithi044.stdout:7/637: mkdir dd/d99/deb 0 2024-04-22T23:46:14.747 INFO:tasks.workunit.client.1.smithi044.stdout:2/526: dwrite d8/f1a [0,4194304] 0 2024-04-22T23:46:14.748 INFO:tasks.workunit.client.1.smithi044.stdout:8/705: getdents db/d14/dc7/dc9 0 2024-04-22T23:46:14.750 INFO:tasks.workunit.client.1.smithi044.stdout:3/668: dwrite d9/db/d15/d54/d3c/f81 [0,4194304] 0 2024-04-22T23:46:14.752 INFO:tasks.workunit.client.1.smithi044.stdout:7/638: mknod dd/d27/cec 0 2024-04-22T23:46:14.757 INFO:tasks.workunit.client.1.smithi044.stdout:2/527: rename d8/d19/f32 to d8/d19/d36/d93/fbc 0 2024-04-22T23:46:14.760 INFO:tasks.workunit.client.1.smithi044.stdout:2/528: rmdir d8/d19 39 2024-04-22T23:46:14.762 INFO:tasks.workunit.client.1.smithi044.stdout:8/706: creat db/d14/d72/ffd x:0 0 0 2024-04-22T23:46:14.763 INFO:tasks.workunit.client.1.smithi044.stdout:7/639: creat dd/d55/d5e/d60/d68/d94/dbd/dc9/fed x:0 0 0 2024-04-22T23:46:14.771 INFO:tasks.workunit.client.1.smithi044.stdout:8/707: dread - db/d52/f5e zero size 2024-04-22T23:46:14.771 INFO:tasks.workunit.client.1.smithi044.stdout:7/640: dwrite dd/d55/d5e/d60/d68/d94/fbe [0,4194304] 0 2024-04-22T23:46:14.774 INFO:tasks.workunit.client.1.smithi044.stdout:8/708: write db/d14/d24/fd7 [358048,45362] 0 2024-04-22T23:46:14.775 INFO:tasks.workunit.client.1.smithi044.stdout:7/641: rename dd/f20 to dd/d79/d8c/db8/fee 0 2024-04-22T23:46:14.776 INFO:tasks.workunit.client.1.smithi044.stdout:7/642: fsync dd/d55/f58 0 2024-04-22T23:46:14.777 INFO:tasks.workunit.client.1.smithi044.stdout:7/643: write fc [3933013,43894] 0 2024-04-22T23:46:14.781 INFO:tasks.workunit.client.1.smithi044.stdout:8/709: dwrite db/d68/f7a [0,4194304] 0 2024-04-22T23:46:14.790 INFO:tasks.workunit.client.1.smithi044.stdout:7/644: symlink dd/d55/dcf/lef 0 2024-04-22T23:46:14.803 INFO:tasks.workunit.client.1.smithi044.stdout:2/529: write d8/d27/d5b/f7f [478433,40548] 0 2024-04-22T23:46:14.824 INFO:tasks.workunit.client.1.smithi044.stdout:7/645: dread dd/d1f/d24/f19 [0,4194304] 0 2024-04-22T23:46:14.830 INFO:tasks.workunit.client.1.smithi044.stdout:7/646: creat dd/d98/de8/ff0 x:0 0 0 2024-04-22T23:46:14.833 INFO:tasks.workunit.client.1.smithi044.stdout:7/647: creat dd/d79/d8c/db8/ff1 x:0 0 0 2024-04-22T23:46:14.834 INFO:tasks.workunit.client.1.smithi044.stdout:2/530: dread d8/d19/d36/d44/f55 [0,4194304] 0 2024-04-22T23:46:14.836 INFO:tasks.workunit.client.1.smithi044.stdout:7/648: fsync f8 0 2024-04-22T23:46:14.838 INFO:tasks.workunit.client.1.smithi044.stdout:2/531: fsync d8/d19/d1c/f24 0 2024-04-22T23:46:14.841 INFO:tasks.workunit.client.1.smithi044.stdout:7/649: mkdir dd/d98/de8/df2 0 2024-04-22T23:46:14.849 INFO:tasks.workunit.client.1.smithi044.stdout:2/532: rename d8/d19/d36/d46/lb1 to d8/d27/d5b/d74/lbd 0 2024-04-22T23:46:14.857 INFO:tasks.workunit.client.1.smithi044.stdout:7/650: dwrite dd/d1f/d24/f70 [0,4194304] 0 2024-04-22T23:46:14.862 INFO:tasks.workunit.client.1.smithi044.stdout:2/533: dread d8/ff [0,4194304] 0 2024-04-22T23:46:14.862 INFO:tasks.workunit.client.1.smithi044.stdout:7/651: creat dd/d79/d8c/ff3 x:0 0 0 2024-04-22T23:46:14.863 INFO:tasks.workunit.client.1.smithi044.stdout:7/652: readlink dd/d1f/l50 0 2024-04-22T23:46:14.872 INFO:tasks.workunit.client.1.smithi044.stdout:2/534: write d8/d27/f28 [684868,52015] 0 2024-04-22T23:46:14.874 INFO:tasks.workunit.client.1.smithi044.stdout:2/535: unlink d8/d19/d36/l4a 0 2024-04-22T23:46:14.943 INFO:tasks.workunit.client.1.smithi044.stdout:1/626: sync 2024-04-22T23:46:14.946 INFO:tasks.workunit.client.1.smithi044.stdout:2/536: creat d8/d19/d36/d46/d63/d65/d6c/fbe x:0 0 0 2024-04-22T23:46:14.950 INFO:tasks.workunit.client.1.smithi044.stdout:2/537: mkdir d8/d19/d36/d93/dbf 0 2024-04-22T23:46:14.951 INFO:tasks.workunit.client.1.smithi044.stdout:1/627: dread d1/d8/f26 [0,4194304] 0 2024-04-22T23:46:14.954 INFO:tasks.workunit.client.1.smithi044.stdout:2/538: creat d8/d27/d5b/d74/d80/fc0 x:0 0 0 2024-04-22T23:46:14.954 INFO:tasks.workunit.client.1.smithi044.stdout:2/539: dread - d8/d19/d36/d46/f89 zero size 2024-04-22T23:46:14.958 INFO:tasks.workunit.client.1.smithi044.stdout:1/628: rename d1/d14/f36 to d1/d8/d1c/d7b/d80/fe4 0 2024-04-22T23:46:14.965 INFO:tasks.workunit.client.1.smithi044.stdout:1/629: dwrite d1/d8/d20/d45/d75/d77/dad/fb1 [0,4194304] 0 2024-04-22T23:46:14.979 INFO:tasks.workunit.client.1.smithi044.stdout:2/540: dread f7 [0,4194304] 0 2024-04-22T23:46:14.979 INFO:tasks.workunit.client.1.smithi044.stdout:2/541: chown f4 1048190 1 2024-04-22T23:46:14.980 INFO:tasks.workunit.client.1.smithi044.stdout:1/630: dwrite d1/d8/d1c/d7b/fd9 [0,4194304] 0 2024-04-22T23:46:14.989 INFO:tasks.workunit.client.1.smithi044.stdout:1/631: dwrite d1/d8/d1c/d23/d48/d99/fc4 [0,4194304] 0 2024-04-22T23:46:15.007 INFO:tasks.workunit.client.1.smithi044.stdout:1/632: dread d1/d8/fa0 [0,4194304] 0 2024-04-22T23:46:15.014 INFO:tasks.workunit.client.1.smithi044.stdout:1/633: rename d1/d8/f12 to d1/d8/d1c/d7b/d80/fe5 0 2024-04-22T23:46:15.017 INFO:tasks.workunit.client.1.smithi044.stdout:1/634: mknod d1/d8/d1c/d7b/d80/de3/ce6 0 2024-04-22T23:46:15.018 INFO:tasks.workunit.client.1.smithi044.stdout:1/635: rmdir d1/d8/d20/d45 39 2024-04-22T23:46:15.023 INFO:tasks.workunit.client.1.smithi044.stdout:1/636: rename d1/d8/d20/d47/d4a/l6e to d1/d8/d20/d45/d75/d77/dbf/le7 0 2024-04-22T23:46:15.023 INFO:tasks.workunit.client.1.smithi044.stdout:1/637: fdatasync d1/d8/fa0 0 2024-04-22T23:46:15.025 INFO:tasks.workunit.client.1.smithi044.stdout:2/542: dwrite d8/d19/d36/d46/f67 [0,4194304] 0 2024-04-22T23:46:15.026 INFO:tasks.workunit.client.1.smithi044.stdout:2/543: chown d8/d19/f1f 4631094 1 2024-04-22T23:46:15.028 INFO:tasks.workunit.client.1.smithi044.stdout:1/638: creat d1/d8/d34/da1/dab/fe8 x:0 0 0 2024-04-22T23:46:15.029 INFO:tasks.workunit.client.1.smithi044.stdout:1/639: chown d1/d8/d20/d9e/db9 0 1 2024-04-22T23:46:15.031 INFO:tasks.workunit.client.1.smithi044.stdout:2/544: rename d8/d19/d9e to d8/d60/dc1 0 2024-04-22T23:46:15.035 INFO:tasks.workunit.client.1.smithi044.stdout:1/640: rename d1/d8/d20/d9e to d1/d8/d34/da1/dab/de9 0 2024-04-22T23:46:15.041 INFO:tasks.workunit.client.1.smithi044.stdout:1/641: rename d1/d14/d95/f96 to d1/d8/d20/fea 0 2024-04-22T23:46:15.045 INFO:tasks.workunit.client.1.smithi044.stdout:1/642: rename d1/d8/d1c/d7b/d80/d4b/dd7 to d1/d8/d20/dae/deb 0 2024-04-22T23:46:15.046 INFO:tasks.workunit.client.1.smithi044.stdout:1/643: chown d1/d8/d1c/d23/d48 17 1 2024-04-22T23:46:15.052 INFO:tasks.workunit.client.1.smithi044.stdout:1/644: dwrite d1/d8/d1c/d7b/d80/d46/f68 [0,4194304] 0 2024-04-22T23:46:15.056 INFO:tasks.workunit.client.1.smithi044.stdout:1/645: creat d1/d8/d20/d45/d75/d77/dbf/fec x:0 0 0 2024-04-22T23:46:15.056 INFO:tasks.workunit.client.1.smithi044.stdout:2/545: dread d8/d19/d5d/f7c [0,4194304] 0 2024-04-22T23:46:15.058 INFO:tasks.workunit.client.1.smithi044.stdout:2/546: dread d8/d19/f29 [0,4194304] 0 2024-04-22T23:46:15.059 INFO:tasks.workunit.client.1.smithi044.stdout:1/646: mknod d1/d8/d1c/d23/d48/ced 0 2024-04-22T23:46:15.062 INFO:tasks.workunit.client.1.smithi044.stdout:2/547: dread d8/d19/d5d/f7c [0,4194304] 0 2024-04-22T23:46:15.063 INFO:tasks.workunit.client.1.smithi044.stdout:1/647: write d1/d8/d20/d45/d75/f84 [265392,46473] 0 2024-04-22T23:46:15.066 INFO:tasks.workunit.client.1.smithi044.stdout:1/648: creat d1/d8/d20/d45/d5d/fee x:0 0 0 2024-04-22T23:46:15.067 INFO:tasks.workunit.client.1.smithi044.stdout:1/649: stat d1/d8/d20/d45/d75/d77/dbf/fec 0 2024-04-22T23:46:15.070 INFO:tasks.workunit.client.1.smithi044.stdout:1/650: truncate d1/d8/d34/f43 1384892 0 2024-04-22T23:46:15.072 INFO:tasks.workunit.client.1.smithi044.stdout:1/651: truncate d1/d8/d34/da1/fbd 495792 0 2024-04-22T23:46:15.080 INFO:tasks.workunit.client.1.smithi044.stdout:2/548: dread d8/d19/d36/f3d [0,4194304] 0 2024-04-22T23:46:15.088 INFO:tasks.workunit.client.1.smithi044.stdout:1/652: dread d1/d8/d1c/d7b/f65 [0,4194304] 0 2024-04-22T23:46:15.149 INFO:tasks.workunit.client.1.smithi044.stdout:1/653: rmdir d1/d8/d20/d47/d92 39 2024-04-22T23:46:15.163 INFO:tasks.workunit.client.1.smithi044.stdout:1/654: creat d1/d8/d20/d45/d75/d77/dbf/ddb/fef x:0 0 0 2024-04-22T23:46:15.168 INFO:tasks.workunit.client.1.smithi044.stdout:1/655: creat d1/d8/d20/d45/d75/d77/dbf/dc2/ff0 x:0 0 0 2024-04-22T23:46:15.221 INFO:tasks.workunit.client.1.smithi044.stdout:4/675: sync 2024-04-22T23:46:15.221 INFO:tasks.workunit.client.1.smithi044.stdout:6/528: sync 2024-04-22T23:46:15.222 INFO:tasks.workunit.client.1.smithi044.stdout:6/529: write d5/f4d [585217,50237] 0 2024-04-22T23:46:15.224 INFO:tasks.workunit.client.1.smithi044.stdout:6/530: dread - d5/d3f/d50/d79/f8c zero size 2024-04-22T23:46:15.225 INFO:tasks.workunit.client.1.smithi044.stdout:4/676: rename d5/db/d34/f51 to d5/db/d3c/fdb 0 2024-04-22T23:46:15.225 INFO:tasks.workunit.client.1.smithi044.stdout:6/531: write d5/dd/f68 [237656,37036] 0 2024-04-22T23:46:15.226 INFO:tasks.workunit.client.1.smithi044.stdout:4/677: read - d5/d31/d24/d69/fa2 zero size 2024-04-22T23:46:15.228 INFO:tasks.workunit.client.1.smithi044.stdout:6/532: fsync d5/f12 0 2024-04-22T23:46:15.230 INFO:tasks.workunit.client.1.smithi044.stdout:4/678: unlink d5/d31/d24/d69/f81 0 2024-04-22T23:46:15.232 INFO:tasks.workunit.client.1.smithi044.stdout:1/656: write d1/d8/d1c/d7b/f8b [2479048,89174] 0 2024-04-22T23:46:15.233 INFO:tasks.workunit.client.1.smithi044.stdout:6/533: truncate d5/dd/f16 2466163 0 2024-04-22T23:46:15.236 INFO:tasks.workunit.client.1.smithi044.stdout:4/679: rename d5/d31/d24/d38/d6a/l92 to d5/db/d16/d4d/dad/dc5/ldc 0 2024-04-22T23:46:15.244 INFO:tasks.workunit.client.1.smithi044.stdout:4/680: link d5/d31/fce d5/d31/d24/d69/d90/fdd 0 2024-04-22T23:46:15.244 INFO:tasks.workunit.client.1.smithi044.stdout:4/681: readlink d5/db/d16/l68 0 2024-04-22T23:46:15.246 INFO:tasks.workunit.client.1.smithi044.stdout:2/549: symlink d8/lc2 0 2024-04-22T23:46:15.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/682: dread d5/db/d16/fc9 [0,4194304] 0 2024-04-22T23:46:15.257 INFO:tasks.workunit.client.1.smithi044.stdout:2/550: symlink d8/d19/lc3 0 2024-04-22T23:46:15.258 INFO:tasks.workunit.client.1.smithi044.stdout:4/683: dwrite d5/d31/d44/d20/d22/d8d/f97 [0,4194304] 0 2024-04-22T23:46:15.263 INFO:tasks.workunit.client.1.smithi044.stdout:6/534: link d5/d3f/d4c/d23/d3d/f4a d5/dd/fa8 0 2024-04-22T23:46:15.263 INFO:tasks.workunit.client.1.smithi044.stdout:6/535: dread - d5/f90 zero size 2024-04-22T23:46:15.264 INFO:tasks.workunit.client.1.smithi044.stdout:8/710: sync 2024-04-22T23:46:15.264 INFO:tasks.workunit.client.1.smithi044.stdout:9/509: sync 2024-04-22T23:46:15.264 INFO:tasks.workunit.client.1.smithi044.stdout:7/653: sync 2024-04-22T23:46:15.264 INFO:tasks.workunit.client.1.smithi044.stdout:3/669: sync 2024-04-22T23:46:15.264 INFO:tasks.workunit.client.1.smithi044.stdout:0/775: sync 2024-04-22T23:46:15.264 INFO:tasks.workunit.client.1.smithi044.stdout:5/710: sync 2024-04-22T23:46:15.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/551: fsync d8/f43 0 2024-04-22T23:46:15.267 INFO:tasks.workunit.client.1.smithi044.stdout:5/711: dread - d15/d16/d1d/d22/d81/fdb zero size 2024-04-22T23:46:15.269 INFO:tasks.workunit.client.1.smithi044.stdout:5/712: read - d15/d5d/db5/dc0/d38/fd8 zero size 2024-04-22T23:46:15.269 INFO:tasks.workunit.client.1.smithi044.stdout:5/713: stat d15/l63 0 2024-04-22T23:46:15.272 INFO:tasks.workunit.client.1.smithi044.stdout:8/711: chown db/l23 0 1 2024-04-22T23:46:15.274 INFO:tasks.workunit.client.1.smithi044.stdout:7/654: fdatasync dd/d1f/d24/f28 0 2024-04-22T23:46:15.279 INFO:tasks.workunit.client.1.smithi044.stdout:8/712: read db/f12 [1729852,20455] 0 2024-04-22T23:46:15.283 INFO:tasks.workunit.client.1.smithi044.stdout:4/684: rename d5/db/d16/f3e to d5/d31/d24/fde 0 2024-04-22T23:46:15.285 INFO:tasks.workunit.client.1.smithi044.stdout:5/714: creat d15/d3c/fec x:0 0 0 2024-04-22T23:46:15.287 INFO:tasks.workunit.client.1.smithi044.stdout:2/552: dread d8/d19/d1c/f1d [0,4194304] 0 2024-04-22T23:46:15.287 INFO:tasks.workunit.client.1.smithi044.stdout:4/685: chown d5/db/lc8 1 1 2024-04-22T23:46:15.288 INFO:tasks.workunit.client.1.smithi044.stdout:2/553: readlink d8/d19/d1c/l3a 0 2024-04-22T23:46:15.290 INFO:tasks.workunit.client.1.smithi044.stdout:9/510: rename d2/f19 to d2/d6/fa8 0 2024-04-22T23:46:15.291 INFO:tasks.workunit.client.1.smithi044.stdout:7/655: dread dd/d27/fb5 [0,4194304] 0 2024-04-22T23:46:15.291 INFO:tasks.workunit.client.1.smithi044.stdout:8/713: creat db/ddd/ffe x:0 0 0 2024-04-22T23:46:15.292 INFO:tasks.workunit.client.1.smithi044.stdout:9/511: truncate d2/d6/fa1 54258 0 2024-04-22T23:46:15.299 INFO:tasks.workunit.client.1.smithi044.stdout:4/686: creat d5/d31/d24/d38/d6a/d75/fdf x:0 0 0 2024-04-22T23:46:15.300 INFO:tasks.workunit.client.1.smithi044.stdout:2/554: truncate d8/d27/d5b/d74/f79 699878 0 2024-04-22T23:46:15.303 INFO:tasks.workunit.client.1.smithi044.stdout:3/670: rename d9/db/d15/d2e/d48/d8f to d9/db/d9f/d38/dd9 0 2024-04-22T23:46:15.304 INFO:tasks.workunit.client.1.smithi044.stdout:7/656: rmdir dd/d55/d5e/d60/d68/d94/dbd/dd8 39 2024-04-22T23:46:15.308 INFO:tasks.workunit.client.1.smithi044.stdout:7/657: dread dd/d79/d8c/fa9 [0,4194304] 0 2024-04-22T23:46:15.309 INFO:tasks.workunit.client.1.smithi044.stdout:1/657: dwrite d1/d8/d20/d47/d92/fb2 [0,4194304] 0 2024-04-22T23:46:15.309 INFO:tasks.workunit.client.1.smithi044.stdout:9/512: unlink d2/d6/l29 0 2024-04-22T23:46:15.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/658: dread - d1/d8/d1c/d7b/d80/d46/fc9 zero size 2024-04-22T23:46:15.311 INFO:tasks.workunit.client.1.smithi044.stdout:1/659: chown d1/d8/d20/d47/d4a/d60 2 1 2024-04-22T23:46:15.314 INFO:tasks.workunit.client.1.smithi044.stdout:1/660: fsync d1/d8/f19 0 2024-04-22T23:46:15.324 INFO:tasks.workunit.client.1.smithi044.stdout:3/671: unlink d9/db/d33/l93 0 2024-04-22T23:46:15.329 INFO:tasks.workunit.client.1.smithi044.stdout:7/658: dread dd/d1f/d24/f30 [0,4194304] 0 2024-04-22T23:46:15.330 INFO:tasks.workunit.client.1.smithi044.stdout:6/536: dwrite d5/d30/f44 [0,4194304] 0 2024-04-22T23:46:15.332 INFO:tasks.workunit.client.1.smithi044.stdout:6/537: chown d5/dd/f10 293341 1 2024-04-22T23:46:15.335 INFO:tasks.workunit.client.1.smithi044.stdout:0/776: dwrite d0/d2e/dbc/fdd [0,4194304] 0 2024-04-22T23:46:15.337 INFO:tasks.workunit.client.1.smithi044.stdout:0/777: readlink d0/d2e/d2f/ld0 0 2024-04-22T23:46:15.338 INFO:tasks.workunit.client.1.smithi044.stdout:0/778: chown d0/d2/cea 162711 1 2024-04-22T23:46:15.339 INFO:tasks.workunit.client.1.smithi044.stdout:7/659: dwrite f5 [0,4194304] 0 2024-04-22T23:46:15.342 INFO:tasks.workunit.client.1.smithi044.stdout:1/661: unlink d1/d14/f83 0 2024-04-22T23:46:15.344 INFO:tasks.workunit.client.1.smithi044.stdout:1/662: dread - d1/d8/d20/d45/d5d/fdf zero size 2024-04-22T23:46:15.348 INFO:tasks.workunit.client.1.smithi044.stdout:7/660: dread dd/d1f/d24/f70 [0,4194304] 0 2024-04-22T23:46:15.349 INFO:tasks.workunit.client.1.smithi044.stdout:0/779: dread d0/d2/dd/d3e/d40/d43/d7b/f5c [0,4194304] 0 2024-04-22T23:46:15.350 INFO:tasks.workunit.client.1.smithi044.stdout:7/661: readlink dd/d55/dcf/lef 0 2024-04-22T23:46:15.353 INFO:tasks.workunit.client.1.smithi044.stdout:8/714: rename db/f22 to db/ddd/df2/fff 0 2024-04-22T23:46:15.358 INFO:tasks.workunit.client.1.smithi044.stdout:2/555: write d8/d19/d5d/f5f [705799,90432] 0 2024-04-22T23:46:15.363 INFO:tasks.workunit.client.1.smithi044.stdout:5/715: dwrite d15/d5d/d64/d6f/fbf [0,4194304] 0 2024-04-22T23:46:15.373 INFO:tasks.workunit.client.1.smithi044.stdout:4/687: write d5/db/d34/f59 [1137842,75178] 0 2024-04-22T23:46:15.378 INFO:tasks.workunit.client.1.smithi044.stdout:5/716: dread d15/d16/dbc/fc7 [0,4194304] 0 2024-04-22T23:46:15.389 INFO:tasks.workunit.client.1.smithi044.stdout:9/513: rename d2/dc/d94/f3b to d2/d6/fa9 0 2024-04-22T23:46:15.390 INFO:tasks.workunit.client.1.smithi044.stdout:1/663: write d1/d8/d1c/f5b [9257519,15756] 0 2024-04-22T23:46:15.396 INFO:tasks.workunit.client.1.smithi044.stdout:4/688: creat d5/d42/fe0 x:0 0 0 2024-04-22T23:46:15.399 INFO:tasks.workunit.client.1.smithi044.stdout:5/717: write d15/d16/f9a [928724,98621] 0 2024-04-22T23:46:15.400 INFO:tasks.workunit.client.1.smithi044.stdout:0/780: mknod d0/d2/c114 0 2024-04-22T23:46:15.405 INFO:tasks.workunit.client.1.smithi044.stdout:7/662: truncate dd/f3a 3584592 0 2024-04-22T23:46:15.406 INFO:tasks.workunit.client.1.smithi044.stdout:7/663: chown dd/d1f/d24 105 1 2024-04-22T23:46:15.406 INFO:tasks.workunit.client.1.smithi044.stdout:5/718: dwrite d15/d5d/db5/dc0/d38/fa5 [0,4194304] 0 2024-04-22T23:46:15.408 INFO:tasks.workunit.client.1.smithi044.stdout:7/664: chown dd/d1a/db7/dc8/fa1 13 1 2024-04-22T23:46:15.412 INFO:tasks.workunit.client.1.smithi044.stdout:3/672: rename d9/db/d15/d2e/f39 to d9/db/d15/d25/d5b/d52/fda 0 2024-04-22T23:46:15.414 INFO:tasks.workunit.client.1.smithi044.stdout:9/514: mkdir d2/d6/d42/daa 0 2024-04-22T23:46:15.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/715: unlink db/d14/d24/l64 0 2024-04-22T23:46:15.420 INFO:tasks.workunit.client.1.smithi044.stdout:2/556: fsync d8/d27/d5b/d74/f79 0 2024-04-22T23:46:15.421 INFO:tasks.workunit.client.1.smithi044.stdout:2/557: readlink d8/d60/dc1/la0 0 2024-04-22T23:46:15.431 INFO:tasks.workunit.client.1.smithi044.stdout:5/719: chown d15/d16/d1d/d22/d2a/d3a/f7a 1 1 2024-04-22T23:46:15.432 INFO:tasks.workunit.client.1.smithi044.stdout:5/720: dread - d15/d5d/db5/dc0/fcb zero size 2024-04-22T23:46:15.433 INFO:tasks.workunit.client.1.smithi044.stdout:1/664: rename d1/d8/d20 to d1/d8/d20/d47/df1 22 2024-04-22T23:46:15.433 INFO:tasks.workunit.client.1.smithi044.stdout:3/673: rmdir d9/db 39 2024-04-22T23:46:15.440 INFO:tasks.workunit.client.1.smithi044.stdout:6/538: link d5/dd/l1a d5/dd/d67/la9 0 2024-04-22T23:46:15.445 INFO:tasks.workunit.client.1.smithi044.stdout:0/781: mknod d0/d2e/dbc/c115 0 2024-04-22T23:46:15.454 INFO:tasks.workunit.client.1.smithi044.stdout:2/558: dwrite d8/d19/f1f [0,4194304] 0 2024-04-22T23:46:15.458 INFO:tasks.workunit.client.1.smithi044.stdout:8/716: creat db/d6c/f100 x:0 0 0 2024-04-22T23:46:15.459 INFO:tasks.workunit.client.1.smithi044.stdout:8/717: stat db/d14/dc7/dc9 0 2024-04-22T23:46:15.461 INFO:tasks.workunit.client.1.smithi044.stdout:4/689: link d5/d31/d24/d38/d6a/f89 d5/d31/d24/d38/d6a/d75/fe1 0 2024-04-22T23:46:15.465 INFO:tasks.workunit.client.1.smithi044.stdout:0/782: chown d0/d2/d4b/dad 6006 1 2024-04-22T23:46:15.471 INFO:tasks.workunit.client.1.smithi044.stdout:6/539: symlink d5/d3f/laa 0 2024-04-22T23:46:15.476 INFO:tasks.workunit.client.1.smithi044.stdout:7/665: rename f7 to dd/ff4 0 2024-04-22T23:46:15.481 INFO:tasks.workunit.client.1.smithi044.stdout:3/674: symlink d9/db/d9f/d38/d74/dd5/ldb 0 2024-04-22T23:46:15.481 INFO:tasks.workunit.client.1.smithi044.stdout:9/515: link d2/dc/l26 d2/dc/d1d/d34/d38/d4a/d72/lab 0 2024-04-22T23:46:15.485 INFO:tasks.workunit.client.1.smithi044.stdout:3/675: write d9/db/d15/d54/f4b [3160687,68382] 0 2024-04-22T23:46:15.485 INFO:tasks.workunit.client.1.smithi044.stdout:6/540: rmdir d5/d3f/d4c/d92 39 2024-04-22T23:46:15.488 INFO:tasks.workunit.client.1.smithi044.stdout:2/559: dwrite d8/d19/f47 [0,4194304] 0 2024-04-22T23:46:15.489 INFO:tasks.workunit.client.1.smithi044.stdout:2/560: chown d8/d27 96 1 2024-04-22T23:46:15.489 INFO:tasks.workunit.client.1.smithi044.stdout:6/541: write d5/d30/d73/f87 [136973,57227] 0 2024-04-22T23:46:15.490 INFO:tasks.workunit.client.1.smithi044.stdout:7/666: symlink dd/d55/d5e/d60/d75/lf5 0 2024-04-22T23:46:15.492 INFO:tasks.workunit.client.1.smithi044.stdout:0/783: fdatasync d0/d2/dd/f2b 0 2024-04-22T23:46:15.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/676: write d9/db/d15/d54/d3c/fbc [121745,1602] 0 2024-04-22T23:46:15.494 INFO:tasks.workunit.client.1.smithi044.stdout:3/677: readlink d9/db/d33/l77 0 2024-04-22T23:46:15.497 INFO:tasks.workunit.client.1.smithi044.stdout:4/690: creat d5/d31/fe2 x:0 0 0 2024-04-22T23:46:15.498 INFO:tasks.workunit.client.1.smithi044.stdout:4/691: chown d5/d31/d44/d20/d22/d8d 255661 1 2024-04-22T23:46:15.500 INFO:tasks.workunit.client.1.smithi044.stdout:4/692: write d5/d31/fe2 [556623,16300] 0 2024-04-22T23:46:15.500 INFO:tasks.workunit.client.1.smithi044.stdout:5/721: rename d15/d16/ca4 to d15/d5d/db5/ced 0 2024-04-22T23:46:15.504 INFO:tasks.workunit.client.1.smithi044.stdout:2/561: mknod d8/d60/cc4 0 2024-04-22T23:46:15.507 INFO:tasks.workunit.client.1.smithi044.stdout:7/667: creat dd/d99/dbb/ff6 x:0 0 0 2024-04-22T23:46:15.511 INFO:tasks.workunit.client.1.smithi044.stdout:0/784: rmdir d0/d2e/d33 39 2024-04-22T23:46:15.517 INFO:tasks.workunit.client.1.smithi044.stdout:1/665: rename d1/d8/d1c/d23/d48 to d1/d8/d20/d45/d75/df2 0 2024-04-22T23:46:15.519 INFO:tasks.workunit.client.1.smithi044.stdout:5/722: mknod d15/d5d/d64/dc5/cee 0 2024-04-22T23:46:15.520 INFO:tasks.workunit.client.1.smithi044.stdout:1/666: read d1/d8/d1c/d7b/fd1 [145014,75430] 0 2024-04-22T23:46:15.522 INFO:tasks.workunit.client.1.smithi044.stdout:5/723: truncate d15/d5d/db5/dc0/d38/f50 4868926 0 2024-04-22T23:46:15.523 INFO:tasks.workunit.client.1.smithi044.stdout:9/516: link d2/dc/d1d/d34/f9c d2/dc/d1d/d34/d38/d5c/d69/fac 0 2024-04-22T23:46:15.524 INFO:tasks.workunit.client.1.smithi044.stdout:2/562: symlink d8/d19/d36/lc5 0 2024-04-22T23:46:15.526 INFO:tasks.workunit.client.1.smithi044.stdout:9/517: truncate d2/dc/d1d/d34/d38/d5c/d69/f96 683388 0 2024-04-22T23:46:15.528 INFO:tasks.workunit.client.1.smithi044.stdout:3/678: link d9/db/d15/d25/d5b/d52/fda d9/d12/fdc 0 2024-04-22T23:46:15.532 INFO:tasks.workunit.client.1.smithi044.stdout:2/563: dwrite d8/d27/d5b/f7f [0,4194304] 0 2024-04-22T23:46:15.533 INFO:tasks.workunit.client.1.smithi044.stdout:8/718: rename db/d68/d8b/ded/ff8 to db/d14/dc7/dc9/f101 0 2024-04-22T23:46:15.538 INFO:tasks.workunit.client.1.smithi044.stdout:6/542: symlink d5/d3f/d4c/d92/d97/lab 0 2024-04-22T23:46:15.540 INFO:tasks.workunit.client.1.smithi044.stdout:1/667: dread d1/d8/d1c/d23/d3e/f7e [0,4194304] 0 2024-04-22T23:46:15.543 INFO:tasks.workunit.client.1.smithi044.stdout:0/785: mkdir d0/dd6/d116 0 2024-04-22T23:46:15.545 INFO:tasks.workunit.client.1.smithi044.stdout:9/518: dread d2/dc/d94/f1f [0,4194304] 0 2024-04-22T23:46:15.550 INFO:tasks.workunit.client.1.smithi044.stdout:4/693: creat d5/d31/d44/d20/fe3 x:0 0 0 2024-04-22T23:46:15.558 INFO:tasks.workunit.client.1.smithi044.stdout:3/679: dread d9/f9b [0,4194304] 0 2024-04-22T23:46:15.564 INFO:tasks.workunit.client.1.smithi044.stdout:6/543: fdatasync d5/d3f/d4c/d23/d3d/f51 0 2024-04-22T23:46:15.566 INFO:tasks.workunit.client.1.smithi044.stdout:6/544: chown d5/d3f/d4c/d83 28056382 1 2024-04-22T23:46:15.571 INFO:tasks.workunit.client.1.smithi044.stdout:4/694: chown d5/d31/d24/d38/d84/fa4 56228123 1 2024-04-22T23:46:15.574 INFO:tasks.workunit.client.1.smithi044.stdout:4/695: write d5/db/d34/f59 [631723,40379] 0 2024-04-22T23:46:15.576 INFO:tasks.workunit.client.1.smithi044.stdout:3/680: dread d9/d14/d7d/fb4 [0,4194304] 0 2024-04-22T23:46:15.577 INFO:tasks.workunit.client.1.smithi044.stdout:0/786: dwrite d0/d2/dd/d3e/d40/f98 [0,4194304] 0 2024-04-22T23:46:15.577 INFO:tasks.workunit.client.1.smithi044.stdout:1/668: write d1/d8/d20/d45/d5d/f86 [1972880,68126] 0 2024-04-22T23:46:15.580 INFO:tasks.workunit.client.1.smithi044.stdout:2/564: link d8/d19/d36/f3d d8/d19/d96/fc6 0 2024-04-22T23:46:15.582 INFO:tasks.workunit.client.1.smithi044.stdout:3/681: write d9/db/d9f/d38/d74/fab [805649,43368] 0 2024-04-22T23:46:15.584 INFO:tasks.workunit.client.1.smithi044.stdout:9/519: truncate d2/dc/d94/f30 2327851 0 2024-04-22T23:46:15.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/565: read - d8/d19/d36/d46/d63/d65/d6c/fbe zero size 2024-04-22T23:46:15.590 INFO:tasks.workunit.client.1.smithi044.stdout:7/668: rename dd/d79/d8c to dd/df7 0 2024-04-22T23:46:15.592 INFO:tasks.workunit.client.1.smithi044.stdout:1/669: creat d1/d8/d1c/d7b/d80/de3/ff3 x:0 0 0 2024-04-22T23:46:15.594 INFO:tasks.workunit.client.1.smithi044.stdout:1/670: read d1/d8/f18 [6036019,5101] 0 2024-04-22T23:46:15.596 INFO:tasks.workunit.client.1.smithi044.stdout:6/545: link d5/d30/f39 d5/d3f/d50/d79/d96/fac 0 2024-04-22T23:46:15.598 INFO:tasks.workunit.client.1.smithi044.stdout:8/719: symlink db/d52/d9c/dad/l102 0 2024-04-22T23:46:15.600 INFO:tasks.workunit.client.1.smithi044.stdout:4/696: dread d5/d31/d44/d20/f23 [0,4194304] 0 2024-04-22T23:46:15.606 INFO:tasks.workunit.client.1.smithi044.stdout:0/787: rename d0/d2/dd/f2b to d0/d2/d4b/dad/f117 0 2024-04-22T23:46:15.606 INFO:tasks.workunit.client.1.smithi044.stdout:0/788: fdatasync d0/d2e/f112 0 2024-04-22T23:46:15.607 INFO:tasks.workunit.client.1.smithi044.stdout:0/789: dread - d0/d60/ff2 zero size 2024-04-22T23:46:15.753 INFO:tasks.workunit.client.1.smithi044.stdout:7/669: symlink dd/d1f/d24/lf8 0 2024-04-22T23:46:15.758 INFO:tasks.workunit.client.1.smithi044.stdout:1/671: mknod d1/d8/d20/dae/deb/cf4 0 2024-04-22T23:46:15.759 INFO:tasks.workunit.client.1.smithi044.stdout:3/682: write d9/db/d15/d25/d4c/f6a [167894,111594] 0 2024-04-22T23:46:15.759 INFO:tasks.workunit.client.1.smithi044.stdout:1/672: fsync d1/d8/d20/d47/d4a/f54 0 2024-04-22T23:46:15.763 INFO:tasks.workunit.client.1.smithi044.stdout:2/566: mkdir d8/d19/d5d/db0/dc7 0 2024-04-22T23:46:15.768 INFO:tasks.workunit.client.1.smithi044.stdout:0/790: mknod d0/d2e/dbc/c118 0 2024-04-22T23:46:15.771 INFO:tasks.workunit.client.1.smithi044.stdout:6/546: dread d5/f21 [0,4194304] 0 2024-04-22T23:46:15.772 INFO:tasks.workunit.client.1.smithi044.stdout:6/547: readlink d5/d3f/d4c/l91 0 2024-04-22T23:46:15.775 INFO:tasks.workunit.client.1.smithi044.stdout:3/683: dread d9/d14/f2b [0,4194304] 0 2024-04-22T23:46:15.776 INFO:tasks.workunit.client.1.smithi044.stdout:1/673: creat d1/d8/d20/d45/d75/df2/ff5 x:0 0 0 2024-04-22T23:46:15.778 INFO:tasks.workunit.client.1.smithi044.stdout:8/720: chown db/d14/dc7/dc9/fdc 55632316 1 2024-04-22T23:46:15.778 INFO:tasks.workunit.client.1.smithi044.stdout:4/697: getdents d5/d31/d44/dd2 0 2024-04-22T23:46:15.781 INFO:tasks.workunit.client.1.smithi044.stdout:0/791: creat d0/dd6/f119 x:0 0 0 2024-04-22T23:46:15.782 INFO:tasks.workunit.client.1.smithi044.stdout:7/670: mknod dd/cf9 0 2024-04-22T23:46:15.787 INFO:tasks.workunit.client.1.smithi044.stdout:1/674: mknod d1/d8/d20/d45/d75/df2/cf6 0 2024-04-22T23:46:15.790 INFO:tasks.workunit.client.1.smithi044.stdout:9/520: dwrite d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f [0,4194304] 0 2024-04-22T23:46:15.790 INFO:tasks.workunit.client.1.smithi044.stdout:8/721: creat db/d52/f103 x:0 0 0 2024-04-22T23:46:15.791 INFO:tasks.workunit.client.1.smithi044.stdout:4/698: creat d5/d31/d24/d38/d84/fe4 x:0 0 0 2024-04-22T23:46:15.797 INFO:tasks.workunit.client.1.smithi044.stdout:0/792: mknod d0/d2e/dbc/c11a 0 2024-04-22T23:46:15.798 INFO:tasks.workunit.client.1.smithi044.stdout:0/793: chown d0/d2/dd/d3e/f3f 13272 1 2024-04-22T23:46:15.799 INFO:tasks.workunit.client.1.smithi044.stdout:7/671: creat dd/d55/d5e/d60/ffa x:0 0 0 2024-04-22T23:46:15.799 INFO:tasks.workunit.client.1.smithi044.stdout:7/672: write dd/d1f/d24/fcc [505361,39079] 0 2024-04-22T23:46:15.802 INFO:tasks.workunit.client.1.smithi044.stdout:3/684: mkdir d9/dc8/ddd 0 2024-04-22T23:46:15.805 INFO:tasks.workunit.client.1.smithi044.stdout:1/675: rename d1/d8/d20/d45/d5d/fa3 to d1/d8/d1c/d23/d3e/dde/ff7 0 2024-04-22T23:46:15.807 INFO:tasks.workunit.client.1.smithi044.stdout:8/722: dread - db/d14/d72/f8f zero size 2024-04-22T23:46:15.807 INFO:tasks.workunit.client.1.smithi044.stdout:4/699: mknod d5/d31/d44/d7e/ce5 0 2024-04-22T23:46:15.813 INFO:tasks.workunit.client.1.smithi044.stdout:2/567: creat d8/d27/fc8 x:0 0 0 2024-04-22T23:46:15.821 INFO:tasks.workunit.client.1.smithi044.stdout:8/723: symlink db/d68/d8b/dca/dd6/l104 0 2024-04-22T23:46:15.827 INFO:tasks.workunit.client.1.smithi044.stdout:2/568: unlink d8/d19/d1c/f24 0 2024-04-22T23:46:15.831 INFO:tasks.workunit.client.1.smithi044.stdout:6/548: write d5/d3f/d4c/d40/f78 [529400,59395] 0 2024-04-22T23:46:15.832 INFO:tasks.workunit.client.1.smithi044.stdout:3/685: mknod d9/cde 0 2024-04-22T23:46:15.833 INFO:tasks.workunit.client.1.smithi044.stdout:1/676: symlink d1/d8/d1c/d7b/d80/d4b/d9c/da8/lf8 0 2024-04-22T23:46:15.834 INFO:tasks.workunit.client.1.smithi044.stdout:1/677: stat d1/d8/d1c/d7b/f66 0 2024-04-22T23:46:15.836 INFO:tasks.workunit.client.1.smithi044.stdout:9/521: link d2/d6/d4d/f5e d2/dc/d1d/fad 0 2024-04-22T23:46:15.837 INFO:tasks.workunit.client.1.smithi044.stdout:1/678: read d1/d8/d1c/d7b/fd9 [2686886,99616] 0 2024-04-22T23:46:15.843 INFO:tasks.workunit.client.1.smithi044.stdout:8/724: dread f5 [4194304,4194304] 0 2024-04-22T23:46:15.845 INFO:tasks.workunit.client.1.smithi044.stdout:8/725: readlink db/d14/l6f 0 2024-04-22T23:46:15.845 INFO:tasks.workunit.client.1.smithi044.stdout:7/673: link dd/l16 dd/df7/db8/lfb 0 2024-04-22T23:46:15.854 INFO:tasks.workunit.client.1.smithi044.stdout:3/686: unlink d9/db/d15/d54/d3c/f6c 0 2024-04-22T23:46:15.856 INFO:tasks.workunit.client.1.smithi044.stdout:5/724: sync 2024-04-22T23:46:15.865 INFO:tasks.workunit.client.1.smithi044.stdout:9/522: creat d2/d6/d42/daa/fae x:0 0 0 2024-04-22T23:46:15.867 INFO:tasks.workunit.client.1.smithi044.stdout:9/523: dread - d2/fa3 zero size 2024-04-22T23:46:15.867 INFO:tasks.workunit.client.1.smithi044.stdout:9/524: readlink d2/d6/l8d 0 2024-04-22T23:46:15.870 INFO:tasks.workunit.client.1.smithi044.stdout:7/674: unlink dd/d1a/c4d 0 2024-04-22T23:46:15.871 INFO:tasks.workunit.client.1.smithi044.stdout:5/725: fsync d15/d16/d1d/d22/f97 0 2024-04-22T23:46:15.872 INFO:tasks.workunit.client.1.smithi044.stdout:5/726: dread - d15/d5d/db5/dc0/fd0 zero size 2024-04-22T23:46:15.874 INFO:tasks.workunit.client.1.smithi044.stdout:9/525: mknod d2/dc/d1d/d34/d38/d4a/d72/caf 0 2024-04-22T23:46:15.878 INFO:tasks.workunit.client.1.smithi044.stdout:8/726: rename db/ddd/df2/ld5 to db/d52/d9c/l105 0 2024-04-22T23:46:15.879 INFO:tasks.workunit.client.1.smithi044.stdout:5/727: symlink d15/d16/d1d/d22/d81/lef 0 2024-04-22T23:46:15.882 INFO:tasks.workunit.client.1.smithi044.stdout:5/728: unlink d15/d16/d1d/d22/f2d 0 2024-04-22T23:46:15.883 INFO:tasks.workunit.client.1.smithi044.stdout:9/526: fsync d2/dc/d94/f30 0 2024-04-22T23:46:15.884 INFO:tasks.workunit.client.1.smithi044.stdout:9/527: readlink d2/d6/l21 0 2024-04-22T23:46:15.889 INFO:tasks.workunit.client.1.smithi044.stdout:9/528: creat d2/dc/fb0 x:0 0 0 2024-04-22T23:46:15.892 INFO:tasks.workunit.client.1.smithi044.stdout:4/700: write d5/d31/d24/d69/d90/fdd [353076,93626] 0 2024-04-22T23:46:15.895 INFO:tasks.workunit.client.1.smithi044.stdout:9/529: rename d2/dc/d1d/d34/f84 to d2/dc/d1d/d34/d38/d4a/d72/fb1 0 2024-04-22T23:46:15.896 INFO:tasks.workunit.client.1.smithi044.stdout:9/530: chown d2/d6/l2f 2380581 1 2024-04-22T23:46:15.897 INFO:tasks.workunit.client.1.smithi044.stdout:9/531: chown d2/dc/d1d/c6c 7157023 1 2024-04-22T23:46:15.897 INFO:tasks.workunit.client.1.smithi044.stdout:0/794: write d0/f16 [3509300,116745] 0 2024-04-22T23:46:15.900 INFO:tasks.workunit.client.1.smithi044.stdout:9/532: write d2/dc/d1d/d34/f92 [120533,42361] 0 2024-04-22T23:46:15.906 INFO:tasks.workunit.client.1.smithi044.stdout:9/533: fdatasync d2/dc/d94/f1f 0 2024-04-22T23:46:15.908 INFO:tasks.workunit.client.1.smithi044.stdout:0/795: mkdir d0/d2/d4b/dad/dc8/d10e/d11b 0 2024-04-22T23:46:15.908 INFO:tasks.workunit.client.1.smithi044.stdout:2/569: write d8/d19/d5d/f83 [415620,123877] 0 2024-04-22T23:46:15.910 INFO:tasks.workunit.client.1.smithi044.stdout:0/796: chown d0/d2/dd/d3e/d40/d43/f66 188573645 1 2024-04-22T23:46:15.911 INFO:tasks.workunit.client.1.smithi044.stdout:4/701: dread d5/d31/d24/f72 [0,4194304] 0 2024-04-22T23:46:15.914 INFO:tasks.workunit.client.1.smithi044.stdout:6/549: truncate d5/dd/f1b 7255783 0 2024-04-22T23:46:15.916 INFO:tasks.workunit.client.1.smithi044.stdout:0/797: rmdir d0/d4c/dbd 39 2024-04-22T23:46:15.916 INFO:tasks.workunit.client.1.smithi044.stdout:0/798: fdatasync d0/d2e/f53 0 2024-04-22T23:46:15.920 INFO:tasks.workunit.client.1.smithi044.stdout:4/702: rename d5/d42/f61 to d5/db/d16/d4d/dad/fe6 0 2024-04-22T23:46:15.920 INFO:tasks.workunit.client.1.smithi044.stdout:3/687: write d9/db/d9f/f68 [81742,54604] 0 2024-04-22T23:46:15.921 INFO:tasks.workunit.client.1.smithi044.stdout:9/534: symlink d2/d9/lb2 0 2024-04-22T23:46:15.924 INFO:tasks.workunit.client.1.smithi044.stdout:4/703: write d5/d31/fce [673579,97140] 0 2024-04-22T23:46:15.924 INFO:tasks.workunit.client.1.smithi044.stdout:9/535: fdatasync d2/d6/f43 0 2024-04-22T23:46:15.925 INFO:tasks.workunit.client.1.smithi044.stdout:6/550: fdatasync d5/d3f/d4c/d4f/f59 0 2024-04-22T23:46:15.925 INFO:tasks.workunit.client.1.smithi044.stdout:1/679: dwrite d1/d8/d1c/d7b/d80/fe4 [0,4194304] 0 2024-04-22T23:46:15.927 INFO:tasks.workunit.client.1.smithi044.stdout:7/675: write dd/d1f/f66 [4237120,9298] 0 2024-04-22T23:46:15.929 INFO:tasks.workunit.client.1.smithi044.stdout:4/704: truncate d5/d52/f95 1741238 0 2024-04-22T23:46:15.937 INFO:tasks.workunit.client.1.smithi044.stdout:6/551: dwrite d5/dd/f68 [0,4194304] 0 2024-04-22T23:46:15.942 INFO:tasks.workunit.client.1.smithi044.stdout:1/680: dwrite d1/d8/d1c/f5b [8388608,4194304] 0 2024-04-22T23:46:15.944 INFO:tasks.workunit.client.1.smithi044.stdout:3/688: creat d9/d14/d7d/fdf x:0 0 0 2024-04-22T23:46:15.955 INFO:tasks.workunit.client.1.smithi044.stdout:7/676: mkdir dd/df7/db8/dfc 0 2024-04-22T23:46:15.957 INFO:tasks.workunit.client.1.smithi044.stdout:9/536: dwrite d2/dc/f87 [0,4194304] 0 2024-04-22T23:46:15.963 INFO:tasks.workunit.client.1.smithi044.stdout:4/705: mknod d5/d31/d24/d38/d84/ce7 0 2024-04-22T23:46:15.966 INFO:tasks.workunit.client.1.smithi044.stdout:7/677: dwrite dd/df7/d9c/fe1 [0,4194304] 0 2024-04-22T23:46:15.968 INFO:tasks.workunit.client.1.smithi044.stdout:0/799: creat d0/d2e/d33/f11c x:0 0 0 2024-04-22T23:46:15.968 INFO:tasks.workunit.client.1.smithi044.stdout:9/537: dread d2/dc/d1d/d34/f92 [0,4194304] 0 2024-04-22T23:46:15.969 INFO:tasks.workunit.client.1.smithi044.stdout:6/552: creat d5/d3f/d4c/d4f/fad x:0 0 0 2024-04-22T23:46:15.971 INFO:tasks.workunit.client.1.smithi044.stdout:2/570: truncate d8/d19/d36/d46/d63/d65/f85 2478967 0 2024-04-22T23:46:15.971 INFO:tasks.workunit.client.1.smithi044.stdout:0/800: stat d0/d4c/d59/d5a/c97 0 2024-04-22T23:46:15.972 INFO:tasks.workunit.client.1.smithi044.stdout:1/681: mknod d1/d8/d1c/d74/cf9 0 2024-04-22T23:46:15.977 INFO:tasks.workunit.client.1.smithi044.stdout:4/706: dwrite d5/d31/d24/d69/fbc [0,4194304] 0 2024-04-22T23:46:15.984 INFO:tasks.workunit.client.1.smithi044.stdout:1/682: truncate d1/d8/d20/d45/d75/df2/d99/fc4 4566835 0 2024-04-22T23:46:15.987 INFO:tasks.workunit.client.1.smithi044.stdout:4/707: dread d5/db/f8f [0,4194304] 0 2024-04-22T23:46:15.988 INFO:tasks.workunit.client.1.smithi044.stdout:3/689: creat d9/d14/dd2/fe0 x:0 0 0 2024-04-22T23:46:15.992 INFO:tasks.workunit.client.1.smithi044.stdout:2/571: mknod d8/d19/d36/d44/cc9 0 2024-04-22T23:46:15.993 INFO:tasks.workunit.client.1.smithi044.stdout:0/801: dwrite d0/d60/fb6 [0,4194304] 0 2024-04-22T23:46:16.037 INFO:tasks.workunit.client.1.smithi044.stdout:9/538: unlink d2/dc/c9e 0 2024-04-22T23:46:16.042 INFO:tasks.workunit.client.1.smithi044.stdout:1/683: symlink d1/d8/d1c/d7b/d80/de3/lfa 0 2024-04-22T23:46:16.047 INFO:tasks.workunit.client.1.smithi044.stdout:7/678: rename dd/d55/d5e/fc1 to dd/ffd 0 2024-04-22T23:46:16.052 INFO:tasks.workunit.client.1.smithi044.stdout:0/802: symlink d0/l11d 0 2024-04-22T23:46:16.055 INFO:tasks.workunit.client.1.smithi044.stdout:7/679: rmdir dd/df7 39 2024-04-22T23:46:16.058 INFO:tasks.workunit.client.1.smithi044.stdout:2/572: rename d8/d19/f4c to d8/d19/d53/fca 0 2024-04-22T23:46:16.060 INFO:tasks.workunit.client.1.smithi044.stdout:6/553: getdents d5/d3f 0 2024-04-22T23:46:16.061 INFO:tasks.workunit.client.1.smithi044.stdout:6/554: write d5/d30/d73/f87 [899077,15096] 0 2024-04-22T23:46:16.061 INFO:tasks.workunit.client.1.smithi044.stdout:9/539: mknod d2/dc/d94/cb3 0 2024-04-22T23:46:16.063 INFO:tasks.workunit.client.1.smithi044.stdout:6/555: chown d5/d3f/d4c/d23/d3d/d64/c85 1352772 1 2024-04-22T23:46:16.063 INFO:tasks.workunit.client.1.smithi044.stdout:9/540: write d2/dc/d1d/d3f/f4c [1674616,15240] 0 2024-04-22T23:46:16.064 INFO:tasks.workunit.client.1.smithi044.stdout:0/803: symlink d0/d2/d4b/dad/l11e 0 2024-04-22T23:46:16.066 INFO:tasks.workunit.client.1.smithi044.stdout:6/556: truncate d5/d3f/d4c/d40/d53/f8e 456461 0 2024-04-22T23:46:16.066 INFO:tasks.workunit.client.1.smithi044.stdout:7/680: mknod dd/d55/d5e/d60/d68/cfe 0 2024-04-22T23:46:16.067 INFO:tasks.workunit.client.1.smithi044.stdout:0/804: truncate d0/d2e/d33/f85 810601 0 2024-04-22T23:46:16.067 INFO:tasks.workunit.client.1.smithi044.stdout:7/681: chown dd/d1a 4 1 2024-04-22T23:46:16.071 INFO:tasks.workunit.client.1.smithi044.stdout:1/684: rename d1/d14 to d1/d8/d20/d45/d75/dfb 0 2024-04-22T23:46:16.073 INFO:tasks.workunit.client.1.smithi044.stdout:2/573: creat d8/d19/d1c/fcb x:0 0 0 2024-04-22T23:46:16.074 INFO:tasks.workunit.client.1.smithi044.stdout:8/727: sync 2024-04-22T23:46:16.075 INFO:tasks.workunit.client.1.smithi044.stdout:3/690: getdents d9/d12 0 2024-04-22T23:46:16.075 INFO:tasks.workunit.client.1.smithi044.stdout:0/805: dwrite d0/d2/dd/d3e/d40/d43/dde/ff3 [0,4194304] 0 2024-04-22T23:46:16.077 INFO:tasks.workunit.client.1.smithi044.stdout:9/541: creat d2/dc/d1d/d3f/fb4 x:0 0 0 2024-04-22T23:46:16.078 INFO:tasks.workunit.client.1.smithi044.stdout:0/806: write d0/d60/de6/d109/f113 [14747,83286] 0 2024-04-22T23:46:16.079 INFO:tasks.workunit.client.1.smithi044.stdout:6/557: creat d5/d3f/d50/d79/fae x:0 0 0 2024-04-22T23:46:16.085 INFO:tasks.workunit.client.1.smithi044.stdout:8/728: stat c2 0 2024-04-22T23:46:16.086 INFO:tasks.workunit.client.1.smithi044.stdout:8/729: stat db/d14/f63 0 2024-04-22T23:46:16.089 INFO:tasks.workunit.client.1.smithi044.stdout:3/691: truncate d9/db/d15/d25/d4c/f75 787396 0 2024-04-22T23:46:16.090 INFO:tasks.workunit.client.1.smithi044.stdout:3/692: rename d9/db to d9/db/d9f/d38/d74/dd5/de1 22 2024-04-22T23:46:16.091 INFO:tasks.workunit.client.1.smithi044.stdout:0/807: creat d0/d2/dd/d3e/d40/d43/d7b/f11f x:0 0 0 2024-04-22T23:46:16.097 INFO:tasks.workunit.client.1.smithi044.stdout:9/542: truncate d2/dc/d1d/fad 1036679 0 2024-04-22T23:46:16.097 INFO:tasks.workunit.client.1.smithi044.stdout:6/558: mkdir d5/d3f/d4c/d40/d74/daf 0 2024-04-22T23:46:16.098 INFO:tasks.workunit.client.1.smithi044.stdout:6/559: write d5/d30/f80 [3206710,116706] 0 2024-04-22T23:46:16.099 INFO:tasks.workunit.client.1.smithi044.stdout:4/708: dwrite d5/d31/d24/f3d [0,4194304] 0 2024-04-22T23:46:16.102 INFO:tasks.workunit.client.1.smithi044.stdout:3/693: creat d9/db/d15/d25/fe2 x:0 0 0 2024-04-22T23:46:16.104 INFO:tasks.workunit.client.1.smithi044.stdout:3/694: dread - d9/d14/d7d/fdf zero size 2024-04-22T23:46:16.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/543: truncate d2/dc/d1d/d34/f92 1001321 0 2024-04-22T23:46:16.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/544: chown d2/dc/d94 240161649 1 2024-04-22T23:46:16.116 INFO:tasks.workunit.client.1.smithi044.stdout:4/709: mknod d5/d31/d24/da6/ce8 0 2024-04-22T23:46:16.118 INFO:tasks.workunit.client.1.smithi044.stdout:7/682: write dd/d1a/db7/dc8/f8f [208948,32101] 0 2024-04-22T23:46:16.119 INFO:tasks.workunit.client.1.smithi044.stdout:8/730: creat db/f106 x:0 0 0 2024-04-22T23:46:16.121 INFO:tasks.workunit.client.1.smithi044.stdout:1/685: write d1/d8/d20/d45/d75/df2/f94 [260432,127021] 0 2024-04-22T23:46:16.125 INFO:tasks.workunit.client.1.smithi044.stdout:0/808: creat d0/d2/dd/d3e/d40/d43/f120 x:0 0 0 2024-04-22T23:46:16.126 INFO:tasks.workunit.client.1.smithi044.stdout:4/710: symlink d5/d31/d24/d38/d84/le9 0 2024-04-22T23:46:16.129 INFO:tasks.workunit.client.1.smithi044.stdout:1/686: read d1/d8/d1c/d7b/fb6 [872353,94984] 0 2024-04-22T23:46:16.130 INFO:tasks.workunit.client.1.smithi044.stdout:7/683: dwrite dd/d1f/d24/fbf [0,4194304] 0 2024-04-22T23:46:16.131 INFO:tasks.workunit.client.1.smithi044.stdout:8/731: chown db/d14/dc7/dc9/f101 48 1 2024-04-22T23:46:16.133 INFO:tasks.workunit.client.1.smithi044.stdout:6/560: rmdir d5/dd/d67/d84 0 2024-04-22T23:46:16.137 INFO:tasks.workunit.client.1.smithi044.stdout:2/574: dwrite d8/d19/f30 [0,4194304] 0 2024-04-22T23:46:16.144 INFO:tasks.workunit.client.1.smithi044.stdout:2/575: dread d8/d19/d36/f3d [0,4194304] 0 2024-04-22T23:46:16.145 INFO:tasks.workunit.client.1.smithi044.stdout:3/695: creat d9/db/d33/d5d/fe3 x:0 0 0 2024-04-22T23:46:16.146 INFO:tasks.workunit.client.1.smithi044.stdout:3/696: write d9/f2a [4265578,1705] 0 2024-04-22T23:46:16.147 INFO:tasks.workunit.client.1.smithi044.stdout:1/687: unlink d1/d8/d20/d47/d4a/f54 0 2024-04-22T23:46:16.149 INFO:tasks.workunit.client.1.smithi044.stdout:1/688: read - d1/d8/d20/d45/d75/d77/dbf/ddb/fef zero size 2024-04-22T23:46:16.150 INFO:tasks.workunit.client.1.smithi044.stdout:4/711: rename d5/d31/d24/d38/d84/fa4 to d5/d31/d44/d20/d93/fea 0 2024-04-22T23:46:16.151 INFO:tasks.workunit.client.1.smithi044.stdout:4/712: dread - d5/d31/d44/d20/fe3 zero size 2024-04-22T23:46:16.151 INFO:tasks.workunit.client.1.smithi044.stdout:1/689: write d1/d8/d20/d45/d75/d77/dbf/ddb/fef [94891,129046] 0 2024-04-22T23:46:16.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/684: fsync dd/d27/f48 0 2024-04-22T23:46:16.160 INFO:tasks.workunit.client.1.smithi044.stdout:9/545: truncate d2/dc/d1d/d34/f49 8342184 0 2024-04-22T23:46:16.160 INFO:tasks.workunit.client.1.smithi044.stdout:4/713: dwrite d5/d31/d24/f2f [4194304,4194304] 0 2024-04-22T23:46:16.167 INFO:tasks.workunit.client.1.smithi044.stdout:0/809: unlink d0/d4c/dbd/fee 0 2024-04-22T23:46:16.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/810: fdatasync d0/d2e/d33/f11c 0 2024-04-22T23:46:16.169 INFO:tasks.workunit.client.1.smithi044.stdout:2/576: creat d8/d19/d36/d93/fcc x:0 0 0 2024-04-22T23:46:16.170 INFO:tasks.workunit.client.1.smithi044.stdout:0/811: chown d0/d2e/l6f 32740039 1 2024-04-22T23:46:16.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/697: mknod d9/db/d15/d25/ce4 0 2024-04-22T23:46:16.174 INFO:tasks.workunit.client.1.smithi044.stdout:3/698: chown d9/db/d15/d25/d67/d6b/lc3 281562 1 2024-04-22T23:46:16.174 INFO:tasks.workunit.client.1.smithi044.stdout:3/699: chown d9/db/fa2 2 1 2024-04-22T23:46:16.176 INFO:tasks.workunit.client.1.smithi044.stdout:3/700: chown d9/db/d9f/fd7 13 1 2024-04-22T23:46:16.178 INFO:tasks.workunit.client.1.smithi044.stdout:9/546: dread d2/f4 [0,4194304] 0 2024-04-22T23:46:16.179 INFO:tasks.workunit.client.1.smithi044.stdout:9/547: readlink d2/dc/d1d/d34/d38/d5c/d69/l71 0 2024-04-22T23:46:16.182 INFO:tasks.workunit.client.1.smithi044.stdout:6/561: symlink d5/d3f/d4c/lb0 0 2024-04-22T23:46:16.187 INFO:tasks.workunit.client.1.smithi044.stdout:4/714: mkdir d5/db/d16/d4d/dad/deb 0 2024-04-22T23:46:16.189 INFO:tasks.workunit.client.1.smithi044.stdout:0/812: dread d0/d2/dd/d3e/d40/d43/d7b/f5c [0,4194304] 0 2024-04-22T23:46:16.191 INFO:tasks.workunit.client.1.smithi044.stdout:3/701: rename d9/d12/f60 to d9/db/d15/d54/d3c/d95/fe5 0 2024-04-22T23:46:16.193 INFO:tasks.workunit.client.1.smithi044.stdout:9/548: creat d2/dc/d1d/d34/d38/d5c/d69/d75/fb5 x:0 0 0 2024-04-22T23:46:16.195 INFO:tasks.workunit.client.1.smithi044.stdout:4/715: dwrite d5/d31/d44/d7e/f98 [0,4194304] 0 2024-04-22T23:46:16.198 INFO:tasks.workunit.client.1.smithi044.stdout:6/562: unlink d5/d30/fa4 0 2024-04-22T23:46:16.199 INFO:tasks.workunit.client.1.smithi044.stdout:7/685: getdents dd/d99/deb 0 2024-04-22T23:46:16.201 INFO:tasks.workunit.client.1.smithi044.stdout:0/813: creat d0/d2/dd/d3e/d40/d43/d7b/f121 x:0 0 0 2024-04-22T23:46:16.203 INFO:tasks.workunit.client.1.smithi044.stdout:3/702: chown d9/db/d15/d54/d3c/d95/cc9 1677709 1 2024-04-22T23:46:16.204 INFO:tasks.workunit.client.1.smithi044.stdout:0/814: fdatasync d0/d2e/dbc/fdd 0 2024-04-22T23:46:16.205 INFO:tasks.workunit.client.1.smithi044.stdout:9/549: mknod d2/dc/d1d/d34/d38/cb6 0 2024-04-22T23:46:16.207 INFO:tasks.workunit.client.1.smithi044.stdout:4/716: unlink d5/db/d3c/c43 0 2024-04-22T23:46:16.208 INFO:tasks.workunit.client.1.smithi044.stdout:3/703: write d9/db/d15/d25/d5b/f7e [3370706,60269] 0 2024-04-22T23:46:16.209 INFO:tasks.workunit.client.1.smithi044.stdout:1/690: link d1/d8/d20/d45/d75/df2/d99/lb4 d1/d8/d20/d45/d75/d77/dbf/dc2/lfc 0 2024-04-22T23:46:16.211 INFO:tasks.workunit.client.1.smithi044.stdout:7/686: dwrite dd/d99/dbb/ff6 [0,4194304] 0 2024-04-22T23:46:16.213 INFO:tasks.workunit.client.1.smithi044.stdout:4/717: truncate d5/d31/d24/d38/d84/fcb 573151 0 2024-04-22T23:46:16.217 INFO:tasks.workunit.client.1.smithi044.stdout:4/718: write d5/d31/d24/d38/d84/fe4 [455467,36951] 0 2024-04-22T23:46:16.220 INFO:tasks.workunit.client.1.smithi044.stdout:6/563: rename d5/dd/l1a to d5/d3f/d50/d79/lb1 0 2024-04-22T23:46:16.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/815: link d0/d2/fa d0/d2e/dbc/f122 0 2024-04-22T23:46:16.227 INFO:tasks.workunit.client.1.smithi044.stdout:0/816: write d0/d2/dd/f2a [1397394,64724] 0 2024-04-22T23:46:16.228 INFO:tasks.workunit.client.1.smithi044.stdout:7/687: chown dd/df7/fa9 2511171 1 2024-04-22T23:46:16.231 INFO:tasks.workunit.client.1.smithi044.stdout:3/704: dread f8 [0,4194304] 0 2024-04-22T23:46:16.233 INFO:tasks.workunit.client.1.smithi044.stdout:4/719: symlink d5/d31/d24/d69/lec 0 2024-04-22T23:46:16.234 INFO:tasks.workunit.client.1.smithi044.stdout:9/550: dread d2/f3d [0,4194304] 0 2024-04-22T23:46:16.234 INFO:tasks.workunit.client.1.smithi044.stdout:9/551: chown d2/dc/d94 135 1 2024-04-22T23:46:16.237 INFO:tasks.workunit.client.1.smithi044.stdout:6/564: mknod d5/d30/d73/cb2 0 2024-04-22T23:46:16.238 INFO:tasks.workunit.client.1.smithi044.stdout:6/565: chown d5/d3f/laa 2 1 2024-04-22T23:46:16.242 INFO:tasks.workunit.client.1.smithi044.stdout:3/705: creat d9/db/d15/d54/d3c/fe6 x:0 0 0 2024-04-22T23:46:16.243 INFO:tasks.workunit.client.1.smithi044.stdout:4/720: fsync d5/d31/d44/d20/f23 0 2024-04-22T23:46:16.247 INFO:tasks.workunit.client.1.smithi044.stdout:0/817: mkdir d0/d4c/d123 0 2024-04-22T23:46:16.248 INFO:tasks.workunit.client.1.smithi044.stdout:0/818: dread - d0/d2e/dbf/fec zero size 2024-04-22T23:46:16.251 INFO:tasks.workunit.client.1.smithi044.stdout:9/552: dread d2/d6/d4d/f5e [0,4194304] 0 2024-04-22T23:46:16.257 INFO:tasks.workunit.client.1.smithi044.stdout:1/691: rename d1/d8/d20/d45/d75/d77 to d1/d8/d1c/d7b/d80/dfd 0 2024-04-22T23:46:16.261 INFO:tasks.workunit.client.1.smithi044.stdout:7/688: write dd/d1f/f6a [192785,14541] 0 2024-04-22T23:46:16.264 INFO:tasks.workunit.client.1.smithi044.stdout:0/819: creat d0/d4c/d59/f124 x:0 0 0 2024-04-22T23:46:16.266 INFO:tasks.workunit.client.1.smithi044.stdout:9/553: mkdir d2/dc/d1d/d34/d38/d4a/db7 0 2024-04-22T23:46:16.269 INFO:tasks.workunit.client.1.smithi044.stdout:4/721: creat d5/db/d16/d4d/dad/dc5/dd9/fed x:0 0 0 2024-04-22T23:46:16.272 INFO:tasks.workunit.client.1.smithi044.stdout:3/706: dwrite d9/db/d15/d2e/f32 [0,4194304] 0 2024-04-22T23:46:16.273 INFO:tasks.workunit.client.1.smithi044.stdout:3/707: truncate d9/d12/f78 734186 0 2024-04-22T23:46:16.275 INFO:tasks.workunit.client.1.smithi044.stdout:0/820: mkdir d0/d2/d4b/dad/d125 0 2024-04-22T23:46:16.278 INFO:tasks.workunit.client.1.smithi044.stdout:9/554: creat d2/dc/d1d/d34/d38/d5c/d69/fb8 x:0 0 0 2024-04-22T23:46:16.282 INFO:tasks.workunit.client.1.smithi044.stdout:1/692: link d1/d8/d20/d45/d75/df2/ff5 d1/d8/d20/d45/d75/dfb/d95/ffe 0 2024-04-22T23:46:16.285 INFO:tasks.workunit.client.1.smithi044.stdout:1/693: truncate d1/d8/d1c/d7b/d80/dfd/f89 1697075 0 2024-04-22T23:46:16.287 INFO:tasks.workunit.client.1.smithi044.stdout:4/722: symlink d5/d31/d44/d7e/lee 0 2024-04-22T23:46:16.287 INFO:tasks.workunit.client.1.smithi044.stdout:6/566: getdents d5/d3f/d4c/d23/d3d/d64 0 2024-04-22T23:46:16.290 INFO:tasks.workunit.client.1.smithi044.stdout:7/689: creat dd/d55/d5e/d60/d68/d94/dab/dd5/fff x:0 0 0 2024-04-22T23:46:16.291 INFO:tasks.workunit.client.1.smithi044.stdout:7/690: dread - dd/d55/d5e/fe4 zero size 2024-04-22T23:46:16.294 INFO:tasks.workunit.client.1.smithi044.stdout:5/729: sync 2024-04-22T23:46:16.296 INFO:tasks.workunit.client.1.smithi044.stdout:1/694: rmdir d1/d8/d1c/d7b/d80/d46 39 2024-04-22T23:46:16.297 INFO:tasks.workunit.client.1.smithi044.stdout:3/708: dwrite d9/d14/d7d/fb4 [0,4194304] 0 2024-04-22T23:46:16.302 INFO:tasks.workunit.client.1.smithi044.stdout:7/691: mknod dd/d79/c100 0 2024-04-22T23:46:16.310 INFO:tasks.workunit.client.1.smithi044.stdout:6/567: symlink d5/d3f/d4c/d40/d74/daf/lb3 0 2024-04-22T23:46:16.312 INFO:tasks.workunit.client.1.smithi044.stdout:7/692: write dd/d27/fb5 [2933204,91428] 0 2024-04-22T23:46:16.313 INFO:tasks.workunit.client.1.smithi044.stdout:0/821: write d0/d2/dd/d3e/d40/d43/d7b/f26 [3514395,54107] 0 2024-04-22T23:46:16.314 INFO:tasks.workunit.client.1.smithi044.stdout:0/822: chown d0/d2/d4b/f74 122200 1 2024-04-22T23:46:16.317 INFO:tasks.workunit.client.1.smithi044.stdout:5/730: creat d15/d16/d1d/da2/ddc/ff0 x:0 0 0 2024-04-22T23:46:16.318 INFO:tasks.workunit.client.1.smithi044.stdout:5/731: dread - d15/d5d/db5/dc0/d38/d9f/fc4 zero size 2024-04-22T23:46:16.319 INFO:tasks.workunit.client.1.smithi044.stdout:3/709: mknod d9/db/d9f/ce7 0 2024-04-22T23:46:16.319 INFO:tasks.workunit.client.1.smithi044.stdout:4/723: creat d5/d31/fef x:0 0 0 2024-04-22T23:46:16.320 INFO:tasks.workunit.client.1.smithi044.stdout:5/732: fdatasync d15/d16/d1d/d22/d4f/dc9/fbd 0 2024-04-22T23:46:16.323 INFO:tasks.workunit.client.1.smithi044.stdout:5/733: read d15/d5d/d64/d6f/fbe [4074487,23073] 0 2024-04-22T23:46:16.323 INFO:tasks.workunit.client.1.smithi044.stdout:6/568: unlink d5/d30/d73/f87 0 2024-04-22T23:46:16.327 INFO:tasks.workunit.client.1.smithi044.stdout:0/823: mkdir d0/d2/dd/d3e/d40/d43/de4/d126 0 2024-04-22T23:46:16.330 INFO:tasks.workunit.client.1.smithi044.stdout:4/724: mkdir d5/d31/d24/d38/df0 0 2024-04-22T23:46:16.331 INFO:tasks.workunit.client.1.smithi044.stdout:4/725: readlink d5/d31/d24/d38/d6a/l7a 0 2024-04-22T23:46:16.336 INFO:tasks.workunit.client.1.smithi044.stdout:3/710: dread d9/db/f13 [0,4194304] 0 2024-04-22T23:46:16.337 INFO:tasks.workunit.client.1.smithi044.stdout:4/726: dread d5/d31/d24/f72 [0,4194304] 0 2024-04-22T23:46:16.341 INFO:tasks.workunit.client.1.smithi044.stdout:1/695: getdents d1/d8/d20 0 2024-04-22T23:46:16.344 INFO:tasks.workunit.client.1.smithi044.stdout:6/569: dread d5/d3f/f22 [4194304,4194304] 0 2024-04-22T23:46:16.350 INFO:tasks.workunit.client.1.smithi044.stdout:4/727: read d5/d31/d24/fde [1847768,58340] 0 2024-04-22T23:46:16.352 INFO:tasks.workunit.client.1.smithi044.stdout:3/711: symlink d9/db/d33/d5d/d64/le8 0 2024-04-22T23:46:16.353 INFO:tasks.workunit.client.1.smithi044.stdout:7/693: dwrite dd/d1a/d61/f96 [0,4194304] 0 2024-04-22T23:46:16.355 INFO:tasks.workunit.client.1.smithi044.stdout:5/734: rename d15/d16/d1d/la7 to d15/d5d/d64/d6f/lf1 0 2024-04-22T23:46:16.359 INFO:tasks.workunit.client.1.smithi044.stdout:0/824: mkdir d0/d2/dd/d3e/d40/d43/de4/d126/d127 0 2024-04-22T23:46:16.362 INFO:tasks.workunit.client.1.smithi044.stdout:7/694: stat dd/d98/lc5 0 2024-04-22T23:46:16.364 INFO:tasks.workunit.client.1.smithi044.stdout:1/696: rename d1/d8/d20/d45/d75/dfb/d95/ffe to d1/d8/d1c/d7b/d80/dfd/dad/fff 0 2024-04-22T23:46:16.367 INFO:tasks.workunit.client.1.smithi044.stdout:6/570: mkdir d5/d3f/d4c/db4 0 2024-04-22T23:46:16.373 INFO:tasks.workunit.client.1.smithi044.stdout:7/695: symlink dd/d1a/d61/l101 0 2024-04-22T23:46:16.375 INFO:tasks.workunit.client.1.smithi044.stdout:4/728: rename d5/d42/ca9 to d5/db/d3c/cf1 0 2024-04-22T23:46:16.377 INFO:tasks.workunit.client.1.smithi044.stdout:1/697: mknod d1/d8/d34/da1/dab/de9/c100 0 2024-04-22T23:46:16.378 INFO:tasks.workunit.client.1.smithi044.stdout:5/735: dread d15/d16/d1d/f51 [0,4194304] 0 2024-04-22T23:46:16.379 INFO:tasks.workunit.client.1.smithi044.stdout:6/571: creat d5/d3f/d50/fb5 x:0 0 0 2024-04-22T23:46:16.379 INFO:tasks.workunit.client.1.smithi044.stdout:5/736: read - d15/d16/d1d/d22/d4f/dc9/fbd zero size 2024-04-22T23:46:16.380 INFO:tasks.workunit.client.1.smithi044.stdout:5/737: fsync d15/d5d/d64/d6f/fbf 0 2024-04-22T23:46:16.384 INFO:tasks.workunit.client.1.smithi044.stdout:3/712: rename d9/db/f65 to d9/db/d33/d5d/d64/dbe/fe9 0 2024-04-22T23:46:16.386 INFO:tasks.workunit.client.1.smithi044.stdout:4/729: unlink d5/d31/d24/d69/fcd 0 2024-04-22T23:46:16.387 INFO:tasks.workunit.client.1.smithi044.stdout:0/825: write d0/d2/dd/d3e/d40/fb0 [919449,123260] 0 2024-04-22T23:46:16.388 INFO:tasks.workunit.client.1.smithi044.stdout:1/698: creat d1/d8/d20/dae/deb/f101 x:0 0 0 2024-04-22T23:46:16.391 INFO:tasks.workunit.client.1.smithi044.stdout:6/572: dread - d5/d3f/d4c/f72 zero size 2024-04-22T23:46:16.395 INFO:tasks.workunit.client.1.smithi044.stdout:7/696: write dd/ff [4846751,25189] 0 2024-04-22T23:46:16.395 INFO:tasks.workunit.client.1.smithi044.stdout:1/699: dwrite d1/d8/d1c/d7b/d80/dfd/dad/fb1 [0,4194304] 0 2024-04-22T23:46:16.397 INFO:tasks.workunit.client.1.smithi044.stdout:5/738: mkdir d15/d5d/d82/df2 0 2024-04-22T23:46:16.397 INFO:tasks.workunit.client.1.smithi044.stdout:1/700: chown d1/d8/d20/d45/d75/df2/cc8 591089022 1 2024-04-22T23:46:16.400 INFO:tasks.workunit.client.1.smithi044.stdout:5/739: dread - d15/d16/d1d/da2/ddc/ff0 zero size 2024-04-22T23:46:16.402 INFO:tasks.workunit.client.1.smithi044.stdout:5/740: chown d15/d16/d1d/d22/d2a/d3a/d43/fc6 19 1 2024-04-22T23:46:16.402 INFO:tasks.workunit.client.1.smithi044.stdout:0/826: rmdir d0/d2/dd/d3e/d40/d67/d7c 39 2024-04-22T23:46:16.407 INFO:tasks.workunit.client.1.smithi044.stdout:7/697: symlink dd/d1f/d24/l102 0 2024-04-22T23:46:16.408 INFO:tasks.workunit.client.1.smithi044.stdout:7/698: chown dd/d98/de8/df2 155125097 1 2024-04-22T23:46:16.410 INFO:tasks.workunit.client.1.smithi044.stdout:3/713: creat d9/dc8/ddd/fea x:0 0 0 2024-04-22T23:46:16.410 INFO:tasks.workunit.client.1.smithi044.stdout:0/827: dread d0/d4c/fb9 [0,4194304] 0 2024-04-22T23:46:16.410 INFO:tasks.workunit.client.1.smithi044.stdout:1/701: symlink d1/d8/d1c/d7b/d80/d4b/l102 0 2024-04-22T23:46:16.411 INFO:tasks.workunit.client.1.smithi044.stdout:3/714: readlink d9/db/d9f/dcd/ld3 0 2024-04-22T23:46:16.412 INFO:tasks.workunit.client.1.smithi044.stdout:3/715: chown d9/db/d15/d54/d3c/d95/lc7 18263691 1 2024-04-22T23:46:16.414 INFO:tasks.workunit.client.1.smithi044.stdout:4/730: fsync d5/d31/d44/d20/d93/fea 0 2024-04-22T23:46:16.421 INFO:tasks.workunit.client.1.smithi044.stdout:3/716: dwrite d9/db/d9f/d38/d74/fc4 [0,4194304] 0 2024-04-22T23:46:16.425 INFO:tasks.workunit.client.1.smithi044.stdout:7/699: mknod dd/d55/d5e/d60/d68/d94/c103 0 2024-04-22T23:46:16.426 INFO:tasks.workunit.client.1.smithi044.stdout:0/828: write d0/d60/de6/d109/fd8 [752308,90607] 0 2024-04-22T23:46:16.433 INFO:tasks.workunit.client.1.smithi044.stdout:4/731: rename d5/l6b to d5/d31/d44/dd2/lf2 0 2024-04-22T23:46:16.436 INFO:tasks.workunit.client.1.smithi044.stdout:6/573: creat d5/d3f/d4c/fb6 x:0 0 0 2024-04-22T23:46:16.437 INFO:tasks.workunit.client.1.smithi044.stdout:3/717: rmdir d9/db/d9f/dcd 39 2024-04-22T23:46:16.438 INFO:tasks.workunit.client.1.smithi044.stdout:5/741: write d15/d3c/f53 [4072537,79466] 0 2024-04-22T23:46:16.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:15 smithi031 ceph-mon[26721]: pgmap v11: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 47 MiB/s rd, 164 MiB/s wr, 379 op/s 2024-04-22T23:46:16.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:15 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:16.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:15 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:16.446 INFO:tasks.workunit.client.1.smithi044.stdout:4/732: mknod d5/db/d16/d4d/dad/cf3 0 2024-04-22T23:46:16.448 INFO:tasks.workunit.client.1.smithi044.stdout:6/574: mkdir d5/d3f/d4c/d40/d53/d56/db7 0 2024-04-22T23:46:16.451 INFO:tasks.workunit.client.1.smithi044.stdout:0/829: write d0/d2/fa [3041519,110873] 0 2024-04-22T23:46:16.452 INFO:tasks.workunit.client.1.smithi044.stdout:5/742: fsync d15/d5d/db5/dc0/f5a 0 2024-04-22T23:46:16.460 INFO:tasks.workunit.client.1.smithi044.stdout:1/702: link d1/d8/d1c/d74/cf9 d1/d8/d20/c103 0 2024-04-22T23:46:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:15 smithi044 ceph-mon[34463]: pgmap v11: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 47 MiB/s rd, 164 MiB/s wr, 379 op/s 2024-04-22T23:46:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:15 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:15 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:16.460 INFO:tasks.workunit.client.1.smithi044.stdout:5/743: dwrite d15/d5d/db5/dc0/f54 [8388608,4194304] 0 2024-04-22T23:46:16.466 INFO:tasks.workunit.client.1.smithi044.stdout:6/575: symlink d5/d3f/d4c/d40/d53/d56/lb8 0 2024-04-22T23:46:16.466 INFO:tasks.workunit.client.1.smithi044.stdout:6/576: chown d5/d3f/d4c/d23/d3d 27 1 2024-04-22T23:46:16.475 INFO:tasks.workunit.client.1.smithi044.stdout:8/732: sync 2024-04-22T23:46:16.476 INFO:tasks.workunit.client.1.smithi044.stdout:4/733: dwrite d5/db/d34/f78 [0,4194304] 0 2024-04-22T23:46:16.476 INFO:tasks.workunit.client.1.smithi044.stdout:1/703: fdatasync d1/d8/d1c/f8c 0 2024-04-22T23:46:16.478 INFO:tasks.workunit.client.1.smithi044.stdout:7/700: getdents dd/d99/dbb 0 2024-04-22T23:46:16.480 INFO:tasks.workunit.client.1.smithi044.stdout:6/577: mkdir d5/d3f/d4c/d40/db9 0 2024-04-22T23:46:16.480 INFO:tasks.workunit.client.1.smithi044.stdout:4/734: chown d5/db/d16/d4d/dad/dc5/dd9 11077995 1 2024-04-22T23:46:16.480 INFO:tasks.workunit.client.1.smithi044.stdout:0/830: mknod d0/d60/c128 0 2024-04-22T23:46:16.481 INFO:tasks.workunit.client.1.smithi044.stdout:4/735: fdatasync d5/d31/d24/d38/f91 0 2024-04-22T23:46:16.485 INFO:tasks.workunit.client.1.smithi044.stdout:0/831: fdatasync d0/d2/f1e 0 2024-04-22T23:46:16.489 INFO:tasks.workunit.client.1.smithi044.stdout:3/718: rename d9/d14/d7d/fb4 to d9/db/d15/d25/d5b/feb 0 2024-04-22T23:46:16.491 INFO:tasks.workunit.client.1.smithi044.stdout:5/744: symlink d15/d16/d1d/d22/d4f/d68/d76/lf3 0 2024-04-22T23:46:16.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/719: dread d9/d14/f2b [0,4194304] 0 2024-04-22T23:46:16.493 INFO:tasks.workunit.client.1.smithi044.stdout:1/704: creat d1/d8/d1c/d23/d3e/dde/f104 x:0 0 0 2024-04-22T23:46:16.494 INFO:tasks.workunit.client.1.smithi044.stdout:7/701: dwrite dd/d55/d5e/d60/ffa [0,4194304] 0 2024-04-22T23:46:16.497 INFO:tasks.workunit.client.1.smithi044.stdout:3/720: truncate d9/db/d15/d25/fb6 120428 0 2024-04-22T23:46:16.501 INFO:tasks.workunit.client.1.smithi044.stdout:7/702: truncate dd/d1f/f6a 347553 0 2024-04-22T23:46:16.501 INFO:tasks.workunit.client.1.smithi044.stdout:6/578: mknod d5/d3f/d4c/d40/d74/daf/cba 0 2024-04-22T23:46:16.504 INFO:tasks.workunit.client.1.smithi044.stdout:6/579: write d5/d3f/d4c/d40/d53/f8e [777998,32420] 0 2024-04-22T23:46:16.506 INFO:tasks.workunit.client.1.smithi044.stdout:0/832: mknod d0/d4c/d8b/c129 0 2024-04-22T23:46:16.511 INFO:tasks.workunit.client.1.smithi044.stdout:4/736: dread d5/d31/d44/d20/d22/f67 [0,4194304] 0 2024-04-22T23:46:16.518 INFO:tasks.workunit.client.1.smithi044.stdout:3/721: symlink d9/db/d15/d2e/d48/lec 0 2024-04-22T23:46:16.521 INFO:tasks.workunit.client.1.smithi044.stdout:1/705: write d1/d8/d1c/d23/d3e/f90 [130483,17801] 0 2024-04-22T23:46:16.523 INFO:tasks.workunit.client.1.smithi044.stdout:0/833: mkdir d0/d2/dd/d3e/d40/d43/d79/d12a 0 2024-04-22T23:46:16.524 INFO:tasks.workunit.client.1.smithi044.stdout:4/737: fsync d5/d31/d24/fde 0 2024-04-22T23:46:16.527 INFO:tasks.workunit.client.1.smithi044.stdout:3/722: mkdir d9/db/d15/d25/d67/d6b/ded 0 2024-04-22T23:46:16.528 INFO:tasks.workunit.client.1.smithi044.stdout:5/745: dwrite d15/d5d/d64/d6f/fbe [0,4194304] 0 2024-04-22T23:46:16.530 INFO:tasks.workunit.client.1.smithi044.stdout:7/703: creat dd/d99/deb/f104 x:0 0 0 2024-04-22T23:46:16.534 INFO:tasks.workunit.client.1.smithi044.stdout:6/580: creat d5/d3f/d4c/db4/fbb x:0 0 0 2024-04-22T23:46:16.534 INFO:tasks.workunit.client.1.smithi044.stdout:5/746: dread - d15/d16/d1d/d22/d2a/d3a/d43/f88 zero size 2024-04-22T23:46:16.535 INFO:tasks.workunit.client.1.smithi044.stdout:6/581: chown d5/l1c 7041 1 2024-04-22T23:46:16.535 INFO:tasks.workunit.client.1.smithi044.stdout:1/706: symlink d1/d8/d1c/d23/l105 0 2024-04-22T23:46:16.536 INFO:tasks.workunit.client.1.smithi044.stdout:3/723: dwrite d9/db/d33/d5d/f66 [0,4194304] 0 2024-04-22T23:46:16.545 INFO:tasks.workunit.client.1.smithi044.stdout:8/733: rmdir db/d14/dc7/dc9/df3 0 2024-04-22T23:46:16.546 INFO:tasks.workunit.client.1.smithi044.stdout:3/724: dwrite d9/db/d9f/d38/d74/fc4 [0,4194304] 0 2024-04-22T23:46:16.630 INFO:tasks.workunit.client.1.smithi044.stdout:0/834: mkdir d0/d2/dd/d3e/d40/d43/d79/d12a/d12b 0 2024-04-22T23:46:16.630 INFO:tasks.workunit.client.1.smithi044.stdout:4/738: truncate d5/db/d16/d4d/dad/fe6 1360818 0 2024-04-22T23:46:16.632 INFO:tasks.workunit.client.1.smithi044.stdout:7/704: creat dd/d1a/db7/dc8/d97/db9/f105 x:0 0 0 2024-04-22T23:46:16.641 INFO:tasks.workunit.client.1.smithi044.stdout:0/835: creat d0/d2/d4b/dad/dc8/d10e/f12c x:0 0 0 2024-04-22T23:46:16.641 INFO:tasks.workunit.client.1.smithi044.stdout:8/734: write db/d14/d24/f51 [612610,17440] 0 2024-04-22T23:46:16.644 INFO:tasks.workunit.client.1.smithi044.stdout:4/739: write d5/db/d34/fb9 [2049619,11338] 0 2024-04-22T23:46:16.645 INFO:tasks.workunit.client.1.smithi044.stdout:6/582: dread d5/dd/f3e [0,4194304] 0 2024-04-22T23:46:16.649 INFO:tasks.workunit.client.1.smithi044.stdout:0/836: fdatasync d0/d4c/f8f 0 2024-04-22T23:46:16.651 INFO:tasks.workunit.client.1.smithi044.stdout:8/735: mknod db/d52/d9c/dad/db8/c107 0 2024-04-22T23:46:16.652 INFO:tasks.workunit.client.1.smithi044.stdout:4/740: creat d5/d31/d24/da6/ff4 x:0 0 0 2024-04-22T23:46:16.654 INFO:tasks.workunit.client.1.smithi044.stdout:3/725: getdents d9/dc8 0 2024-04-22T23:46:16.654 INFO:tasks.workunit.client.1.smithi044.stdout:0/837: stat d0/d2/d4b/d6b/fc2 0 2024-04-22T23:46:16.658 INFO:tasks.workunit.client.1.smithi044.stdout:8/736: symlink db/d52/d9c/dad/l108 0 2024-04-22T23:46:16.659 INFO:tasks.workunit.client.1.smithi044.stdout:1/707: dwrite d1/d8/d20/d47/d92/fba [4194304,4194304] 0 2024-04-22T23:46:16.660 INFO:tasks.workunit.client.1.smithi044.stdout:3/726: write d9/db/d15/d25/d5b/f7e [329139,62367] 0 2024-04-22T23:46:16.663 INFO:tasks.workunit.client.1.smithi044.stdout:6/583: dwrite d5/d3f/f99 [0,4194304] 0 2024-04-22T23:46:16.664 INFO:tasks.workunit.client.1.smithi044.stdout:8/737: fsync db/d68/d8b/da7/fcd 0 2024-04-22T23:46:16.667 INFO:tasks.workunit.client.1.smithi044.stdout:7/705: dread dd/ff4 [0,4194304] 0 2024-04-22T23:46:16.668 INFO:tasks.workunit.client.1.smithi044.stdout:3/727: chown d9/db/d9f/d38/f4a 412290790 1 2024-04-22T23:46:16.669 INFO:tasks.workunit.client.1.smithi044.stdout:4/741: creat d5/d31/d24/d38/ff5 x:0 0 0 2024-04-22T23:46:16.669 INFO:tasks.workunit.client.1.smithi044.stdout:4/742: fdatasync d5/d31/d24/d38/d84/fcb 0 2024-04-22T23:46:16.670 INFO:tasks.workunit.client.1.smithi044.stdout:0/838: mknod d0/d4c/d123/c12d 0 2024-04-22T23:46:16.671 INFO:tasks.workunit.client.1.smithi044.stdout:8/738: fdatasync db/ddd/df2/fe1 0 2024-04-22T23:46:16.672 INFO:tasks.workunit.client.1.smithi044.stdout:1/708: mkdir d1/d8/d20/d106 0 2024-04-22T23:46:16.672 INFO:tasks.workunit.client.1.smithi044.stdout:3/728: write d9/db/f10 [10956164,75461] 0 2024-04-22T23:46:16.676 INFO:tasks.workunit.client.1.smithi044.stdout:1/709: write d1/d8/d1c/d7b/f66 [7436581,126392] 0 2024-04-22T23:46:16.679 INFO:tasks.workunit.client.1.smithi044.stdout:6/584: read - d5/d3f/d4c/d23/d3d/f51 zero size 2024-04-22T23:46:16.681 INFO:tasks.workunit.client.1.smithi044.stdout:3/729: dread d9/db/f13 [0,4194304] 0 2024-04-22T23:46:16.682 INFO:tasks.workunit.client.1.smithi044.stdout:4/743: mknod d5/db/d16/d4d/dad/cf6 0 2024-04-22T23:46:16.683 INFO:tasks.workunit.client.1.smithi044.stdout:0/839: fdatasync d0/d4c/d59/d5a/fd3 0 2024-04-22T23:46:16.686 INFO:tasks.workunit.client.1.smithi044.stdout:6/585: rmdir d5/d3f/d4c/d83 39 2024-04-22T23:46:16.687 INFO:tasks.workunit.client.1.smithi044.stdout:6/586: rename d5/d3f/d4c/db4 to d5/d3f/d4c/db4/dbc 22 2024-04-22T23:46:16.688 INFO:tasks.workunit.client.1.smithi044.stdout:3/730: creat d9/db/d15/d25/d4c/fee x:0 0 0 2024-04-22T23:46:16.689 INFO:tasks.workunit.client.1.smithi044.stdout:6/587: write d5/d3f/f63 [4421975,72308] 0 2024-04-22T23:46:16.690 INFO:tasks.workunit.client.1.smithi044.stdout:8/739: unlink db/d14/d24/l45 0 2024-04-22T23:46:16.691 INFO:tasks.workunit.client.1.smithi044.stdout:6/588: write d5/f4d [990107,23153] 0 2024-04-22T23:46:16.694 INFO:tasks.workunit.client.1.smithi044.stdout:0/840: truncate d0/d2e/d2f/d77/fba 3485935 0 2024-04-22T23:46:16.695 INFO:tasks.workunit.client.1.smithi044.stdout:7/706: write dd/d27/f62 [419454,62489] 0 2024-04-22T23:46:16.696 INFO:tasks.workunit.client.1.smithi044.stdout:7/707: fdatasync dd/d1a/db7/dc8/fc6 0 2024-04-22T23:46:16.698 INFO:tasks.workunit.client.1.smithi044.stdout:7/708: write dd/d55/d5e/d60/d7b/fce [119354,17687] 0 2024-04-22T23:46:16.701 INFO:tasks.workunit.client.1.smithi044.stdout:7/709: stat dd/d55/d5e/d60/d68/cfe 0 2024-04-22T23:46:16.702 INFO:tasks.workunit.client.1.smithi044.stdout:4/744: dread d5/d31/d24/d38/d6a/d75/fe1 [0,4194304] 0 2024-04-22T23:46:16.703 INFO:tasks.workunit.client.1.smithi044.stdout:3/731: rename d9/db/d15/d54/d3c/fe6 to d9/dce/fef 0 2024-04-22T23:46:16.705 INFO:tasks.workunit.client.1.smithi044.stdout:2/577: sync 2024-04-22T23:46:16.711 INFO:tasks.workunit.client.1.smithi044.stdout:0/841: mknod d0/d2/d4b/dad/c12e 0 2024-04-22T23:46:16.711 INFO:tasks.workunit.client.1.smithi044.stdout:0/842: readlink d0/d2e/dbf/lce 0 2024-04-22T23:46:16.714 INFO:tasks.workunit.client.1.smithi044.stdout:6/589: write d5/f12 [3003273,515] 0 2024-04-22T23:46:16.715 INFO:tasks.workunit.client.1.smithi044.stdout:7/710: mkdir dd/d55/d5e/d60/d106 0 2024-04-22T23:46:16.716 INFO:tasks.workunit.client.1.smithi044.stdout:7/711: truncate dd/d27/fb6 1334159 0 2024-04-22T23:46:16.719 INFO:tasks.workunit.client.1.smithi044.stdout:3/732: rmdir d9/db/d33 39 2024-04-22T23:46:16.725 INFO:tasks.workunit.client.1.smithi044.stdout:1/710: getdents d1/d8/d1c/d7b 0 2024-04-22T23:46:16.727 INFO:tasks.workunit.client.1.smithi044.stdout:6/590: truncate d5/dd/d28/f2c 511985 0 2024-04-22T23:46:16.728 INFO:tasks.workunit.client.1.smithi044.stdout:6/591: fdatasync d5/d3f/d4c/d40/f78 0 2024-04-22T23:46:16.730 INFO:tasks.workunit.client.1.smithi044.stdout:7/712: creat dd/d98/de8/f107 x:0 0 0 2024-04-22T23:46:16.732 INFO:tasks.workunit.client.1.smithi044.stdout:4/745: dread d5/d31/f11 [0,4194304] 0 2024-04-22T23:46:16.732 INFO:tasks.workunit.client.1.smithi044.stdout:8/740: rmdir db/d52/d9c/dad/db8/df7 0 2024-04-22T23:46:16.733 INFO:tasks.workunit.client.1.smithi044.stdout:9/555: sync 2024-04-22T23:46:16.734 INFO:tasks.workunit.client.1.smithi044.stdout:2/578: link f6 d8/d19/d36/fcd 0 2024-04-22T23:46:16.736 INFO:tasks.workunit.client.1.smithi044.stdout:0/843: symlink d0/l12f 0 2024-04-22T23:46:16.739 INFO:tasks.workunit.client.1.smithi044.stdout:6/592: unlink d5/d3f/d4c/fb6 0 2024-04-22T23:46:16.741 INFO:tasks.workunit.client.1.smithi044.stdout:3/733: mknod d9/db/d33/d5d/d70/cf0 0 2024-04-22T23:46:16.742 INFO:tasks.workunit.client.1.smithi044.stdout:9/556: dwrite d2/dc/d1d/d34/d38/d5c/f65 [0,4194304] 0 2024-04-22T23:46:16.743 INFO:tasks.workunit.client.1.smithi044.stdout:3/734: readlink d9/db/d33/l77 0 2024-04-22T23:46:16.753 INFO:tasks.workunit.client.1.smithi044.stdout:3/735: dwrite d9/db/d9f/fd7 [0,4194304] 0 2024-04-22T23:46:16.756 INFO:tasks.workunit.client.1.smithi044.stdout:1/711: symlink d1/d8/d20/d45/d75/df2/l107 0 2024-04-22T23:46:16.759 INFO:tasks.workunit.client.1.smithi044.stdout:2/579: read d8/d19/f2b [580748,125345] 0 2024-04-22T23:46:16.768 INFO:tasks.workunit.client.1.smithi044.stdout:9/557: dread d2/dc/d1d/d3f/f6e [0,4194304] 0 2024-04-22T23:46:16.768 INFO:tasks.workunit.client.1.smithi044.stdout:2/580: dwrite d8/d19/f25 [0,4194304] 0 2024-04-22T23:46:16.769 INFO:tasks.workunit.client.1.smithi044.stdout:9/558: chown d2/d9/c1b 0 1 2024-04-22T23:46:16.771 INFO:tasks.workunit.client.1.smithi044.stdout:7/713: write dd/d1f/fa0 [1040763,111270] 0 2024-04-22T23:46:16.775 INFO:tasks.workunit.client.1.smithi044.stdout:6/593: symlink d5/d3f/d4c/d23/d3d/d64/lbd 0 2024-04-22T23:46:16.776 INFO:tasks.workunit.client.1.smithi044.stdout:2/581: dread d8/d27/f28 [0,4194304] 0 2024-04-22T23:46:16.779 INFO:tasks.workunit.client.1.smithi044.stdout:8/741: mknod db/d14/d84/c109 0 2024-04-22T23:46:16.779 INFO:tasks.workunit.client.1.smithi044.stdout:4/746: dwrite d5/d31/d44/f4a [0,4194304] 0 2024-04-22T23:46:16.783 INFO:tasks.workunit.client.1.smithi044.stdout:3/736: rmdir d9/d14 39 2024-04-22T23:46:16.785 INFO:tasks.workunit.client.1.smithi044.stdout:1/712: fdatasync d1/d8/d1c/d7b/fb6 0 2024-04-22T23:46:16.789 INFO:tasks.workunit.client.1.smithi044.stdout:1/713: write d1/d8/d20/d45/d75/dfb/d95/fd5 [1014430,72271] 0 2024-04-22T23:46:16.791 INFO:tasks.workunit.client.1.smithi044.stdout:1/714: fdatasync d1/d8/d1c/d7b/d80/dfd/dbf/dc2/ff0 0 2024-04-22T23:46:16.798 INFO:tasks.workunit.client.1.smithi044.stdout:5/747: sync 2024-04-22T23:46:16.799 INFO:tasks.workunit.client.1.smithi044.stdout:7/714: mkdir dd/df7/db8/d108 0 2024-04-22T23:46:16.799 INFO:tasks.workunit.client.1.smithi044.stdout:2/582: mknod d8/d19/d36/d93/cce 0 2024-04-22T23:46:16.804 INFO:tasks.workunit.client.1.smithi044.stdout:5/748: dread d15/f94 [0,4194304] 0 2024-04-22T23:46:16.809 INFO:tasks.workunit.client.1.smithi044.stdout:6/594: dread d5/d3f/d50/d79/f81 [0,4194304] 0 2024-04-22T23:46:16.814 INFO:tasks.workunit.client.1.smithi044.stdout:9/559: link d2/dc/fb0 d2/dc/d1d/d34/d38/d5c/d69/d9f/fb9 0 2024-04-22T23:46:16.815 INFO:tasks.workunit.client.1.smithi044.stdout:7/715: creat dd/df7/db8/f109 x:0 0 0 2024-04-22T23:46:16.816 INFO:tasks.workunit.client.1.smithi044.stdout:2/583: rmdir d8/d19/d36/d46/d63 39 2024-04-22T23:46:16.817 INFO:tasks.workunit.client.1.smithi044.stdout:2/584: chown d8/d27/d5b/f9a 32 1 2024-04-22T23:46:16.821 INFO:tasks.workunit.client.1.smithi044.stdout:4/747: symlink d5/d31/lf7 0 2024-04-22T23:46:16.822 INFO:tasks.workunit.client.1.smithi044.stdout:4/748: stat d5/db/d16/d4d/lba 0 2024-04-22T23:46:16.823 INFO:tasks.workunit.client.1.smithi044.stdout:6/595: creat d5/d3f/d4c/d92/fbe x:0 0 0 2024-04-22T23:46:16.823 INFO:tasks.workunit.client.1.smithi044.stdout:3/737: dread f5 [0,4194304] 0 2024-04-22T23:46:16.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/596: write d5/dd/d28/f93 [346110,119565] 0 2024-04-22T23:46:16.830 INFO:tasks.workunit.client.1.smithi044.stdout:7/716: dread dd/df7/fa9 [0,4194304] 0 2024-04-22T23:46:16.831 INFO:tasks.workunit.client.1.smithi044.stdout:3/738: dread d9/db/d15/d25/d4c/d86/fc6 [0,4194304] 0 2024-04-22T23:46:16.833 INFO:tasks.workunit.client.1.smithi044.stdout:9/560: symlink d2/lba 0 2024-04-22T23:46:16.836 INFO:tasks.workunit.client.1.smithi044.stdout:4/749: rename d5/d31/f76 to d5/d31/d24/d38/d84/ff8 0 2024-04-22T23:46:16.839 INFO:tasks.workunit.client.1.smithi044.stdout:9/561: write d2/dc/d1d/d34/d38/d5c/d69/fb8 [761239,57795] 0 2024-04-22T23:46:16.839 INFO:tasks.workunit.client.1.smithi044.stdout:1/715: getdents d1/d8/d1c/d23/d3e/dde 0 2024-04-22T23:46:16.839 INFO:tasks.workunit.client.1.smithi044.stdout:6/597: dwrite d5/dd/d28/f2e [0,4194304] 0 2024-04-22T23:46:16.843 INFO:tasks.workunit.client.1.smithi044.stdout:3/739: readlink d9/db/d33/d5d/d64/lb8 0 2024-04-22T23:46:16.843 INFO:tasks.workunit.client.1.smithi044.stdout:0/844: dwrite d0/d2/d4b/f68 [0,4194304] 0 2024-04-22T23:46:16.844 INFO:tasks.workunit.client.1.smithi044.stdout:2/585: rename d8/d19/d36 to d8/d19/d36/d46/d63/dcf 22 2024-04-22T23:46:16.846 INFO:tasks.workunit.client.1.smithi044.stdout:4/750: truncate d5/d31/f1b 2791015 0 2024-04-22T23:46:16.848 INFO:tasks.workunit.client.1.smithi044.stdout:4/751: chown d5/d52/l8c 106324 1 2024-04-22T23:46:16.849 INFO:tasks.workunit.client.1.smithi044.stdout:6/598: symlink d5/d3f/d4c/d92/d97/lbf 0 2024-04-22T23:46:16.850 INFO:tasks.workunit.client.1.smithi044.stdout:4/752: dread - d5/fd3 zero size 2024-04-22T23:46:16.851 INFO:tasks.workunit.client.1.smithi044.stdout:9/562: dread d2/f7d [0,4194304] 0 2024-04-22T23:46:16.854 INFO:tasks.workunit.client.1.smithi044.stdout:7/717: link dd/df7/db8/ff1 dd/d55/d5e/d60/d68/f10a 0 2024-04-22T23:46:16.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/599: read d5/d30/f44 [2041196,48626] 0 2024-04-22T23:46:16.858 INFO:tasks.workunit.client.1.smithi044.stdout:3/740: creat d9/db/d33/d5d/d64/ff1 x:0 0 0 2024-04-22T23:46:16.858 INFO:tasks.workunit.client.1.smithi044.stdout:1/716: truncate d1/d8/d1c/d7b/d80/dfd/dad/fff 219638 0 2024-04-22T23:46:16.862 INFO:tasks.workunit.client.1.smithi044.stdout:1/717: fsync d1/d8/d20/d45/d5d/fee 0 2024-04-22T23:46:16.864 INFO:tasks.workunit.client.1.smithi044.stdout:1/718: fdatasync d1/d8/d1c/d23/f42 0 2024-04-22T23:46:16.866 INFO:tasks.workunit.client.1.smithi044.stdout:1/719: truncate d1/d8/d20/d45/d5d/fdf 1038794 0 2024-04-22T23:46:16.867 INFO:tasks.workunit.client.1.smithi044.stdout:4/753: creat d5/db/d16/d4d/dad/dc5/ff9 x:0 0 0 2024-04-22T23:46:16.868 INFO:tasks.workunit.client.1.smithi044.stdout:0/845: mkdir d0/d4c/d130 0 2024-04-22T23:46:16.869 INFO:tasks.workunit.client.1.smithi044.stdout:4/754: stat d5/d31/d44/d20/d22/c33 0 2024-04-22T23:46:16.871 INFO:tasks.workunit.client.1.smithi044.stdout:7/718: symlink dd/d55/d5e/l10b 0 2024-04-22T23:46:16.883 INFO:tasks.workunit.client.1.smithi044.stdout:3/741: creat d9/db/d9f/d38/d9e/ff2 x:0 0 0 2024-04-22T23:46:16.885 INFO:tasks.workunit.client.1.smithi044.stdout:1/720: creat d1/d8/d1c/d74/f108 x:0 0 0 2024-04-22T23:46:16.886 INFO:tasks.workunit.client.1.smithi044.stdout:5/749: write f14 [3965079,42107] 0 2024-04-22T23:46:16.888 INFO:tasks.workunit.client.1.smithi044.stdout:6/600: mkdir d5/d3f/d4c/d40/db9/dc0 0 2024-04-22T23:46:16.892 INFO:tasks.workunit.client.1.smithi044.stdout:3/742: symlink d9/db/d15/d54/lf3 0 2024-04-22T23:46:16.894 INFO:tasks.workunit.client.1.smithi044.stdout:8/742: truncate db/d14/dc7/dc9/fe9 1391705 0 2024-04-22T23:46:16.895 INFO:tasks.workunit.client.1.smithi044.stdout:0/846: getdents d0/d2/dd/d3e/d40/d43/d79/d12a/d12b 0 2024-04-22T23:46:16.896 INFO:tasks.workunit.client.1.smithi044.stdout:0/847: fsync d0/d2e/d33/ffb 0 2024-04-22T23:46:16.903 INFO:tasks.workunit.client.1.smithi044.stdout:7/719: link dd/d27/f6d dd/df7/db8/f10c 0 2024-04-22T23:46:16.904 INFO:tasks.workunit.client.1.smithi044.stdout:2/586: getdents d8/d19/d36/d46 0 2024-04-22T23:46:16.907 INFO:tasks.workunit.client.1.smithi044.stdout:2/587: truncate d8/d19/d36/d46/d63/d65/d94/fa7 203285 0 2024-04-22T23:46:16.907 INFO:tasks.workunit.client.1.smithi044.stdout:4/755: rmdir d5/d31/d24/d38/df0 0 2024-04-22T23:46:16.909 INFO:tasks.workunit.client.1.smithi044.stdout:4/756: read d5/d31/d44/d7e/f98 [3991534,101838] 0 2024-04-22T23:46:16.909 INFO:tasks.workunit.client.1.smithi044.stdout:3/743: creat d9/dc8/ff4 x:0 0 0 2024-04-22T23:46:16.911 INFO:tasks.workunit.client.1.smithi044.stdout:5/750: symlink d15/d16/d1d/d22/d2a/d3a/dd2/lf4 0 2024-04-22T23:46:16.915 INFO:tasks.workunit.client.1.smithi044.stdout:9/563: link d2/d6/d42/la2 d2/lbb 0 2024-04-22T23:46:16.917 INFO:tasks.workunit.client.1.smithi044.stdout:9/564: chown d2/dc/d94/c85 0 1 2024-04-22T23:46:16.917 INFO:tasks.workunit.client.1.smithi044.stdout:8/743: dread db/d14/d9e/ff4 [0,4194304] 0 2024-04-22T23:46:16.918 INFO:tasks.workunit.client.1.smithi044.stdout:2/588: rmdir d8/d19/d5d 39 2024-04-22T23:46:16.924 INFO:tasks.workunit.client.1.smithi044.stdout:0/848: symlink d0/d2/d4b/dad/d125/l131 0 2024-04-22T23:46:16.926 INFO:tasks.workunit.client.1.smithi044.stdout:5/751: rmdir d15/d5d/d64/dc5 39 2024-04-22T23:46:16.934 INFO:tasks.workunit.client.1.smithi044.stdout:6/601: getdents d5/d3f/d50/d79/d96 0 2024-04-22T23:46:16.935 INFO:tasks.workunit.client.1.smithi044.stdout:6/602: chown d5/dd/d28/c4b 1506 1 2024-04-22T23:46:16.937 INFO:tasks.workunit.client.1.smithi044.stdout:8/744: mknod db/d52/d9c/dad/dc4/c10a 0 2024-04-22T23:46:16.941 INFO:tasks.workunit.client.1.smithi044.stdout:0/849: link d0/d2/dd/d3e/f6e d0/d2/d4b/dad/d125/f132 0 2024-04-22T23:46:16.945 INFO:tasks.workunit.client.1.smithi044.stdout:6/603: read d5/dd/f1b [7119653,61843] 0 2024-04-22T23:46:16.946 INFO:tasks.workunit.client.1.smithi044.stdout:6/604: chown f1 8868 1 2024-04-22T23:46:16.949 INFO:tasks.workunit.client.1.smithi044.stdout:6/605: stat d5/d3f/d4c/d92/fa5 0 2024-04-22T23:46:16.949 INFO:tasks.workunit.client.1.smithi044.stdout:1/721: write d1/d8/d1c/d7b/d80/d4b/f5a [1233126,5729] 0 2024-04-22T23:46:16.950 INFO:tasks.workunit.client.1.smithi044.stdout:0/850: dwrite d0/d2/dd/d3e/d40/d43/ff9 [0,4194304] 0 2024-04-22T23:46:16.953 INFO:tasks.workunit.client.1.smithi044.stdout:8/745: rename db/c7f to db/d14/d84/dbb/c10b 0 2024-04-22T23:46:16.955 INFO:tasks.workunit.client.1.smithi044.stdout:6/606: creat d5/d30/d73/fc1 x:0 0 0 2024-04-22T23:46:16.956 INFO:tasks.workunit.client.1.smithi044.stdout:6/607: chown d5/d3f/d4c/d23/d3d 18274853 1 2024-04-22T23:46:16.957 INFO:tasks.workunit.client.1.smithi044.stdout:6/608: dread - d5/d3f/d4c/d92/f98 zero size 2024-04-22T23:46:16.959 INFO:tasks.workunit.client.1.smithi044.stdout:0/851: mknod d0/d2/dd/d3e/d40/d43/d79/dbb/c133 0 2024-04-22T23:46:16.960 INFO:tasks.workunit.client.1.smithi044.stdout:0/852: readlink d0/d2/d4b/d6b/l9c 0 2024-04-22T23:46:16.962 INFO:tasks.workunit.client.1.smithi044.stdout:7/720: write dd/df7/fa9 [170671,11792] 0 2024-04-22T23:46:16.964 INFO:tasks.workunit.client.1.smithi044.stdout:1/722: creat d1/d8/d20/d106/f109 x:0 0 0 2024-04-22T23:46:16.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/853: dread d0/d2e/d33/ffb [0,4194304] 0 2024-04-22T23:46:16.967 INFO:tasks.workunit.client.1.smithi044.stdout:4/757: write d5/d31/d44/d20/d93/fea [654895,120398] 0 2024-04-22T23:46:16.968 INFO:tasks.workunit.client.1.smithi044.stdout:8/746: unlink db/d14/d24/ff1 0 2024-04-22T23:46:16.975 INFO:tasks.workunit.client.1.smithi044.stdout:4/758: chown d5/d31/d44/d20/d93/fc4 43291 1 2024-04-22T23:46:16.975 INFO:tasks.workunit.client.1.smithi044.stdout:3/744: dwrite d9/db/d15/f21 [0,4194304] 0 2024-04-22T23:46:16.979 INFO:tasks.workunit.client.1.smithi044.stdout:7/721: symlink dd/df7/db8/d108/l10d 0 2024-04-22T23:46:16.981 INFO:tasks.workunit.client.1.smithi044.stdout:9/565: dwrite d2/dc/d1d/d34/d38/d5c/d69/fac [0,4194304] 0 2024-04-22T23:46:16.981 INFO:tasks.workunit.client.1.smithi044.stdout:3/745: dread - d9/db/d15/d2e/d48/f73 zero size 2024-04-22T23:46:16.982 INFO:tasks.workunit.client.1.smithi044.stdout:2/589: dwrite f6 [0,4194304] 0 2024-04-22T23:46:16.984 INFO:tasks.workunit.client.1.smithi044.stdout:1/723: rename d1/d8/d1c/d7b/d80/fe5 to d1/d8/d20/d45/d75/df2/d99/f10a 0 2024-04-22T23:46:16.993 INFO:tasks.workunit.client.1.smithi044.stdout:0/854: mknod d0/d2/c134 0 2024-04-22T23:46:16.995 INFO:tasks.workunit.client.1.smithi044.stdout:4/759: creat d5/db/d3c/ffa x:0 0 0 2024-04-22T23:46:16.996 INFO:tasks.workunit.client.1.smithi044.stdout:9/566: write d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f [2302127,53052] 0 2024-04-22T23:46:16.997 INFO:tasks.workunit.client.1.smithi044.stdout:7/722: fsync dd/df7/db8/fee 0 2024-04-22T23:46:16.997 INFO:tasks.workunit.client.1.smithi044.stdout:8/747: truncate db/f4f 1135553 0 2024-04-22T23:46:16.998 INFO:tasks.workunit.client.1.smithi044.stdout:7/723: write dd/d98/de8/f107 [283249,119292] 0 2024-04-22T23:46:16.999 INFO:tasks.workunit.client.1.smithi044.stdout:7/724: chown dd 29 1 2024-04-22T23:46:16.999 INFO:tasks.workunit.client.1.smithi044.stdout:3/746: mknod d9/db/d15/d54/d3c/cf5 0 2024-04-22T23:46:17.000 INFO:tasks.workunit.client.1.smithi044.stdout:8/748: read - db/d14/d72/ffd zero size 2024-04-22T23:46:17.001 INFO:tasks.workunit.client.1.smithi044.stdout:3/747: write d9/dc8/ddd/fea [966494,32846] 0 2024-04-22T23:46:17.030 INFO:tasks.workunit.client.1.smithi044.stdout:6/609: getdents d5/d3f/d4c/db4 0 2024-04-22T23:46:17.033 INFO:tasks.workunit.client.1.smithi044.stdout:0/855: creat d0/d2e/dbc/f135 x:0 0 0 2024-04-22T23:46:17.036 INFO:tasks.workunit.client.1.smithi044.stdout:4/760: mknod d5/db/d16/d4d/cfb 0 2024-04-22T23:46:17.037 INFO:tasks.workunit.client.1.smithi044.stdout:4/761: fsync d5/d42/fe0 0 2024-04-22T23:46:17.038 INFO:tasks.workunit.client.1.smithi044.stdout:7/725: rename dd/d27/f37 to dd/d99/f10e 0 2024-04-22T23:46:17.042 INFO:tasks.workunit.client.1.smithi044.stdout:8/749: symlink db/d14/d84/dc0/l10c 0 2024-04-22T23:46:17.045 INFO:tasks.workunit.client.1.smithi044.stdout:2/590: mkdir d8/d19/d1c/daf/dd0 0 2024-04-22T23:46:17.047 INFO:tasks.workunit.client.1.smithi044.stdout:6/610: creat d5/dd/d28/fc2 x:0 0 0 2024-04-22T23:46:17.049 INFO:tasks.workunit.client.1.smithi044.stdout:1/724: mkdir d1/d10b 0 2024-04-22T23:46:17.051 INFO:tasks.workunit.client.1.smithi044.stdout:0/856: mkdir d0/d2/d4b/dad/d125/d136 0 2024-04-22T23:46:17.052 INFO:tasks.workunit.client.1.smithi044.stdout:1/725: stat d1/d8/d1c/d74/f108 0 2024-04-22T23:46:17.053 INFO:tasks.workunit.client.1.smithi044.stdout:8/750: dwrite db/d14/d24/fd7 [0,4194304] 0 2024-04-22T23:46:17.059 INFO:tasks.workunit.client.1.smithi044.stdout:2/591: dwrite d8/d19/d1c/f2f [0,4194304] 0 2024-04-22T23:46:17.064 INFO:tasks.workunit.client.1.smithi044.stdout:7/726: creat dd/df7/dc0/f10f x:0 0 0 2024-04-22T23:46:17.066 INFO:tasks.workunit.client.1.smithi044.stdout:7/727: chown dd/df7/db8/fd2 983377613 1 2024-04-22T23:46:17.066 INFO:tasks.workunit.client.1.smithi044.stdout:7/728: chown dd/d27/f6d 570852636 1 2024-04-22T23:46:17.070 INFO:tasks.workunit.client.1.smithi044.stdout:0/857: rename d0/d2/d4b/f80 to d0/d2e/dbf/f137 0 2024-04-22T23:46:17.072 INFO:tasks.workunit.client.1.smithi044.stdout:4/762: mkdir d5/db/d16/d4d/dad/deb/dfc 0 2024-04-22T23:46:17.073 INFO:tasks.workunit.client.1.smithi044.stdout:7/729: read dd/d1a/db7/dc8/fa1 [4102189,11316] 0 2024-04-22T23:46:17.074 INFO:tasks.workunit.client.1.smithi044.stdout:8/751: truncate db/d6c/fa0 3899150 0 2024-04-22T23:46:17.074 INFO:tasks.workunit.client.1.smithi044.stdout:6/611: dread d5/d3f/f63 [4194304,4194304] 0 2024-04-22T23:46:17.076 INFO:tasks.workunit.client.1.smithi044.stdout:1/726: truncate d1/d8/d1c/d23/d3e/f7e 117144 0 2024-04-22T23:46:17.078 INFO:tasks.workunit.client.1.smithi044.stdout:8/752: truncate db/d68/d8b/da7/fcd 938635 0 2024-04-22T23:46:17.082 INFO:tasks.workunit.client.1.smithi044.stdout:3/748: creat d9/db/ff6 x:0 0 0 2024-04-22T23:46:17.085 INFO:tasks.workunit.client.1.smithi044.stdout:7/730: dwrite dd/d1a/db7/dc8/fc6 [0,4194304] 0 2024-04-22T23:46:17.085 INFO:tasks.workunit.client.1.smithi044.stdout:6/612: dwrite d5/d3f/d4c/db4/fbb [0,4194304] 0 2024-04-22T23:46:17.088 INFO:tasks.workunit.client.1.smithi044.stdout:0/858: symlink d0/d2/dd/d3e/d40/d43/dde/l138 0 2024-04-22T23:46:17.093 INFO:tasks.workunit.client.1.smithi044.stdout:0/859: chown d0/d2/d4b/f8d 96001 1 2024-04-22T23:46:17.096 INFO:tasks.workunit.client.1.smithi044.stdout:4/763: rmdir d5/db/d16/d4d/dad/dc5 39 2024-04-22T23:46:17.097 INFO:tasks.workunit.client.1.smithi044.stdout:7/731: dwrite dd/d1a/db7/dc8/f49 [0,4194304] 0 2024-04-22T23:46:17.100 INFO:tasks.workunit.client.1.smithi044.stdout:1/727: creat d1/d8/d20/d47/d92/f10c x:0 0 0 2024-04-22T23:46:17.100 INFO:tasks.workunit.client.1.smithi044.stdout:6/613: dwrite d5/d3f/f54 [0,4194304] 0 2024-04-22T23:46:17.103 INFO:tasks.workunit.client.1.smithi044.stdout:8/753: chown db/d14/f78 45462 1 2024-04-22T23:46:17.106 INFO:tasks.workunit.client.1.smithi044.stdout:8/754: dread db/d52/f83 [0,4194304] 0 2024-04-22T23:46:17.113 INFO:tasks.workunit.client.1.smithi044.stdout:6/614: dwrite d5/d3f/d50/f5a [0,4194304] 0 2024-04-22T23:46:17.113 INFO:tasks.workunit.client.1.smithi044.stdout:6/615: fsync f0 0 2024-04-22T23:46:17.116 INFO:tasks.workunit.client.1.smithi044.stdout:7/732: dread dd/d27/fb6 [0,4194304] 0 2024-04-22T23:46:17.119 INFO:tasks.workunit.client.1.smithi044.stdout:7/733: truncate dd/df7/db8/ff1 1020809 0 2024-04-22T23:46:17.122 INFO:tasks.workunit.client.1.smithi044.stdout:0/860: creat d0/d2e/dbc/f139 x:0 0 0 2024-04-22T23:46:17.130 INFO:tasks.workunit.client.1.smithi044.stdout:7/734: dwrite dd/d55/d5e/d60/d68/d94/dbd/dc9/fed [0,4194304] 0 2024-04-22T23:46:17.130 INFO:tasks.workunit.client.1.smithi044.stdout:9/567: truncate d2/d6/fa8 24933 0 2024-04-22T23:46:17.143 INFO:tasks.workunit.client.1.smithi044.stdout:1/728: dread d1/d8/d1c/d7b/fd1 [0,4194304] 0 2024-04-22T23:46:17.149 INFO:tasks.workunit.client.1.smithi044.stdout:2/592: write d8/fd [4458139,114173] 0 2024-04-22T23:46:17.154 INFO:tasks.workunit.client.1.smithi044.stdout:8/755: mknod db/c10d 0 2024-04-22T23:46:17.156 INFO:tasks.workunit.client.1.smithi044.stdout:7/735: rename dd/d1f/d24/c7a to dd/d98/de8/df2/c110 0 2024-04-22T23:46:17.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/736: fdatasync dd/d98/de8/ff0 0 2024-04-22T23:46:17.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/737: readlink dd/d55/dcf/lef 0 2024-04-22T23:46:17.158 INFO:tasks.workunit.client.1.smithi044.stdout:9/568: symlink d2/dc/d1d/d34/d38/d5c/d69/lbc 0 2024-04-22T23:46:17.168 INFO:tasks.workunit.client.1.smithi044.stdout:2/593: creat d8/d60/fd1 x:0 0 0 2024-04-22T23:46:17.170 INFO:tasks.workunit.client.1.smithi044.stdout:2/594: write d8/d19/d96/fad [1012909,79219] 0 2024-04-22T23:46:17.170 INFO:tasks.workunit.client.1.smithi044.stdout:8/756: readlink db/d52/d9c/l105 0 2024-04-22T23:46:17.185 INFO:tasks.workunit.client.1.smithi044.stdout:4/764: getdents d5/d31/d24/d38/d6a/d75 0 2024-04-22T23:46:17.185 INFO:tasks.workunit.client.1.smithi044.stdout:4/765: chown d5/d52/fc1 4 1 2024-04-22T23:46:17.186 INFO:tasks.workunit.client.1.smithi044.stdout:2/595: mknod d8/d19/d36/d46/d63/d65/cd2 0 2024-04-22T23:46:17.188 INFO:tasks.workunit.client.1.smithi044.stdout:2/596: dread - d8/d27/d5b/d74/fab zero size 2024-04-22T23:46:17.189 INFO:tasks.workunit.client.1.smithi044.stdout:8/757: mkdir db/d52/d9c/dad/dc4/d10e 0 2024-04-22T23:46:17.190 INFO:tasks.workunit.client.1.smithi044.stdout:7/738: mkdir dd/d27/de7/d111 0 2024-04-22T23:46:17.192 INFO:tasks.workunit.client.1.smithi044.stdout:0/861: dwrite d0/d2e/f70 [0,4194304] 0 2024-04-22T23:46:17.197 INFO:tasks.workunit.client.1.smithi044.stdout:6/616: getdents d5/d3f/d4c/d23/d3d/d64 0 2024-04-22T23:46:17.198 INFO:tasks.workunit.client.1.smithi044.stdout:1/729: rmdir d1/d8/d20/d45/d75/df2/d72/dd3 0 2024-04-22T23:46:17.198 INFO:tasks.workunit.client.1.smithi044.stdout:6/617: dread - d5/d3f/d4c/d92/fbe zero size 2024-04-22T23:46:17.200 INFO:tasks.workunit.client.1.smithi044.stdout:1/730: write d1/d8/d20/d45/f97 [3678996,53299] 0 2024-04-22T23:46:17.200 INFO:tasks.workunit.client.1.smithi044.stdout:4/766: fdatasync d5/db/d16/d4d/dad/dc5/ff9 0 2024-04-22T23:46:17.203 INFO:tasks.workunit.client.1.smithi044.stdout:2/597: creat d8/d27/d5b/d74/d81/fd3 x:0 0 0 2024-04-22T23:46:17.205 INFO:tasks.workunit.client.1.smithi044.stdout:8/758: mkdir db/d68/d8b/dca/d10f 0 2024-04-22T23:46:17.206 INFO:tasks.workunit.client.1.smithi044.stdout:7/739: dread - dd/d99/fc2 zero size 2024-04-22T23:46:17.211 INFO:tasks.workunit.client.1.smithi044.stdout:9/569: fdatasync d2/d6/fa8 0 2024-04-22T23:46:17.217 INFO:tasks.workunit.client.1.smithi044.stdout:6/618: symlink d5/d3f/d50/d79/lc3 0 2024-04-22T23:46:17.217 INFO:tasks.workunit.client.1.smithi044.stdout:6/619: readlink d5/dd/d28/d75/l8b 0 2024-04-22T23:46:17.218 INFO:tasks.workunit.client.1.smithi044.stdout:6/620: read - d5/d3f/d4c/f9e zero size 2024-04-22T23:46:17.220 INFO:tasks.workunit.client.1.smithi044.stdout:4/767: rmdir d5/d52 39 2024-04-22T23:46:17.222 INFO:tasks.workunit.client.1.smithi044.stdout:1/731: dread - d1/d8/d20/d47/d92/fa2 zero size 2024-04-22T23:46:17.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/862: dwrite d0/d4c/f8f [4194304,4194304] 0 2024-04-22T23:46:17.228 INFO:tasks.workunit.client.1.smithi044.stdout:8/759: creat db/d52/f110 x:0 0 0 2024-04-22T23:46:17.231 INFO:tasks.workunit.client.1.smithi044.stdout:7/740: mkdir dd/df7/d112 0 2024-04-22T23:46:17.231 INFO:tasks.workunit.client.1.smithi044.stdout:0/863: dread d0/d4c/f8f [4194304,4194304] 0 2024-04-22T23:46:17.238 INFO:tasks.workunit.client.1.smithi044.stdout:4/768: dread d5/d31/f54 [0,4194304] 0 2024-04-22T23:46:17.239 INFO:tasks.workunit.client.1.smithi044.stdout:8/760: mknod db/d68/d8b/ded/c111 0 2024-04-22T23:46:17.241 INFO:tasks.workunit.client.1.smithi044.stdout:7/741: mkdir dd/df7/d9c/d113 0 2024-04-22T23:46:17.242 INFO:tasks.workunit.client.1.smithi044.stdout:0/864: creat d0/d2e/d2f/f13a x:0 0 0 2024-04-22T23:46:17.244 INFO:tasks.workunit.client.1.smithi044.stdout:0/865: write d0/d4c/d59/f124 [552852,111499] 0 2024-04-22T23:46:17.244 INFO:tasks.workunit.client.1.smithi044.stdout:9/570: mkdir d2/dc/d1d/d34/d8e/dbd 0 2024-04-22T23:46:17.248 INFO:tasks.workunit.client.1.smithi044.stdout:1/732: mkdir d1/d8/d1c/d10d 0 2024-04-22T23:46:17.250 INFO:tasks.workunit.client.1.smithi044.stdout:8/761: mkdir db/d68/d74/d112 0 2024-04-22T23:46:17.252 INFO:tasks.workunit.client.1.smithi044.stdout:4/769: rename d5/d31/d44/d7e/f98 to d5/db/d3c/ffd 0 2024-04-22T23:46:17.256 INFO:tasks.workunit.client.1.smithi044.stdout:0/866: chown d0/d2e/d2f/c42 17618 1 2024-04-22T23:46:17.256 INFO:tasks.workunit.client.1.smithi044.stdout:6/621: dwrite d5/dd/d67/f5d [0,4194304] 0 2024-04-22T23:46:17.260 INFO:tasks.workunit.client.1.smithi044.stdout:2/598: getdents d8/d27 0 2024-04-22T23:46:17.265 INFO:tasks.workunit.client.1.smithi044.stdout:7/742: mkdir dd/d114 0 2024-04-22T23:46:17.267 INFO:tasks.workunit.client.1.smithi044.stdout:8/762: rmdir db/d68/d74 39 2024-04-22T23:46:17.268 INFO:tasks.workunit.client.1.smithi044.stdout:6/622: dread - d5/d3f/d50/f86 zero size 2024-04-22T23:46:17.268 INFO:tasks.workunit.client.1.smithi044.stdout:8/763: truncate db/d52/f110 302872 0 2024-04-22T23:46:17.271 INFO:tasks.workunit.client.1.smithi044.stdout:2/599: mknod d8/d19/d36/d44/cd4 0 2024-04-22T23:46:17.275 INFO:tasks.workunit.client.1.smithi044.stdout:4/770: symlink d5/d31/lfe 0 2024-04-22T23:46:17.276 INFO:tasks.workunit.client.1.smithi044.stdout:7/743: mkdir dd/df7/db8/d115 0 2024-04-22T23:46:17.280 INFO:tasks.workunit.client.1.smithi044.stdout:1/733: dwrite d1/d8/fa9 [0,4194304] 0 2024-04-22T23:46:17.281 INFO:tasks.workunit.client.1.smithi044.stdout:0/867: dread d0/d4c/d59/f76 [4194304,4194304] 0 2024-04-22T23:46:17.282 INFO:tasks.workunit.client.1.smithi044.stdout:6/623: truncate d5/f21 2716771 0 2024-04-22T23:46:17.282 INFO:tasks.workunit.client.1.smithi044.stdout:0/868: chown d0/d2/d4b/dad/dc8/d10e/d11b 1935 1 2024-04-22T23:46:17.283 INFO:tasks.workunit.client.1.smithi044.stdout:1/734: dread - d1/d8/d34/f9d zero size 2024-04-22T23:46:17.285 INFO:tasks.workunit.client.1.smithi044.stdout:6/624: chown d5/dd/d28/fc2 29631926 1 2024-04-22T23:46:17.288 INFO:tasks.workunit.client.1.smithi044.stdout:6/625: write d5/d3f/d4c/d23/d3d/d64/f94 [776855,124646] 0 2024-04-22T23:46:17.289 INFO:tasks.workunit.client.1.smithi044.stdout:6/626: fsync d5/d3f/d4c/f57 0 2024-04-22T23:46:17.291 INFO:tasks.workunit.client.1.smithi044.stdout:8/764: link db/f12 db/d52/d9c/dad/f113 0 2024-04-22T23:46:17.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/735: write d1/d8/d1c/d7b/d80/dfd/dbf/fec [613187,125436] 0 2024-04-22T23:46:17.295 INFO:tasks.workunit.client.1.smithi044.stdout:7/744: creat dd/d1a/dde/f116 x:0 0 0 2024-04-22T23:46:17.296 INFO:tasks.workunit.client.1.smithi044.stdout:4/771: unlink d5/d31/d44/d20/d22/d8d/c9a 0 2024-04-22T23:46:17.297 INFO:tasks.workunit.client.1.smithi044.stdout:7/745: read dd/d1f/d24/f70 [1326545,21725] 0 2024-04-22T23:46:17.298 INFO:tasks.workunit.client.1.smithi044.stdout:6/627: mkdir d5/d3f/d4c/db4/dc4 0 2024-04-22T23:46:17.300 INFO:tasks.workunit.client.1.smithi044.stdout:7/746: write dd/df7/ff3 [680680,45494] 0 2024-04-22T23:46:17.300 INFO:tasks.workunit.client.1.smithi044.stdout:8/765: chown db/d14/c85 9 1 2024-04-22T23:46:17.303 INFO:tasks.workunit.client.1.smithi044.stdout:1/736: creat d1/d8/d20/d45/d75/df2/d99/f10e x:0 0 0 2024-04-22T23:46:17.305 INFO:tasks.workunit.client.1.smithi044.stdout:4/772: creat d5/db/d16/d4d/dad/deb/fff x:0 0 0 2024-04-22T23:46:17.307 INFO:tasks.workunit.client.1.smithi044.stdout:6/628: dwrite d5/dd/d67/f5d [0,4194304] 0 2024-04-22T23:46:17.309 INFO:tasks.workunit.client.1.smithi044.stdout:6/629: dread - d5/d3f/d4c/f9e zero size 2024-04-22T23:46:17.311 INFO:tasks.workunit.client.1.smithi044.stdout:0/869: dread d0/d2/dd/d3e/d40/d43/dde/ff3 [0,4194304] 0 2024-04-22T23:46:17.311 INFO:tasks.workunit.client.1.smithi044.stdout:8/766: write db/d14/d24/f4e [1197611,129752] 0 2024-04-22T23:46:17.311 INFO:tasks.workunit.client.1.smithi044.stdout:7/747: creat dd/d55/d5e/d60/d7b/f117 x:0 0 0 2024-04-22T23:46:17.320 INFO:tasks.workunit.client.1.smithi044.stdout:0/870: dread d0/d2/dd/d3e/d40/d67/ff5 [0,4194304] 0 2024-04-22T23:46:17.320 INFO:tasks.workunit.client.1.smithi044.stdout:2/600: truncate d8/d19/f30 733140 0 2024-04-22T23:46:17.392 INFO:tasks.workunit.client.1.smithi044.stdout:5/752: sync 2024-04-22T23:46:17.406 INFO:tasks.workunit.client.1.smithi044.stdout:8/767: read - db/d14/f36 zero size 2024-04-22T23:46:17.406 INFO:tasks.workunit.client.1.smithi044.stdout:1/737: dread d1/d8/d1c/d7b/d80/d4b/f88 [0,4194304] 0 2024-04-22T23:46:17.407 INFO:tasks.workunit.client.1.smithi044.stdout:6/630: dwrite d5/d30/d73/fc1 [0,4194304] 0 2024-04-22T23:46:17.409 INFO:tasks.workunit.client.1.smithi044.stdout:4/773: dread d5/d31/d44/d20/d22/fbf [0,4194304] 0 2024-04-22T23:46:17.414 INFO:tasks.workunit.client.1.smithi044.stdout:8/768: mknod db/d6c/c114 0 2024-04-22T23:46:17.415 INFO:tasks.workunit.client.1.smithi044.stdout:8/769: stat db/ddd/ffe 0 2024-04-22T23:46:17.416 INFO:tasks.workunit.client.1.smithi044.stdout:8/770: chown db/d52/l70 1648740116 1 2024-04-22T23:46:17.423 INFO:tasks.workunit.client.1.smithi044.stdout:1/738: creat d1/d8/d1c/d7b/d80/dfd/dbf/ddb/f10f x:0 0 0 2024-04-22T23:46:17.424 INFO:tasks.workunit.client.1.smithi044.stdout:4/774: creat d5/db/d16/d4d/dad/dc5/f100 x:0 0 0 2024-04-22T23:46:17.427 INFO:tasks.workunit.client.1.smithi044.stdout:2/601: getdents d8/d19/d36/d46 0 2024-04-22T23:46:17.429 INFO:tasks.workunit.client.1.smithi044.stdout:5/753: mkdir d15/d16/d1d/d22/df5 0 2024-04-22T23:46:17.429 INFO:tasks.workunit.client.1.smithi044.stdout:0/871: getdents d0/dd6/df4 0 2024-04-22T23:46:17.432 INFO:tasks.workunit.client.1.smithi044.stdout:1/739: truncate d1/d8/d20/d45/f39 4813758 0 2024-04-22T23:46:17.433 INFO:tasks.workunit.client.1.smithi044.stdout:4/775: truncate d5/d31/d44/d20/d22/f4c 3149474 0 2024-04-22T23:46:17.437 INFO:tasks.workunit.client.1.smithi044.stdout:0/872: readlink d0/d2/dd/d3e/d40/d43/d79/dbb/lc4 0 2024-04-22T23:46:17.440 INFO:tasks.workunit.client.1.smithi044.stdout:6/631: dread d5/d3f/d4c/d40/f71 [0,4194304] 0 2024-04-22T23:46:17.440 INFO:tasks.workunit.client.1.smithi044.stdout:5/754: dwrite d15/d16/d1d/d22/d2a/d3a/f4b [4194304,4194304] 0 2024-04-22T23:46:17.440 INFO:tasks.workunit.client.1.smithi044.stdout:1/740: symlink d1/d8/d34/da1/dab/de9/l110 0 2024-04-22T23:46:17.441 INFO:tasks.workunit.client.1.smithi044.stdout:5/755: write d15/f94 [184802,108824] 0 2024-04-22T23:46:17.443 INFO:tasks.workunit.client.1.smithi044.stdout:2/602: creat d8/d19/d36/d93/dbf/fd5 x:0 0 0 2024-04-22T23:46:17.446 INFO:tasks.workunit.client.1.smithi044.stdout:4/776: rename d5/db/f21 to d5/d31/d44/d7e/daa/f101 0 2024-04-22T23:46:17.449 INFO:tasks.workunit.client.1.smithi044.stdout:0/873: mknod d0/d2/dd/d3e/d40/d43/d79/d12a/c13b 0 2024-04-22T23:46:17.450 INFO:tasks.workunit.client.1.smithi044.stdout:8/771: getdents db/ddd/df2 0 2024-04-22T23:46:17.456 INFO:tasks.workunit.client.1.smithi044.stdout:6/632: dread d5/d3f/d4c/d23/f9b [0,4194304] 0 2024-04-22T23:46:17.456 INFO:tasks.workunit.client.1.smithi044.stdout:1/741: write d1/d8/d20/f2f [947136,42015] 0 2024-04-22T23:46:17.456 INFO:tasks.workunit.client.1.smithi044.stdout:4/777: dwrite d5/db/d16/d4d/dad/dc5/f100 [0,4194304] 0 2024-04-22T23:46:17.463 INFO:tasks.workunit.client.1.smithi044.stdout:5/756: creat d15/d16/d1d/d22/d4f/d68/d76/ff6 x:0 0 0 2024-04-22T23:46:17.468 INFO:tasks.workunit.client.1.smithi044.stdout:6/633: dwrite d5/dd/d67/f47 [0,4194304] 0 2024-04-22T23:46:17.468 INFO:tasks.workunit.client.1.smithi044.stdout:5/757: read d15/d16/d1d/d22/d4f/d68/fc3 [2324574,99452] 0 2024-04-22T23:46:17.473 INFO:tasks.workunit.client.1.smithi044.stdout:2/603: symlink d8/d19/d1c/daf/dd0/ld6 0 2024-04-22T23:46:17.478 INFO:tasks.workunit.client.1.smithi044.stdout:6/634: rmdir d5/d3f/d4c/d4f 39 2024-04-22T23:46:17.481 INFO:tasks.workunit.client.1.smithi044.stdout:1/742: dwrite d1/d8/d1c/d7b/d80/d46/fc9 [0,4194304] 0 2024-04-22T23:46:17.488 INFO:tasks.workunit.client.1.smithi044.stdout:5/758: symlink d15/d16/d1d/d22/d2a/d3a/d43/da1/lf7 0 2024-04-22T23:46:17.492 INFO:tasks.workunit.client.1.smithi044.stdout:4/778: creat d5/d31/f102 x:0 0 0 2024-04-22T23:46:17.499 INFO:tasks.workunit.client.1.smithi044.stdout:0/874: getdents d0/d2/dd/d3e 0 2024-04-22T23:46:17.503 INFO:tasks.workunit.client.1.smithi044.stdout:6/635: getdents d5/d3f/d50/d79/d96 0 2024-04-22T23:46:17.507 INFO:tasks.workunit.client.1.smithi044.stdout:4/779: symlink d5/db/d16/d4d/l103 0 2024-04-22T23:46:17.508 INFO:tasks.workunit.client.1.smithi044.stdout:1/743: write d1/d8/d20/d47/f85 [1505590,125351] 0 2024-04-22T23:46:17.510 INFO:tasks.workunit.client.1.smithi044.stdout:0/875: dwrite d0/d2/dd/d3e/d40/d43/ff8 [0,4194304] 0 2024-04-22T23:46:17.512 INFO:tasks.workunit.client.1.smithi044.stdout:2/604: getdents d8/d19/d1c 0 2024-04-22T23:46:17.514 INFO:tasks.workunit.client.1.smithi044.stdout:5/759: dwrite d15/d16/d1d/d22/d4f/dc9/d8d/fc8 [0,4194304] 0 2024-04-22T23:46:17.516 INFO:tasks.workunit.client.1.smithi044.stdout:1/744: write d1/d8/d20/d47/d92/f10c [310396,126580] 0 2024-04-22T23:46:17.517 INFO:tasks.workunit.client.1.smithi044.stdout:0/876: truncate d0/dd6/f119 253427 0 2024-04-22T23:46:17.521 INFO:tasks.workunit.client.1.smithi044.stdout:4/780: mknod d5/d31/d24/d38/d6a/d75/c104 0 2024-04-22T23:46:17.522 INFO:tasks.workunit.client.1.smithi044.stdout:6/636: creat d5/dd/fc5 x:0 0 0 2024-04-22T23:46:17.525 INFO:tasks.workunit.client.1.smithi044.stdout:0/877: truncate d0/d2e/d2f/d77/fc6 668321 0 2024-04-22T23:46:17.529 INFO:tasks.workunit.client.1.smithi044.stdout:5/760: write d15/d16/d1d/d22/d4f/d68/f6c [2867691,112708] 0 2024-04-22T23:46:17.530 INFO:tasks.workunit.client.1.smithi044.stdout:1/745: dwrite d1/d8/d1c/d7b/d80/d4b/f5a [0,4194304] 0 2024-04-22T23:46:17.532 INFO:tasks.workunit.client.1.smithi044.stdout:6/637: creat d5/d3f/d50/d79/d96/fc6 x:0 0 0 2024-04-22T23:46:17.533 INFO:tasks.workunit.client.1.smithi044.stdout:1/746: write d1/d8/d1c/d7b/d80/fe4 [708818,109137] 0 2024-04-22T23:46:17.534 INFO:tasks.workunit.client.1.smithi044.stdout:0/878: fsync d0/d2/dd/f13 0 2024-04-22T23:46:17.535 INFO:tasks.workunit.client.1.smithi044.stdout:0/879: chown d0/d2e/d2f/f4d 38711 1 2024-04-22T23:46:17.537 INFO:tasks.workunit.client.1.smithi044.stdout:4/781: mknod d5/d31/c105 0 2024-04-22T23:46:17.537 INFO:tasks.workunit.client.1.smithi044.stdout:5/761: mkdir d15/d16/d1d/d22/d81/df8 0 2024-04-22T23:46:17.540 INFO:tasks.workunit.client.1.smithi044.stdout:5/762: chown d15/d16/d1d/d22/d2a/d3a/f96 0 1 2024-04-22T23:46:17.554 INFO:tasks.workunit.client.1.smithi044.stdout:0/880: mknod d0/d4c/c13c 0 2024-04-22T23:46:17.556 INFO:tasks.workunit.client.1.smithi044.stdout:4/782: mkdir d5/db/d106 0 2024-04-22T23:46:17.560 INFO:tasks.workunit.client.1.smithi044.stdout:1/747: creat d1/d8/f111 x:0 0 0 2024-04-22T23:46:17.566 INFO:tasks.workunit.client.1.smithi044.stdout:0/881: dread d0/d2e/d2f/d77/fc6 [0,4194304] 0 2024-04-22T23:46:17.567 INFO:tasks.workunit.client.1.smithi044.stdout:6/638: dwrite d5/dd/f9d [0,4194304] 0 2024-04-22T23:46:17.570 INFO:tasks.workunit.client.1.smithi044.stdout:4/783: dwrite d5/d31/d44/d20/d93/fea [0,4194304] 0 2024-04-22T23:46:17.570 INFO:tasks.workunit.client.1.smithi044.stdout:5/763: dwrite d15/d5d/db5/dc0/f2e [0,4194304] 0 2024-04-22T23:46:17.581 INFO:tasks.workunit.client.1.smithi044.stdout:5/764: mkdir d15/d3c/df9 0 2024-04-22T23:46:17.582 INFO:tasks.workunit.client.1.smithi044.stdout:1/748: write d1/f1b [207369,36769] 0 2024-04-22T23:46:17.582 INFO:tasks.workunit.client.1.smithi044.stdout:9/571: sync 2024-04-22T23:46:17.582 INFO:tasks.workunit.client.1.smithi044.stdout:3/749: sync 2024-04-22T23:46:17.586 INFO:tasks.workunit.client.1.smithi044.stdout:9/572: write d2/d6/d42/daa/fae [584445,117828] 0 2024-04-22T23:46:17.586 INFO:tasks.workunit.client.1.smithi044.stdout:1/749: write d1/d8/d34/da1/dab/fe8 [409179,69179] 0 2024-04-22T23:46:17.591 INFO:tasks.workunit.client.1.smithi044.stdout:3/750: rmdir d9 39 2024-04-22T23:46:17.592 INFO:tasks.workunit.client.1.smithi044.stdout:9/573: mknod d2/dc/d1d/d34/d38/d5c/d69/cbe 0 2024-04-22T23:46:17.593 INFO:tasks.workunit.client.1.smithi044.stdout:9/574: fsync d2/d9/f5a 0 2024-04-22T23:46:17.595 INFO:tasks.workunit.client.1.smithi044.stdout:3/751: write d9/db/d15/d54/d3c/f94 [950741,31937] 0 2024-04-22T23:46:17.595 INFO:tasks.workunit.client.1.smithi044.stdout:9/575: read d2/f4 [1279413,82145] 0 2024-04-22T23:46:17.595 INFO:tasks.workunit.client.1.smithi044.stdout:1/750: getdents d1/d8/d20 0 2024-04-22T23:46:17.600 INFO:tasks.workunit.client.1.smithi044.stdout:0/882: dread d0/d2e/d2f/f51 [0,4194304] 0 2024-04-22T23:46:17.602 INFO:tasks.workunit.client.1.smithi044.stdout:1/751: rename d1/d8/d1c/d7b/d80/d4b/d9c to d1/d8/d34/d112 0 2024-04-22T23:46:17.605 INFO:tasks.workunit.client.1.smithi044.stdout:9/576: symlink d2/dc/d1d/d34/d8e/dbd/lbf 0 2024-04-22T23:46:17.609 INFO:tasks.workunit.client.1.smithi044.stdout:9/577: symlink d2/dc/d51/lc0 0 2024-04-22T23:46:17.612 INFO:tasks.workunit.client.1.smithi044.stdout:9/578: write d2/dc/d1d/d3f/f4c [4912469,114368] 0 2024-04-22T23:46:17.612 INFO:tasks.workunit.client.1.smithi044.stdout:5/765: dwrite d15/d16/f39 [4194304,4194304] 0 2024-04-22T23:46:17.613 INFO:tasks.workunit.client.1.smithi044.stdout:3/752: write d9/f9b [762218,86334] 0 2024-04-22T23:46:17.616 INFO:tasks.workunit.client.1.smithi044.stdout:0/883: dread d0/d4c/d59/d5a/fd3 [0,4194304] 0 2024-04-22T23:46:17.617 INFO:tasks.workunit.client.1.smithi044.stdout:1/752: getdents d1/d8/d34/d112 0 2024-04-22T23:46:17.625 INFO:tasks.workunit.client.1.smithi044.stdout:5/766: rename d15/d16/f39 to d15/d16/d1d/d22/d2a/d3a/d43/d80/ffa 0 2024-04-22T23:46:17.627 INFO:tasks.workunit.client.1.smithi044.stdout:5/767: write d15/d16/d1d/d22/d2a/d3a/d43/f4c [4587876,21554] 0 2024-04-22T23:46:17.628 INFO:tasks.workunit.client.1.smithi044.stdout:0/884: creat d0/d2/d4b/dad/f13d x:0 0 0 2024-04-22T23:46:17.633 INFO:tasks.workunit.client.1.smithi044.stdout:5/768: creat d15/d16/d1d/daf/ffb x:0 0 0 2024-04-22T23:46:17.633 INFO:tasks.workunit.client.1.smithi044.stdout:5/769: chown d15/f30 1685513 1 2024-04-22T23:46:17.636 INFO:tasks.workunit.client.1.smithi044.stdout:5/770: chown d15/d5d/db5/dc0/d38/f50 16284 1 2024-04-22T23:46:17.638 INFO:tasks.workunit.client.1.smithi044.stdout:9/579: getdents d2/dc/d1d/d34/d38/d5c/d7b 0 2024-04-22T23:46:17.641 INFO:tasks.workunit.client.1.smithi044.stdout:5/771: symlink d15/d5d/db5/dc0/lfc 0 2024-04-22T23:46:17.644 INFO:tasks.workunit.client.1.smithi044.stdout:1/753: write d1/d8/d1c/d23/d3e/f7e [756861,55836] 0 2024-04-22T23:46:17.646 INFO:tasks.workunit.client.1.smithi044.stdout:0/885: getdents d0/d2/dd/d3e/d40/d43/de4/d126 0 2024-04-22T23:46:17.646 INFO:tasks.workunit.client.1.smithi044.stdout:3/753: dread d9/d12/f92 [0,4194304] 0 2024-04-22T23:46:17.647 INFO:tasks.workunit.client.1.smithi044.stdout:5/772: creat d15/d5d/d64/d6f/ffd x:0 0 0 2024-04-22T23:46:17.648 INFO:tasks.workunit.client.1.smithi044.stdout:9/580: dwrite d2/dc/d94/f50 [0,4194304] 0 2024-04-22T23:46:17.653 INFO:tasks.workunit.client.1.smithi044.stdout:3/754: mknod d9/db/d33/d5d/d64/cf7 0 2024-04-22T23:46:17.657 INFO:tasks.workunit.client.1.smithi044.stdout:3/755: read d9/db/d15/d25/d4c/f6a [387631,130065] 0 2024-04-22T23:46:17.657 INFO:tasks.workunit.client.1.smithi044.stdout:0/886: rename d0/d2/dd/d3e/d40/d43/d7b/c69 to d0/d2e/d2f/d77/c13e 0 2024-04-22T23:46:17.660 INFO:tasks.workunit.client.1.smithi044.stdout:5/773: dwrite d15/d16/d1d/d22/d2a/d3a/d43/f4c [8388608,4194304] 0 2024-04-22T23:46:17.662 INFO:tasks.workunit.client.1.smithi044.stdout:9/581: creat d2/dc/d1d/d3f/fc1 x:0 0 0 2024-04-22T23:46:17.664 INFO:tasks.workunit.client.1.smithi044.stdout:1/754: write d1/d8/d20/d47/d92/fa2 [828543,119400] 0 2024-04-22T23:46:17.666 INFO:tasks.workunit.client.1.smithi044.stdout:3/756: rmdir d9/d12/db1 39 2024-04-22T23:46:17.668 INFO:tasks.workunit.client.1.smithi044.stdout:1/755: chown d1/d8/d1c/d23/d3e/dde/ff7 166132139 1 2024-04-22T23:46:17.668 INFO:tasks.workunit.client.1.smithi044.stdout:1/756: chown d1 13039117 1 2024-04-22T23:46:17.670 INFO:tasks.workunit.client.1.smithi044.stdout:9/582: read d2/d9/f5a [1415822,110100] 0 2024-04-22T23:46:17.674 INFO:tasks.workunit.client.1.smithi044.stdout:1/757: mknod d1/d8/d20/d47/d4a/d60/c113 0 2024-04-22T23:46:17.676 INFO:tasks.workunit.client.1.smithi044.stdout:3/757: dwrite d9/db/d15/f85 [0,4194304] 0 2024-04-22T23:46:17.686 INFO:tasks.workunit.client.1.smithi044.stdout:1/758: dwrite d1/d8/d1c/d23/d3e/dde/f104 [0,4194304] 0 2024-04-22T23:46:17.694 INFO:tasks.workunit.client.1.smithi044.stdout:1/759: symlink d1/d8/d1c/d7b/d80/dfd/l114 0 2024-04-22T23:46:17.695 INFO:tasks.workunit.client.1.smithi044.stdout:5/774: dread d15/d5d/db5/dc0/f5a [0,4194304] 0 2024-04-22T23:46:17.698 INFO:tasks.workunit.client.1.smithi044.stdout:9/583: write d2/dc/d94/f55 [199862,34047] 0 2024-04-22T23:46:17.700 INFO:tasks.workunit.client.1.smithi044.stdout:5/775: getdents d15/d16/d1d/d22/d81/df8 0 2024-04-22T23:46:17.702 INFO:tasks.workunit.client.1.smithi044.stdout:0/887: dwrite d0/d2/dd/d3e/f6e [0,4194304] 0 2024-04-22T23:46:17.708 INFO:tasks.workunit.client.1.smithi044.stdout:9/584: creat d2/dc/d94/fc2 x:0 0 0 2024-04-22T23:46:17.708 INFO:tasks.workunit.client.1.smithi044.stdout:3/758: write d9/db/d15/f63 [2744605,15990] 0 2024-04-22T23:46:17.708 INFO:tasks.workunit.client.1.smithi044.stdout:0/888: write d0/d2e/d33/f3a [1855783,100153] 0 2024-04-22T23:46:17.710 INFO:tasks.workunit.client.1.smithi044.stdout:1/760: dwrite d1/d8/d20/d106/f109 [0,4194304] 0 2024-04-22T23:46:17.712 INFO:tasks.workunit.client.1.smithi044.stdout:1/761: write d1/d8/f19 [1216955,109055] 0 2024-04-22T23:46:17.712 INFO:tasks.workunit.client.1.smithi044.stdout:5/776: mknod d15/d16/d1d/d22/d2a/d3a/d43/da1/cfe 0 2024-04-22T23:46:17.721 INFO:tasks.workunit.client.1.smithi044.stdout:0/889: creat d0/d2e/dbc/f13f x:0 0 0 2024-04-22T23:46:17.724 INFO:tasks.workunit.client.1.smithi044.stdout:1/762: symlink d1/d8/d34/da1/dab/de9/db9/l115 0 2024-04-22T23:46:17.726 INFO:tasks.workunit.client.1.smithi044.stdout:3/759: mkdir d9/d14/d7d/df8 0 2024-04-22T23:46:17.733 INFO:tasks.workunit.client.1.smithi044.stdout:1/763: truncate d1/d8/d1c/f59 732283 0 2024-04-22T23:46:17.737 INFO:tasks.workunit.client.1.smithi044.stdout:3/760: mkdir d9/d12/db1/df9 0 2024-04-22T23:46:17.738 INFO:tasks.workunit.client.1.smithi044.stdout:9/585: dread d2/d6/fa9 [0,4194304] 0 2024-04-22T23:46:17.739 INFO:tasks.workunit.client.1.smithi044.stdout:3/761: rename d9/db/d15 to d9/db/d15/d25/dfa 22 2024-04-22T23:46:17.739 INFO:tasks.workunit.client.1.smithi044.stdout:1/764: unlink d1/d8/d34/d112/c9f 0 2024-04-22T23:46:17.740 INFO:tasks.workunit.client.1.smithi044.stdout:3/762: readlink d9/db/d33/l53 0 2024-04-22T23:46:17.741 INFO:tasks.workunit.client.1.smithi044.stdout:5/777: dread d15/d16/d1d/f42 [0,4194304] 0 2024-04-22T23:46:17.741 INFO:tasks.workunit.client.1.smithi044.stdout:3/763: chown d9/db/d15/d25/d5b/d52/f6f 400114297 1 2024-04-22T23:46:17.747 INFO:tasks.workunit.client.1.smithi044.stdout:1/765: mkdir d1/d8/d34/da1/dab/de9/d116 0 2024-04-22T23:46:17.749 INFO:tasks.workunit.client.1.smithi044.stdout:0/890: truncate d0/d2/fa 1298576 0 2024-04-22T23:46:17.750 INFO:tasks.workunit.client.1.smithi044.stdout:3/764: fdatasync d9/db/d15/d25/d5b/feb 0 2024-04-22T23:46:17.751 INFO:tasks.workunit.client.1.smithi044.stdout:1/766: dread d1/d8/d1c/d7b/fb6 [0,4194304] 0 2024-04-22T23:46:17.756 INFO:tasks.workunit.client.1.smithi044.stdout:9/586: creat d2/dc/d94/fc3 x:0 0 0 2024-04-22T23:46:17.759 INFO:tasks.workunit.client.1.smithi044.stdout:3/765: dwrite d9/db/d15/d25/fe2 [0,4194304] 0 2024-04-22T23:46:17.759 INFO:tasks.workunit.client.1.smithi044.stdout:9/587: dread d2/d6/d42/d8b/f98 [0,4194304] 0 2024-04-22T23:46:17.783 INFO:tasks.workunit.client.1.smithi044.stdout:8/772: sync 2024-04-22T23:46:17.783 INFO:tasks.workunit.client.1.smithi044.stdout:7/748: sync 2024-04-22T23:46:17.788 INFO:tasks.workunit.client.1.smithi044.stdout:3/766: fdatasync d9/dce/fef 0 2024-04-22T23:46:17.790 INFO:tasks.workunit.client.1.smithi044.stdout:3/767: chown d9/db/d33/l53 533854 1 2024-04-22T23:46:17.791 INFO:tasks.workunit.client.1.smithi044.stdout:9/588: creat d2/dc/d1d/d34/d38/d5c/d69/d75/fc4 x:0 0 0 2024-04-22T23:46:17.795 INFO:tasks.workunit.client.1.smithi044.stdout:0/891: dwrite d0/d4c/d59/fb3 [0,4194304] 0 2024-04-22T23:46:17.795 INFO:tasks.workunit.client.1.smithi044.stdout:3/768: rename d9/db/d33/d5d/d70/dd8 to d9/db/d33/d5d/d70/dd8/dfb 22 2024-04-22T23:46:17.803 INFO:tasks.workunit.client.1.smithi044.stdout:1/767: truncate d1/d8/d20/d45/f97 928599 0 2024-04-22T23:46:17.803 INFO:tasks.workunit.client.1.smithi044.stdout:3/769: dwrite d9/db/d15/d25/f6d [0,4194304] 0 2024-04-22T23:46:17.808 INFO:tasks.workunit.client.1.smithi044.stdout:9/589: write d2/f7d [1968676,65697] 0 2024-04-22T23:46:17.815 INFO:tasks.workunit.client.1.smithi044.stdout:0/892: rename d0/d2e/d2f/f5e to d0/d2/d4b/dad/dc8/d10e/d11b/f140 0 2024-04-22T23:46:17.816 INFO:tasks.workunit.client.1.smithi044.stdout:0/893: truncate d0/d60/fd7 534550 0 2024-04-22T23:46:17.817 INFO:tasks.workunit.client.1.smithi044.stdout:3/770: unlink d9/db/d15/d2e/f32 0 2024-04-22T23:46:17.822 INFO:tasks.workunit.client.1.smithi044.stdout:9/590: rmdir d2/dc/d1d/d34/d38/d5c/d69/d75/d78 39 2024-04-22T23:46:17.823 INFO:tasks.workunit.client.1.smithi044.stdout:8/773: dread db/d14/f46 [0,4194304] 0 2024-04-22T23:46:17.826 INFO:tasks.workunit.client.1.smithi044.stdout:3/771: rmdir d9/dce 39 2024-04-22T23:46:17.826 INFO:tasks.workunit.client.1.smithi044.stdout:7/749: dwrite dd/d1f/f77 [0,4194304] 0 2024-04-22T23:46:17.836 INFO:tasks.workunit.client.1.smithi044.stdout:9/591: creat d2/d6/d42/d8b/fc5 x:0 0 0 2024-04-22T23:46:17.839 INFO:tasks.workunit.client.1.smithi044.stdout:8/774: rename db/d6c/c76 to db/d52/d9c/dad/dc4/c115 0 2024-04-22T23:46:17.840 INFO:tasks.workunit.client.1.smithi044.stdout:7/750: truncate f8 1554942 0 2024-04-22T23:46:17.844 INFO:tasks.workunit.client.1.smithi044.stdout:7/751: dread dd/d98/de8/f107 [0,4194304] 0 2024-04-22T23:46:17.847 INFO:tasks.workunit.client.1.smithi044.stdout:0/894: truncate d0/d2/d4b/dad/f10a 3291333 0 2024-04-22T23:46:17.848 INFO:tasks.workunit.client.1.smithi044.stdout:0/895: truncate d0/d2e/dbc/f139 12676 0 2024-04-22T23:46:17.849 INFO:tasks.workunit.client.1.smithi044.stdout:1/768: link d1/d8/d20/d45/d75/df2/fc0 d1/d8/f117 0 2024-04-22T23:46:17.850 INFO:tasks.workunit.client.1.smithi044.stdout:0/896: write d0/d2e/d2f/fe3 [990919,35477] 0 2024-04-22T23:46:17.856 INFO:tasks.workunit.client.1.smithi044.stdout:8/775: rename db/d68/d8b/da7/fcd to db/d52/d9c/dad/f116 0 2024-04-22T23:46:17.858 INFO:tasks.workunit.client.1.smithi044.stdout:1/769: dwrite d1/d8/d1c/d7b/d80/d4b/f9a [0,4194304] 0 2024-04-22T23:46:17.860 INFO:tasks.workunit.client.1.smithi044.stdout:3/772: getdents d9/db/d15/d25/d67/d6b/ded 0 2024-04-22T23:46:17.864 INFO:tasks.workunit.client.1.smithi044.stdout:7/752: symlink dd/d99/deb/l118 0 2024-04-22T23:46:17.878 INFO:tasks.workunit.client.1.smithi044.stdout:1/770: symlink d1/d8/d20/d45/d75/dfb/d95/l118 0 2024-04-22T23:46:17.883 INFO:tasks.workunit.client.1.smithi044.stdout:0/897: symlink d0/d2/dd/d3e/d40/d43/dde/dd9/l141 0 2024-04-22T23:46:17.885 INFO:tasks.workunit.client.1.smithi044.stdout:0/898: dread - d0/d2/dd/d3e/d40/d43/f120 zero size 2024-04-22T23:46:17.888 INFO:tasks.workunit.client.1.smithi044.stdout:1/771: dread d1/d8/d1c/d7b/d80/d46/fc9 [0,4194304] 0 2024-04-22T23:46:17.890 INFO:tasks.workunit.client.1.smithi044.stdout:8/776: mknod db/d68/d8b/dca/d10f/c117 0 2024-04-22T23:46:17.894 INFO:tasks.workunit.client.1.smithi044.stdout:0/899: chown d0/d2/dd/d3e/d40/d67/d7c 30721010 1 2024-04-22T23:46:17.899 INFO:tasks.workunit.client.1.smithi044.stdout:0/900: mkdir d0/d4c/d123/d142 0 2024-04-22T23:46:17.909 INFO:tasks.workunit.client.1.smithi044.stdout:0/901: symlink d0/d2/d4b/l143 0 2024-04-22T23:46:17.909 INFO:tasks.workunit.client.1.smithi044.stdout:8/777: link db/d14/dc7/dc9/lf9 db/d14/d84/dc0/l118 0 2024-04-22T23:46:17.912 INFO:tasks.workunit.client.1.smithi044.stdout:9/592: dwrite d2/dc/fb0 [0,4194304] 0 2024-04-22T23:46:17.921 INFO:tasks.workunit.client.1.smithi044.stdout:0/902: dwrite d0/d2e/f70 [0,4194304] 0 2024-04-22T23:46:17.923 INFO:tasks.workunit.client.1.smithi044.stdout:0/903: write d0/d2e/d33/f85 [1753053,21944] 0 2024-04-22T23:46:17.932 INFO:tasks.workunit.client.1.smithi044.stdout:7/753: dwrite dd/d1a/f1d [0,4194304] 0 2024-04-22T23:46:17.946 INFO:tasks.workunit.client.1.smithi044.stdout:7/754: dwrite dd/d1f/d24/f19 [0,4194304] 0 2024-04-22T23:46:17.947 INFO:tasks.workunit.client.1.smithi044.stdout:9/593: dread d2/dc/f27 [0,4194304] 0 2024-04-22T23:46:17.949 INFO:tasks.workunit.client.1.smithi044.stdout:9/594: fsync d2/d9/f6a 0 2024-04-22T23:46:17.951 INFO:tasks.workunit.client.1.smithi044.stdout:8/778: dwrite db/d6c/fa0 [0,4194304] 0 2024-04-22T23:46:17.954 INFO:tasks.workunit.client.1.smithi044.stdout:8/779: chown db/d14/d84/fb4 173279814 1 2024-04-22T23:46:17.958 INFO:tasks.workunit.client.1.smithi044.stdout:7/755: truncate dd/d1f/d24/f39 1684986 0 2024-04-22T23:46:17.959 INFO:tasks.workunit.client.1.smithi044.stdout:9/595: write d2/dc/f32 [1246610,48553] 0 2024-04-22T23:46:17.960 INFO:tasks.workunit.client.1.smithi044.stdout:0/904: dread d0/d2/dd/d3e/d40/d43/ff9 [0,4194304] 0 2024-04-22T23:46:17.961 INFO:tasks.workunit.client.1.smithi044.stdout:9/596: chown d2/dc/d1d/d34/d38/d5c/d69/d7a/c8c 3616 1 2024-04-22T23:46:17.961 INFO:tasks.workunit.client.1.smithi044.stdout:0/905: readlink d0/d2/d4b/lc0 0 2024-04-22T23:46:17.965 INFO:tasks.workunit.client.1.smithi044.stdout:2/605: sync 2024-04-22T23:46:17.968 INFO:tasks.workunit.client.1.smithi044.stdout:2/606: read d8/d19/d96/fad [295672,2694] 0 2024-04-22T23:46:17.969 INFO:tasks.workunit.client.1.smithi044.stdout:2/607: chown d8/d60/l8a 2291702 1 2024-04-22T23:46:17.972 INFO:tasks.workunit.client.1.smithi044.stdout:0/906: fdatasync d0/d2e/d2f/d77/f93 0 2024-04-22T23:46:17.972 INFO:tasks.workunit.client.1.smithi044.stdout:0/907: chown d0/d2/f8e 92250 1 2024-04-22T23:46:17.977 INFO:tasks.workunit.client.1.smithi044.stdout:2/608: link d8/d27/d5b/d74/d81/fd3 d8/d19/d36/d44/fd7 0 2024-04-22T23:46:17.978 INFO:tasks.workunit.client.1.smithi044.stdout:2/609: write d8/f11 [1493607,59361] 0 2024-04-22T23:46:17.978 INFO:tasks.workunit.client.1.smithi044.stdout:0/908: mknod d0/d2/dd/d3e/d40/d43/de4/c144 0 2024-04-22T23:46:17.981 INFO:tasks.workunit.client.1.smithi044.stdout:0/909: write d0/d2/dd/d3e/d40/fb0 [892562,94024] 0 2024-04-22T23:46:17.982 INFO:tasks.workunit.client.1.smithi044.stdout:2/610: mkdir d8/d19/d36/d44/dd8 0 2024-04-22T23:46:17.984 INFO:tasks.workunit.client.1.smithi044.stdout:0/910: chown d0/d2e/d2f/d77/c13e 37719 1 2024-04-22T23:46:17.988 INFO:tasks.workunit.client.1.smithi044.stdout:0/911: unlink d0/d2/dd/d3e/d40/d43/d7b/l83 0 2024-04-22T23:46:17.991 INFO:tasks.workunit.client.1.smithi044.stdout:0/912: read d0/d2e/d2f/d77/f9d [1131333,91071] 0 2024-04-22T23:46:17.992 INFO:tasks.workunit.client.1.smithi044.stdout:8/780: write db/f34 [3723760,126319] 0 2024-04-22T23:46:17.998 INFO:tasks.workunit.client.1.smithi044.stdout:9/597: write d2/dc/f27 [982519,124469] 0 2024-04-22T23:46:18.002 INFO:tasks.workunit.client.1.smithi044.stdout:7/756: dwrite dd/d27/f46 [0,4194304] 0 2024-04-22T23:46:18.011 INFO:tasks.workunit.client.1.smithi044.stdout:2/611: dwrite d8/d19/d96/f9b [0,4194304] 0 2024-04-22T23:46:18.042 INFO:tasks.workunit.client.1.smithi044.stdout:0/913: dread d0/d2/dd/d3e/d40/d67/ff5 [0,4194304] 0 2024-04-22T23:46:18.049 INFO:tasks.workunit.client.1.smithi044.stdout:9/598: fdatasync d2/dc/d1d/d34/f49 0 2024-04-22T23:46:18.053 INFO:tasks.workunit.client.1.smithi044.stdout:8/781: dwrite db/d68/d74/ffb [0,4194304] 0 2024-04-22T23:46:18.055 INFO:tasks.workunit.client.1.smithi044.stdout:0/914: unlink d0/d2/d4b/f74 0 2024-04-22T23:46:18.058 INFO:tasks.workunit.client.1.smithi044.stdout:9/599: symlink d2/d6/lc6 0 2024-04-22T23:46:18.060 INFO:tasks.workunit.client.1.smithi044.stdout:9/600: truncate d2/dc/d1d/d34/d38/d4a/d72/fb1 564897 0 2024-04-22T23:46:18.061 INFO:tasks.workunit.client.1.smithi044.stdout:7/757: link dd/cf9 dd/df7/c119 0 2024-04-22T23:46:18.065 INFO:tasks.workunit.client.1.smithi044.stdout:2/612: rename d8/d19/d36/d44/fd7 to d8/fd9 0 2024-04-22T23:46:18.069 INFO:tasks.workunit.client.1.smithi044.stdout:0/915: creat d0/d4c/d123/f145 x:0 0 0 2024-04-22T23:46:18.070 INFO:tasks.workunit.client.1.smithi044.stdout:9/601: mkdir d2/dc/d1d/d34/d38/d4a/dc7 0 2024-04-22T23:46:18.073 INFO:tasks.workunit.client.1.smithi044.stdout:5/778: sync 2024-04-22T23:46:18.073 INFO:tasks.workunit.client.1.smithi044.stdout:4/784: sync 2024-04-22T23:46:18.073 INFO:tasks.workunit.client.1.smithi044.stdout:6/639: sync 2024-04-22T23:46:18.073 INFO:tasks.workunit.client.1.smithi044.stdout:8/782: mknod db/d14/d72/c119 0 2024-04-22T23:46:18.076 INFO:tasks.workunit.client.1.smithi044.stdout:4/785: dread - d5/d31/d24/d38/d6a/d75/fab zero size 2024-04-22T23:46:18.077 INFO:tasks.workunit.client.1.smithi044.stdout:2/613: truncate d8/d19/d1c/f1d 4530085 0 2024-04-22T23:46:18.082 INFO:tasks.workunit.client.1.smithi044.stdout:2/614: chown d8/d19/f51 170320943 1 2024-04-22T23:46:18.083 INFO:tasks.workunit.client.1.smithi044.stdout:4/786: write d5/db/d16/d4d/f8a [2286633,62593] 0 2024-04-22T23:46:18.083 INFO:tasks.workunit.client.1.smithi044.stdout:7/758: dwrite dd/d1f/d24/f30 [4194304,4194304] 0 2024-04-22T23:46:18.085 INFO:tasks.workunit.client.1.smithi044.stdout:6/640: creat d5/d3f/d4c/d40/d74/fc7 x:0 0 0 2024-04-22T23:46:18.088 INFO:tasks.workunit.client.1.smithi044.stdout:5/779: mknod d15/d5d/d82/df2/cff 0 2024-04-22T23:46:18.091 INFO:tasks.workunit.client.1.smithi044.stdout:8/783: dwrite db/d14/f41 [4194304,4194304] 0 2024-04-22T23:46:18.091 INFO:tasks.workunit.client.1.smithi044.stdout:7/759: truncate dd/d1a/db7/fda 109125 0 2024-04-22T23:46:18.094 INFO:tasks.workunit.client.1.smithi044.stdout:7/760: readlink dd/d1a/d61/l101 0 2024-04-22T23:46:18.095 INFO:tasks.workunit.client.1.smithi044.stdout:9/602: dwrite d2/dc/f18 [4194304,4194304] 0 2024-04-22T23:46:18.095 INFO:tasks.workunit.client.1.smithi044.stdout:0/916: truncate d0/d2/d4b/dad/dc8/d10e/d11b/f140 5175157 0 2024-04-22T23:46:18.100 INFO:tasks.workunit.client.1.smithi044.stdout:6/641: symlink d5/d3f/d4c/d40/d74/lc8 0 2024-04-22T23:46:18.103 INFO:tasks.workunit.client.1.smithi044.stdout:2/615: mknod d8/cda 0 2024-04-22T23:46:18.105 INFO:tasks.workunit.client.1.smithi044.stdout:8/784: mknod db/d14/d9e/c11a 0 2024-04-22T23:46:18.105 INFO:tasks.workunit.client.1.smithi044.stdout:8/785: stat db/d68 0 2024-04-22T23:46:18.109 INFO:tasks.workunit.client.1.smithi044.stdout:4/787: unlink d5/la5 0 2024-04-22T23:46:18.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/603: readlink d2/dc/d94/ld 0 2024-04-22T23:46:18.116 INFO:tasks.workunit.client.1.smithi044.stdout:0/917: rename d0/d2e to d0/d2/dd/d3e/d40/d43/de4/d146 0 2024-04-22T23:46:18.120 INFO:tasks.workunit.client.1.smithi044.stdout:6/642: chown d5/dd/f16 32081460 1 2024-04-22T23:46:18.121 INFO:tasks.workunit.client.1.smithi044.stdout:8/786: mknod db/d14/d84/dc0/c11b 0 2024-04-22T23:46:18.129 INFO:tasks.workunit.client.1.smithi044.stdout:9/604: fdatasync d2/f4 0 2024-04-22T23:46:18.132 INFO:tasks.workunit.client.1.smithi044.stdout:2/616: rename d8/d19/d53 to d8/d19/d36/d46/ddb 0 2024-04-22T23:46:18.136 INFO:tasks.workunit.client.1.smithi044.stdout:5/780: creat d15/d16/f100 x:0 0 0 2024-04-22T23:46:18.136 INFO:tasks.workunit.client.1.smithi044.stdout:6/643: dread d5/d3f/f63 [4194304,4194304] 0 2024-04-22T23:46:18.144 INFO:tasks.workunit.client.1.smithi044.stdout:9/605: truncate d2/dc/d94/f30 1091737 0 2024-04-22T23:46:18.144 INFO:tasks.workunit.client.1.smithi044.stdout:9/606: readlink d2/d9/l60 0 2024-04-22T23:46:18.146 INFO:tasks.workunit.client.1.smithi044.stdout:2/617: creat d8/d60/dc1/fdc x:0 0 0 2024-04-22T23:46:18.150 INFO:tasks.workunit.client.1.smithi044.stdout:4/788: write d5/db/d16/fc9 [142120,98870] 0 2024-04-22T23:46:18.153 INFO:tasks.workunit.client.1.smithi044.stdout:9/607: dwrite d2/dc/f27 [4194304,4194304] 0 2024-04-22T23:46:18.156 INFO:tasks.workunit.client.1.smithi044.stdout:9/608: dread d2/dc/d1d/d34/d38/d4a/d72/fb1 [0,4194304] 0 2024-04-22T23:46:18.157 INFO:tasks.workunit.client.1.smithi044.stdout:7/761: getdents dd/d55/d5e/d60/d75 0 2024-04-22T23:46:18.164 INFO:tasks.workunit.client.1.smithi044.stdout:0/918: link d0/d2/d4b/dad/dc8/cca d0/d4c/d123/d142/c147 0 2024-04-22T23:46:18.166 INFO:tasks.workunit.client.1.smithi044.stdout:8/787: dwrite db/d14/dc7/fe7 [0,4194304] 0 2024-04-22T23:46:18.172 INFO:tasks.workunit.client.1.smithi044.stdout:7/762: dwrite dd/ff [4194304,4194304] 0 2024-04-22T23:46:18.174 INFO:tasks.workunit.client.1.smithi044.stdout:9/609: stat d2/dc/d1d/d34/l64 0 2024-04-22T23:46:18.177 INFO:tasks.workunit.client.1.smithi044.stdout:5/781: write d15/d16/f7b [408828,73927] 0 2024-04-22T23:46:18.180 INFO:tasks.workunit.client.1.smithi044.stdout:4/789: mkdir d5/db/d16/d4d/dad/d107 0 2024-04-22T23:46:18.184 INFO:tasks.workunit.client.1.smithi044.stdout:2/618: rename d8/d19/d5d/db0/dc7 to d8/d19/d36/d46/ddb/ddd 0 2024-04-22T23:46:18.189 INFO:tasks.workunit.client.1.smithi044.stdout:0/919: creat d0/d2/d4b/dad/dc8/d10e/d11b/f148 x:0 0 0 2024-04-22T23:46:18.193 INFO:tasks.workunit.client.1.smithi044.stdout:0/920: fsync d0/d2/dd/d3e/d40/d43/de4/d146/d2f/f4d 0 2024-04-22T23:46:18.195 INFO:tasks.workunit.client.1.smithi044.stdout:4/790: rename d5/db/d16/l2a to d5/d31/d44/d20/d93/l108 0 2024-04-22T23:46:18.199 INFO:tasks.workunit.client.1.smithi044.stdout:7/763: write dd/d1f/f34 [814370,52827] 0 2024-04-22T23:46:18.202 INFO:tasks.workunit.client.1.smithi044.stdout:9/610: mknod d2/dc/cc8 0 2024-04-22T23:46:18.203 INFO:tasks.workunit.client.1.smithi044.stdout:9/611: fdatasync d2/dc/f32 0 2024-04-22T23:46:18.204 INFO:tasks.workunit.client.1.smithi044.stdout:5/782: symlink d15/d5d/d64/dc5/l101 0 2024-04-22T23:46:18.205 INFO:tasks.workunit.client.1.smithi044.stdout:8/788: write db/d52/d9c/dad/f113 [3027206,121340] 0 2024-04-22T23:46:18.206 INFO:tasks.workunit.client.1.smithi044.stdout:9/612: read d2/dc/d1d/d3f/f6e [8214,32412] 0 2024-04-22T23:46:18.206 INFO:tasks.workunit.client.1.smithi044.stdout:8/789: stat db/d68/daf/fd0 0 2024-04-22T23:46:18.209 INFO:tasks.workunit.client.1.smithi044.stdout:0/921: rename d0/d2/dd/f13 to d0/d4c/d59/d5a/f149 0 2024-04-22T23:46:18.210 INFO:tasks.workunit.client.1.smithi044.stdout:0/922: dread - d0/d2/dd/d3e/d40/d43/f110 zero size 2024-04-22T23:46:18.217 INFO:tasks.workunit.client.1.smithi044.stdout:5/783: unlink d15/d16/d1d/daf/ffb 0 2024-04-22T23:46:18.220 INFO:tasks.workunit.client.1.smithi044.stdout:8/790: creat db/d68/d8b/dca/f11c x:0 0 0 2024-04-22T23:46:18.222 INFO:tasks.workunit.client.1.smithi044.stdout:7/764: rename dd/d1f/d24/ca4 to dd/df7/db8/d115/c11a 0 2024-04-22T23:46:18.225 INFO:tasks.workunit.client.1.smithi044.stdout:0/923: mknod d0/d2/dd/d3e/d40/d43/de4/d146/d2f/d77/c14a 0 2024-04-22T23:46:18.226 INFO:tasks.workunit.client.1.smithi044.stdout:5/784: dwrite d15/f94 [0,4194304] 0 2024-04-22T23:46:18.227 INFO:tasks.workunit.client.1.smithi044.stdout:4/791: dwrite d5/d31/d24/d69/d90/f88 [0,4194304] 0 2024-04-22T23:46:18.229 INFO:tasks.workunit.client.1.smithi044.stdout:4/792: write d5/d31/fce [6948,72999] 0 2024-04-22T23:46:18.234 INFO:tasks.workunit.client.1.smithi044.stdout:4/793: dread - d5/db/d16/d4d/dad/dc5/ff9 zero size 2024-04-22T23:46:18.237 INFO:tasks.workunit.client.1.smithi044.stdout:0/924: unlink d0/d2/cc 0 2024-04-22T23:46:18.239 INFO:tasks.workunit.client.1.smithi044.stdout:9/613: creat d2/dc/d94/fc9 x:0 0 0 2024-04-22T23:46:18.242 INFO:tasks.workunit.client.1.smithi044.stdout:9/614: chown d2/dc/d1d/d34/d38/d5c/d69/d9f/fb9 112783 1 2024-04-22T23:46:18.243 INFO:tasks.workunit.client.1.smithi044.stdout:9/615: dread - d2/d9/f9b zero size 2024-04-22T23:46:18.245 INFO:tasks.workunit.client.1.smithi044.stdout:4/794: rename d5/d31/d24/d38/d84/le9 to d5/db/d16/d4d/dad/l109 0 2024-04-22T23:46:18.246 INFO:tasks.workunit.client.1.smithi044.stdout:5/785: getdents d15/d16/d1d/d22/d4f/dc9 0 2024-04-22T23:46:18.248 INFO:tasks.workunit.client.1.smithi044.stdout:4/795: creat d5/d31/d44/d20/d22/d8d/f10a x:0 0 0 2024-04-22T23:46:18.251 INFO:tasks.workunit.client.1.smithi044.stdout:9/616: dwrite d2/d6/f43 [0,4194304] 0 2024-04-22T23:46:18.251 INFO:tasks.workunit.client.1.smithi044.stdout:5/786: rename d15/d5d/d64/lcc to d15/d5d/d64/d6f/l102 0 2024-04-22T23:46:18.254 INFO:tasks.workunit.client.1.smithi044.stdout:5/787: read d15/d5d/d64/d6f/d7f/fbb [144038,71159] 0 2024-04-22T23:46:18.256 INFO:tasks.workunit.client.1.smithi044.stdout:5/788: chown d15/d5d/d64/l7d 6527 1 2024-04-22T23:46:18.258 INFO:tasks.workunit.client.1.smithi044.stdout:9/617: truncate d2/dc/d1d/d3f/f48 4812581 0 2024-04-22T23:46:18.261 INFO:tasks.workunit.client.1.smithi044.stdout:9/618: read - d2/dc/d1d/d34/d38/d5c/d69/d75/fc4 zero size 2024-04-22T23:46:18.261 INFO:tasks.workunit.client.1.smithi044.stdout:5/789: creat d15/d5d/db5/dc0/f103 x:0 0 0 2024-04-22T23:46:18.262 INFO:tasks.workunit.client.1.smithi044.stdout:9/619: dread - d2/dc/d1d/d3f/fb4 zero size 2024-04-22T23:46:18.263 INFO:tasks.workunit.client.1.smithi044.stdout:9/620: stat d2/d6/fa8 0 2024-04-22T23:46:18.268 INFO:tasks.workunit.client.1.smithi044.stdout:4/796: rename d5/d31/d24/c47 to d5/c10b 0 2024-04-22T23:46:18.269 INFO:tasks.workunit.client.1.smithi044.stdout:7/765: dwrite dd/d55/d5e/d60/d68/f73 [0,4194304] 0 2024-04-22T23:46:18.270 INFO:tasks.workunit.client.1.smithi044.stdout:5/790: mknod d15/d5d/d64/c104 0 2024-04-22T23:46:18.274 INFO:tasks.workunit.client.1.smithi044.stdout:0/925: write d0/d2/dd/d3e/d40/d43/de4/d146/d2f/d77/f9d [3123089,60591] 0 2024-04-22T23:46:18.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/621: unlink d2/dc/d1d/d34/f49 0 2024-04-22T23:46:18.278 INFO:tasks.workunit.client.1.smithi044.stdout:0/926: chown d0/d2/d4b/dad/d125/f132 155377 1 2024-04-22T23:46:18.279 INFO:tasks.workunit.client.1.smithi044.stdout:4/797: creat d5/d31/d44/d20/d93/f10c x:0 0 0 2024-04-22T23:46:18.280 INFO:tasks.workunit.client.1.smithi044.stdout:8/791: dwrite db/f69 [0,4194304] 0 2024-04-22T23:46:18.282 INFO:tasks.workunit.client.1.smithi044.stdout:5/791: rename d15/d16/lcd to d15/d5d/d64/de0/l105 0 2024-04-22T23:46:18.283 INFO:tasks.workunit.client.1.smithi044.stdout:4/798: write d5/db/d16/d4d/dad/dc5/dd9/fed [671562,35599] 0 2024-04-22T23:46:18.284 INFO:tasks.workunit.client.1.smithi044.stdout:4/799: chown d5/l6e 2144309423 1 2024-04-22T23:46:18.286 INFO:tasks.workunit.client.1.smithi044.stdout:7/766: mknod dd/d55/d5e/d60/d68/d94/dab/dd5/c11b 0 2024-04-22T23:46:18.292 INFO:tasks.workunit.client.1.smithi044.stdout:8/792: chown db/d14/d24/c62 13 1 2024-04-22T23:46:18.293 INFO:tasks.workunit.client.1.smithi044.stdout:9/622: read d2/dc/d1d/d34/d38/d5c/f76 [1413488,78452] 0 2024-04-22T23:46:18.295 INFO:tasks.workunit.client.1.smithi044.stdout:0/927: rename d0/d2/dd/d3e/d40/d43/de4/d146/d33 to d0/d2/dd/d3e/d14b 0 2024-04-22T23:46:18.296 INFO:tasks.workunit.client.1.smithi044.stdout:0/928: chown d0/d4c/d59/d5a 321260 1 2024-04-22T23:46:18.296 INFO:tasks.workunit.client.1.smithi044.stdout:5/792: creat d15/d5d/d64/d6f/f106 x:0 0 0 2024-04-22T23:46:18.299 INFO:tasks.workunit.client.1.smithi044.stdout:9/623: dread d2/dc/fb0 [0,4194304] 0 2024-04-22T23:46:18.299 INFO:tasks.workunit.client.1.smithi044.stdout:4/800: rmdir d5/d31/d44/d7e/daa 39 2024-04-22T23:46:18.301 INFO:tasks.workunit.client.1.smithi044.stdout:9/624: write d2/dc/d1d/d3f/f6e [265423,86633] 0 2024-04-22T23:46:18.302 INFO:tasks.workunit.client.1.smithi044.stdout:9/625: readlink d2/d6/d4d/l8a 0 2024-04-22T23:46:18.307 INFO:tasks.workunit.client.1.smithi044.stdout:8/793: dwrite db/d14/d84/dbb/fbd [0,4194304] 0 2024-04-22T23:46:18.311 INFO:tasks.workunit.client.1.smithi044.stdout:9/626: mkdir d2/dc/d1d/d34/d38/d4a/dca 0 2024-04-22T23:46:18.313 INFO:tasks.workunit.client.1.smithi044.stdout:5/793: creat d15/d16/d1d/d22/df5/f107 x:0 0 0 2024-04-22T23:46:18.315 INFO:tasks.workunit.client.1.smithi044.stdout:4/801: creat d5/db/d16/d4d/dad/deb/dfc/f10d x:0 0 0 2024-04-22T23:46:18.321 INFO:tasks.workunit.client.1.smithi044.stdout:9/627: mkdir d2/dc/d1d/d34/d38/d5c/d69/dcb 0 2024-04-22T23:46:18.321 INFO:tasks.workunit.client.1.smithi044.stdout:5/794: dwrite d15/d16/d1d/d22/d2a/d3a/f4b [0,4194304] 0 2024-04-22T23:46:18.327 INFO:tasks.workunit.client.1.smithi044.stdout:4/802: rename d5/d31/f32 to d5/d31/d44/dd2/f10e 0 2024-04-22T23:46:18.329 INFO:tasks.workunit.client.1.smithi044.stdout:9/628: dread d2/d6/fa9 [0,4194304] 0 2024-04-22T23:46:18.330 INFO:tasks.workunit.client.1.smithi044.stdout:4/803: truncate d5/d31/d24/d38/ff5 637659 0 2024-04-22T23:46:18.332 INFO:tasks.workunit.client.1.smithi044.stdout:4/804: fsync d5/d31/d24/da6/ff4 0 2024-04-22T23:46:18.333 INFO:tasks.workunit.client.1.smithi044.stdout:4/805: fsync d5/d31/d44/d20/d22/f62 0 2024-04-22T23:46:18.335 INFO:tasks.workunit.client.1.smithi044.stdout:9/629: unlink d2/d6/f74 0 2024-04-22T23:46:18.339 INFO:tasks.workunit.client.1.smithi044.stdout:8/794: getdents db/d68/d8b/dca 0 2024-04-22T23:46:18.342 INFO:tasks.workunit.client.1.smithi044.stdout:9/630: dwrite d2/dc/d1d/d34/f9c [0,4194304] 0 2024-04-22T23:46:18.345 INFO:tasks.workunit.client.1.smithi044.stdout:0/929: rmdir d0/d2/dd/d3e/d40/d43/d79/d12a/d12b 0 2024-04-22T23:46:18.349 INFO:tasks.workunit.client.1.smithi044.stdout:8/795: read db/d14/d24/f26 [5707259,16953] 0 2024-04-22T23:46:18.351 INFO:tasks.workunit.client.1.smithi044.stdout:5/795: rename d15/d16/d1d/d22/d81/cac to d15/d16/d1d/d22/c108 0 2024-04-22T23:46:18.351 INFO:tasks.workunit.client.1.smithi044.stdout:5/796: chown d15/d5d/db5/dc0 969 1 2024-04-22T23:46:18.352 INFO:tasks.workunit.client.1.smithi044.stdout:5/797: truncate f5 9425070 0 2024-04-22T23:46:18.353 INFO:tasks.workunit.client.1.smithi044.stdout:5/798: fdatasync d15/d16/f100 0 2024-04-22T23:46:18.358 INFO:tasks.workunit.client.1.smithi044.stdout:9/631: write d2/f3d [9376459,81227] 0 2024-04-22T23:46:18.360 INFO:tasks.workunit.client.1.smithi044.stdout:9/632: dread - d2/dc/d1d/d3f/fb4 zero size 2024-04-22T23:46:18.360 INFO:tasks.workunit.client.1.smithi044.stdout:9/633: chown d2/dc/f18 757131266 1 2024-04-22T23:46:18.362 INFO:tasks.workunit.client.1.smithi044.stdout:5/799: creat d15/d5d/db5/f109 x:0 0 0 2024-04-22T23:46:18.362 INFO:tasks.workunit.client.1.smithi044.stdout:9/634: fdatasync d2/f3d 0 2024-04-22T23:46:18.367 INFO:tasks.workunit.client.1.smithi044.stdout:5/800: creat d15/d16/d1d/d22/d4f/d68/f10a x:0 0 0 2024-04-22T23:46:18.371 INFO:tasks.workunit.client.1.smithi044.stdout:8/796: getdents db/d68/d8b/dca/dd6 0 2024-04-22T23:46:18.372 INFO:tasks.workunit.client.1.smithi044.stdout:4/806: rename d5/d52/l8c to d5/db/d16/d4d/dad/l10f 0 2024-04-22T23:46:18.373 INFO:tasks.workunit.client.1.smithi044.stdout:0/930: dwrite d0/d2/dd/d3e/f3f [0,4194304] 0 2024-04-22T23:46:18.375 INFO:tasks.workunit.client.1.smithi044.stdout:8/797: read db/f6a [149468,23795] 0 2024-04-22T23:46:18.376 INFO:tasks.workunit.client.1.smithi044.stdout:0/931: chown d0/d2/d4b/f68 83 1 2024-04-22T23:46:18.376 INFO:tasks.workunit.client.1.smithi044.stdout:0/932: stat d0/d2/d4b 0 2024-04-22T23:46:18.377 INFO:tasks.workunit.client.1.smithi044.stdout:0/933: readlink d0/d60/de6/d109/la8 0 2024-04-22T23:46:18.379 INFO:tasks.workunit.client.1.smithi044.stdout:9/635: rename d2/dc/d1d/d34/d38/d5c/d69/d75/d78/f90 to d2/d6/d42/daa/fcc 0 2024-04-22T23:46:18.384 INFO:tasks.workunit.client.1.smithi044.stdout:5/801: truncate d15/d16/d1d/d22/d2a/d3a/d43/f4c 3074443 0 2024-04-22T23:46:18.385 INFO:tasks.workunit.client.1.smithi044.stdout:4/807: symlink d5/l110 0 2024-04-22T23:46:18.387 INFO:tasks.workunit.client.1.smithi044.stdout:4/808: dread d5/d31/d24/d38/f40 [0,4194304] 0 2024-04-22T23:46:18.388 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:18 smithi031 ceph-mon[26721]: pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 39 MiB/s rd, 131 MiB/s wr, 336 op/s 2024-04-22T23:46:18.389 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:18.389 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:18.389 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:18.389 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:46:18.389 INFO:tasks.workunit.client.1.smithi044.stdout:5/802: mkdir d15/d5d/db5/d10b 0 2024-04-22T23:46:18.394 INFO:tasks.workunit.client.1.smithi044.stdout:8/798: write db/d14/d9e/fe0 [641531,28895] 0 2024-04-22T23:46:18.395 INFO:tasks.workunit.client.1.smithi044.stdout:4/809: fsync d5/d31/d44/d20/d22/d8d/fa8 0 2024-04-22T23:46:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:18 smithi044 ceph-mon[34463]: pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 39 MiB/s rd, 131 MiB/s wr, 336 op/s 2024-04-22T23:46:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:18.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:18.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:18.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:46:18.466 INFO:tasks.workunit.client.1.smithi044.stdout:3/773: sync 2024-04-22T23:46:18.466 INFO:tasks.workunit.client.1.smithi044.stdout:1/772: sync 2024-04-22T23:46:18.469 INFO:tasks.workunit.client.1.smithi044.stdout:9/636: link d2/d9/cf d2/d6/d42/ccd 0 2024-04-22T23:46:18.470 INFO:tasks.workunit.client.1.smithi044.stdout:1/773: dread d1/d8/d20/d45/d75/df2/f94 [0,4194304] 0 2024-04-22T23:46:18.471 INFO:tasks.workunit.client.1.smithi044.stdout:8/799: stat db/d14/dc7/dc9/lfc 0 2024-04-22T23:46:18.472 INFO:tasks.workunit.client.1.smithi044.stdout:9/637: stat d2/dc/d1d/d34/d38/d5c/d69/cbe 0 2024-04-22T23:46:18.473 INFO:tasks.workunit.client.1.smithi044.stdout:3/774: dwrite d9/db/d15/d54/f58 [4194304,4194304] 0 2024-04-22T23:46:18.473 INFO:tasks.workunit.client.1.smithi044.stdout:8/800: chown db/d14/d84/l98 12 1 2024-04-22T23:46:18.476 INFO:tasks.workunit.client.1.smithi044.stdout:4/810: mkdir d5/db/d3c/d111 0 2024-04-22T23:46:18.477 INFO:tasks.workunit.client.1.smithi044.stdout:9/638: write d2/d6/d42/daa/fae [495682,96735] 0 2024-04-22T23:46:18.483 INFO:tasks.workunit.client.1.smithi044.stdout:5/803: write d15/d5d/db5/dc0/f66 [1687127,106453] 0 2024-04-22T23:46:18.486 INFO:tasks.workunit.client.1.smithi044.stdout:9/639: creat d2/dc/d51/fce x:0 0 0 2024-04-22T23:46:18.486 INFO:tasks.workunit.client.1.smithi044.stdout:5/804: write d15/d16/d1d/d22/d2a/d3a/d43/d80/fcf [477250,61846] 0 2024-04-22T23:46:18.486 INFO:tasks.workunit.client.1.smithi044.stdout:1/774: dwrite d1/d8/d20/d45/d5d/f86 [0,4194304] 0 2024-04-22T23:46:18.487 INFO:tasks.workunit.client.1.smithi044.stdout:4/811: stat d5/d31/d44/dd2/lf2 0 2024-04-22T23:46:18.491 INFO:tasks.workunit.client.1.smithi044.stdout:3/775: link d9/db/d15/d54/l9d d9/db/d15/d25/d67/lfc 0 2024-04-22T23:46:18.493 INFO:tasks.workunit.client.1.smithi044.stdout:1/775: symlink d1/d8/d34/d112/da8/l119 0 2024-04-22T23:46:18.494 INFO:tasks.workunit.client.1.smithi044.stdout:5/805: mknod d15/d16/d1d/d22/d4f/d68/c10c 0 2024-04-22T23:46:18.495 INFO:tasks.workunit.client.1.smithi044.stdout:3/776: write d9/db/d15/f63 [3775297,86738] 0 2024-04-22T23:46:18.502 INFO:tasks.workunit.client.1.smithi044.stdout:9/640: creat d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/fcf x:0 0 0 2024-04-22T23:46:18.506 INFO:tasks.workunit.client.1.smithi044.stdout:5/806: read d15/d16/d1d/d22/d2a/f79 [1656756,77079] 0 2024-04-22T23:46:18.507 INFO:tasks.workunit.client.1.smithi044.stdout:4/812: link d5/d31/d24/d38/d84/ff8 d5/db/d34/f112 0 2024-04-22T23:46:18.510 INFO:tasks.workunit.client.1.smithi044.stdout:3/777: rename d9/db/d15/d54 to d9/d14/d7d/df8/dfd 0 2024-04-22T23:46:18.514 INFO:tasks.workunit.client.1.smithi044.stdout:1/776: write d1/d8/f18 [3585080,77985] 0 2024-04-22T23:46:18.519 INFO:tasks.workunit.client.1.smithi044.stdout:9/641: rename d2/d9/c59 to d2/dc/d1d/d3f/d9d/cd0 0 2024-04-22T23:46:18.520 INFO:tasks.workunit.client.1.smithi044.stdout:3/778: mknod d9/d14/d7d/df8/dfd/d3c/d95/cfe 0 2024-04-22T23:46:18.530 INFO:tasks.workunit.client.1.smithi044.stdout:1/777: unlink d1/d8/d1c/d7b/fd1 0 2024-04-22T23:46:18.532 INFO:tasks.workunit.client.1.smithi044.stdout:5/807: truncate d15/d16/d1d/d22/d2a/d3a/f4b 7437558 0 2024-04-22T23:46:18.533 INFO:tasks.workunit.client.1.smithi044.stdout:5/808: fdatasync d15/d16/d1d/d22/d4f/dc9/fbd 0 2024-04-22T23:46:18.534 INFO:tasks.workunit.client.1.smithi044.stdout:9/642: unlink d2/dc/cc8 0 2024-04-22T23:46:18.537 INFO:tasks.workunit.client.1.smithi044.stdout:4/813: truncate d5/d31/d24/d38/d84/ff8 736194 0 2024-04-22T23:46:18.538 INFO:tasks.workunit.client.1.smithi044.stdout:3/779: creat d9/d14/d7d/df8/dfd/d3c/d95/dcb/fff x:0 0 0 2024-04-22T23:46:18.542 INFO:tasks.workunit.client.1.smithi044.stdout:5/809: rename d15/d16/d1d/d22/d4f/dc9/d8d/fc8 to d15/d16/d1d/d22/df5/f10d 0 2024-04-22T23:46:18.545 INFO:tasks.workunit.client.1.smithi044.stdout:9/643: unlink d2/dc/f32 0 2024-04-22T23:46:18.547 INFO:tasks.workunit.client.1.smithi044.stdout:4/814: fsync d5/db/d3c/f82 0 2024-04-22T23:46:18.549 INFO:tasks.workunit.client.1.smithi044.stdout:1/778: creat d1/d8/d1c/d10d/f11a x:0 0 0 2024-04-22T23:46:18.559 INFO:tasks.workunit.client.1.smithi044.stdout:4/815: dread - d5/db/d16/d4d/dad/faf zero size 2024-04-22T23:46:18.562 INFO:tasks.workunit.client.1.smithi044.stdout:9/644: dread d2/dc/d94/f1f [0,4194304] 0 2024-04-22T23:46:18.564 INFO:tasks.workunit.client.1.smithi044.stdout:1/779: creat d1/d8/d20/d106/f11b x:0 0 0 2024-04-22T23:46:18.565 INFO:tasks.workunit.client.1.smithi044.stdout:3/780: mknod d9/db/d15/d25/d67/d6b/ded/c100 0 2024-04-22T23:46:18.566 INFO:tasks.workunit.client.1.smithi044.stdout:1/780: read - d1/d8/d34/f9d zero size 2024-04-22T23:46:18.569 INFO:tasks.workunit.client.1.smithi044.stdout:1/781: read d1/d8/d20/d45/d75/df2/fc0 [182044,58988] 0 2024-04-22T23:46:18.569 INFO:tasks.workunit.client.1.smithi044.stdout:9/645: dwrite d2/dc/f27 [4194304,4194304] 0 2024-04-22T23:46:18.569 INFO:tasks.workunit.client.1.smithi044.stdout:3/781: chown d9/db/d33/d5d/l8c 758252164 1 2024-04-22T23:46:18.570 INFO:tasks.workunit.client.1.smithi044.stdout:1/782: fsync d1/d8/d1c/d74/f108 0 2024-04-22T23:46:18.571 INFO:tasks.workunit.client.1.smithi044.stdout:1/783: chown d1/d8/d1c/d10d 46 1 2024-04-22T23:46:18.574 INFO:tasks.workunit.client.1.smithi044.stdout:3/782: dread d9/d14/d7d/df8/dfd/d3c/fbc [0,4194304] 0 2024-04-22T23:46:18.575 INFO:tasks.workunit.client.1.smithi044.stdout:5/810: getdents d15/d16/d1d/daf 0 2024-04-22T23:46:18.585 INFO:tasks.workunit.client.1.smithi044.stdout:3/783: dread d9/db/d15/d25/fe2 [0,4194304] 0 2024-04-22T23:46:18.585 INFO:tasks.workunit.client.1.smithi044.stdout:1/784: dwrite d1/d8/d20/d106/f109 [0,4194304] 0 2024-04-22T23:46:18.587 INFO:tasks.workunit.client.1.smithi044.stdout:5/811: dwrite d15/d5d/db5/f109 [0,4194304] 0 2024-04-22T23:46:18.588 INFO:tasks.workunit.client.1.smithi044.stdout:4/816: getdents d5/d31/d24/d38 0 2024-04-22T23:46:18.598 INFO:tasks.workunit.client.1.smithi044.stdout:3/784: rmdir d9/db/d33/d5d/d70 39 2024-04-22T23:46:18.601 INFO:tasks.workunit.client.1.smithi044.stdout:4/817: chown d5/d31/d44/dd2/f10e 265759 1 2024-04-22T23:46:18.604 INFO:tasks.workunit.client.1.smithi044.stdout:4/818: read d5/d31/d24/d69/d90/fdd [549280,103036] 0 2024-04-22T23:46:18.606 INFO:tasks.workunit.client.1.smithi044.stdout:3/785: mkdir d9/d12/db1/d101 0 2024-04-22T23:46:18.607 INFO:tasks.workunit.client.1.smithi044.stdout:3/786: dread - d9/dc8/ff4 zero size 2024-04-22T23:46:18.612 INFO:tasks.workunit.client.1.smithi044.stdout:4/819: mknod d5/db/d16/d4d/c113 0 2024-04-22T23:46:18.615 INFO:tasks.workunit.client.1.smithi044.stdout:5/812: creat d15/d5d/d64/f10e x:0 0 0 2024-04-22T23:46:18.616 INFO:tasks.workunit.client.1.smithi044.stdout:3/787: creat d9/db/d15/d2e/d48/f102 x:0 0 0 2024-04-22T23:46:18.619 INFO:tasks.workunit.client.1.smithi044.stdout:4/820: creat d5/d31/d24/d69/f114 x:0 0 0 2024-04-22T23:46:18.619 INFO:tasks.workunit.client.1.smithi044.stdout:1/785: dwrite d1/d8/f1e [0,4194304] 0 2024-04-22T23:46:18.627 INFO:tasks.workunit.client.1.smithi044.stdout:5/813: dwrite d15/d5d/db5/dc0/f103 [0,4194304] 0 2024-04-22T23:46:18.628 INFO:tasks.workunit.client.1.smithi044.stdout:4/821: creat d5/d31/d24/da6/db6/f115 x:0 0 0 2024-04-22T23:46:18.629 INFO:tasks.workunit.client.1.smithi044.stdout:4/822: stat d5/d31/d44/d20/d22/l8b 0 2024-04-22T23:46:18.638 INFO:tasks.workunit.client.1.smithi044.stdout:5/814: mkdir d15/d5d/db5/d10f 0 2024-04-22T23:46:18.638 INFO:tasks.workunit.client.1.smithi044.stdout:4/823: dwrite d5/d31/d44/d20/d22/f62 [0,4194304] 0 2024-04-22T23:46:18.641 INFO:tasks.workunit.client.1.smithi044.stdout:4/824: dread - d5/d31/d44/d20/d93/fc4 zero size 2024-04-22T23:46:18.642 INFO:tasks.workunit.client.1.smithi044.stdout:3/788: link d9/d14/d7d/df8/dfd/d3c/l8e d9/db/d33/d5d/d64/dbe/l103 0 2024-04-22T23:46:18.642 INFO:tasks.workunit.client.1.smithi044.stdout:3/789: chown c6 473 1 2024-04-22T23:46:18.644 INFO:tasks.workunit.client.1.smithi044.stdout:5/815: mknod d15/d5d/d64/d6f/c110 0 2024-04-22T23:46:18.647 INFO:tasks.workunit.client.1.smithi044.stdout:4/825: unlink d5/db/d16/d4d/l103 0 2024-04-22T23:46:18.653 INFO:tasks.workunit.client.1.smithi044.stdout:5/816: dwrite d15/d5d/db5/dc0/d38/fd8 [0,4194304] 0 2024-04-22T23:46:18.654 INFO:tasks.workunit.client.1.smithi044.stdout:1/786: truncate d1/d8/d20/d47/f85 1064300 0 2024-04-22T23:46:18.660 INFO:tasks.workunit.client.1.smithi044.stdout:3/790: creat d9/db/d33/d5d/f104 x:0 0 0 2024-04-22T23:46:18.664 INFO:tasks.workunit.client.1.smithi044.stdout:3/791: mknod d9/db/d9f/d38/d74/c105 0 2024-04-22T23:46:18.667 INFO:tasks.workunit.client.1.smithi044.stdout:3/792: read d9/db/d9f/f24 [1052036,20178] 0 2024-04-22T23:46:18.669 INFO:tasks.workunit.client.1.smithi044.stdout:3/793: chown d9/dc8/ff4 1 1 2024-04-22T23:46:18.673 INFO:tasks.workunit.client.1.smithi044.stdout:3/794: creat d9/db/d33/d5d/d64/dbe/f106 x:0 0 0 2024-04-22T23:46:18.676 INFO:tasks.workunit.client.1.smithi044.stdout:1/787: write d1/d8/d20/d45/d75/df2/fc0 [703983,82912] 0 2024-04-22T23:46:18.680 INFO:tasks.workunit.client.1.smithi044.stdout:4/826: dwrite d5/d31/d44/d20/d22/d8d/f97 [0,4194304] 0 2024-04-22T23:46:18.684 INFO:tasks.workunit.client.1.smithi044.stdout:5/817: dread d15/d16/d1d/d22/d2a/d3a/d43/d80/ffa [0,4194304] 0 2024-04-22T23:46:18.685 INFO:tasks.workunit.client.1.smithi044.stdout:3/795: creat d9/d14/d7d/df8/dfd/d3c/d95/dcb/f107 x:0 0 0 2024-04-22T23:46:18.687 INFO:tasks.workunit.client.1.smithi044.stdout:5/818: chown d15/d16/d1d/d22/d4f/d68/c10c 0 1 2024-04-22T23:46:18.687 INFO:tasks.workunit.client.1.smithi044.stdout:3/796: chown d9/db/d9f/d38/d9e 9282 1 2024-04-22T23:46:18.687 INFO:tasks.workunit.client.1.smithi044.stdout:1/788: mkdir d1/d8/d20/dae/d11c 0 2024-04-22T23:46:18.690 INFO:tasks.workunit.client.1.smithi044.stdout:4/827: rename d5/db/d3c/d111 to d5/d31/d24/d69/d90/d116 0 2024-04-22T23:46:18.694 INFO:tasks.workunit.client.1.smithi044.stdout:5/819: mkdir d15/d5d/d111 0 2024-04-22T23:46:18.696 INFO:tasks.workunit.client.1.smithi044.stdout:1/789: rename d1/d8/d1c/d7b/d80/dfd/dad/fb1 to d1/d8/d20/d45/d75/dfb/d95/f11d 0 2024-04-22T23:46:18.702 INFO:tasks.workunit.client.1.smithi044.stdout:5/820: fsync f11 0 2024-04-22T23:46:18.704 INFO:tasks.workunit.client.1.smithi044.stdout:3/797: creat d9/db/d15/d2e/f108 x:0 0 0 2024-04-22T23:46:18.705 INFO:tasks.workunit.client.1.smithi044.stdout:4/828: dread d5/d31/f54 [0,4194304] 0 2024-04-22T23:46:18.707 INFO:tasks.workunit.client.1.smithi044.stdout:1/790: dwrite d1/d8/d20/dae/fe2 [0,4194304] 0 2024-04-22T23:46:18.708 INFO:tasks.workunit.client.1.smithi044.stdout:1/791: stat d1/d8/d20/fdd 0 2024-04-22T23:46:18.710 INFO:tasks.workunit.client.1.smithi044.stdout:1/792: write d1/d8/d34/da1/fbd [208301,92049] 0 2024-04-22T23:46:18.713 INFO:tasks.workunit.client.1.smithi044.stdout:6/644: sync 2024-04-22T23:46:18.717 INFO:tasks.workunit.client.1.smithi044.stdout:3/798: mknod d9/db/d9f/d38/d74/dd5/c109 0 2024-04-22T23:46:18.761 INFO:tasks.workunit.client.1.smithi044.stdout:1/793: rename d1/d8/cf to d1/d8/d20/d45/d75/dfb/c11e 0 2024-04-22T23:46:18.762 INFO:tasks.workunit.client.1.smithi044.stdout:3/799: symlink d9/db/d33/l10a 0 2024-04-22T23:46:18.765 INFO:tasks.workunit.client.1.smithi044.stdout:3/800: read d9/db/fa2 [1098920,34485] 0 2024-04-22T23:46:18.769 INFO:tasks.workunit.client.1.smithi044.stdout:5/821: dwrite d15/d5d/fb1 [0,4194304] 0 2024-04-22T23:46:18.776 INFO:tasks.workunit.client.1.smithi044.stdout:4/829: getdents d5/db/d16/d4d/dad/deb/dfc 0 2024-04-22T23:46:18.780 INFO:tasks.workunit.client.1.smithi044.stdout:6/645: write d5/d3f/d4c/d23/d3d/f4a [214081,125799] 0 2024-04-22T23:46:18.783 INFO:tasks.workunit.client.1.smithi044.stdout:1/794: fdatasync d1/d8/d20/d45/f97 0 2024-04-22T23:46:18.787 INFO:tasks.workunit.client.1.smithi044.stdout:4/830: creat d5/d31/d24/d69/d90/f117 x:0 0 0 2024-04-22T23:46:18.792 INFO:tasks.workunit.client.1.smithi044.stdout:1/795: symlink d1/d8/d34/da1/dab/de9/db9/l11f 0 2024-04-22T23:46:18.793 INFO:tasks.workunit.client.1.smithi044.stdout:3/801: creat d9/db/d9f/f10b x:0 0 0 2024-04-22T23:46:18.797 INFO:tasks.workunit.client.1.smithi044.stdout:4/831: dwrite d5/d31/d44/d20/f4b [4194304,4194304] 0 2024-04-22T23:46:18.801 INFO:tasks.workunit.client.1.smithi044.stdout:5/822: getdents d15/d16/d1d/da2 0 2024-04-22T23:46:18.803 INFO:tasks.workunit.client.1.smithi044.stdout:4/832: write d5/d31/d24/d38/f40 [1030720,24869] 0 2024-04-22T23:46:18.804 INFO:tasks.workunit.client.1.smithi044.stdout:6/646: link d5/d3f/d50/d79/fae d5/d3f/d4c/d40/d53/fc9 0 2024-04-22T23:46:18.806 INFO:tasks.workunit.client.1.smithi044.stdout:3/802: mkdir d9/db/d15/d25/d5b/d52/dbb/d10c 0 2024-04-22T23:46:18.807 INFO:tasks.workunit.client.1.smithi044.stdout:3/803: chown d9/db/d9f/d38/d74/dd5/ldb 172640 1 2024-04-22T23:46:18.809 INFO:tasks.workunit.client.1.smithi044.stdout:5/823: creat d15/d16/d1d/d22/d2a/d3a/dd2/f112 x:0 0 0 2024-04-22T23:46:18.812 INFO:tasks.workunit.client.1.smithi044.stdout:3/804: rmdir d9/db/d15/d25/d67/d6b/ded 39 2024-04-22T23:46:18.813 INFO:tasks.workunit.client.1.smithi044.stdout:5/824: dread d15/d5d/db5/f109 [0,4194304] 0 2024-04-22T23:46:18.818 INFO:tasks.workunit.client.1.smithi044.stdout:4/833: rename d5/d31/d44/c9b to d5/d31/d24/d38/c118 0 2024-04-22T23:46:18.819 INFO:tasks.workunit.client.1.smithi044.stdout:1/796: dread d1/d8/fa9 [0,4194304] 0 2024-04-22T23:46:18.820 INFO:tasks.workunit.client.1.smithi044.stdout:1/797: chown d1/d8/d20/fea 6 1 2024-04-22T23:46:18.820 INFO:tasks.workunit.client.1.smithi044.stdout:6/647: symlink d5/d3f/d4c/db4/dc4/lca 0 2024-04-22T23:46:18.821 INFO:tasks.workunit.client.1.smithi044.stdout:6/648: dread - d5/f90 zero size 2024-04-22T23:46:18.821 INFO:tasks.workunit.client.1.smithi044.stdout:3/805: stat d9/dce 0 2024-04-22T23:46:18.826 INFO:tasks.workunit.client.1.smithi044.stdout:5/825: creat d15/d16/d1d/d22/d4f/dc9/d8d/f113 x:0 0 0 2024-04-22T23:46:18.834 INFO:tasks.workunit.client.1.smithi044.stdout:6/649: rename d5/d30/f44 to d5/d3f/d4c/db4/fcb 0 2024-04-22T23:46:18.835 INFO:tasks.workunit.client.1.smithi044.stdout:5/826: symlink d15/d5d/d64/de0/l114 0 2024-04-22T23:46:18.837 INFO:tasks.workunit.client.1.smithi044.stdout:4/834: symlink d5/d31/l119 0 2024-04-22T23:46:18.843 INFO:tasks.workunit.client.1.smithi044.stdout:5/827: mkdir d15/d16/d1d/d22/d81/d115 0 2024-04-22T23:46:18.847 INFO:tasks.workunit.client.1.smithi044.stdout:3/806: creat d9/db/d15/d25/d5b/f10d x:0 0 0 2024-04-22T23:46:18.849 INFO:tasks.workunit.client.1.smithi044.stdout:3/807: write d9/db/d33/d5d/f104 [991370,95846] 0 2024-04-22T23:46:18.850 INFO:tasks.workunit.client.1.smithi044.stdout:3/808: write d9/dce/fef [600662,127571] 0 2024-04-22T23:46:18.854 INFO:tasks.workunit.client.1.smithi044.stdout:1/798: getdents d1/d8/d34/da1/dab 0 2024-04-22T23:46:18.856 INFO:tasks.workunit.client.1.smithi044.stdout:1/799: chown d1/d8/d1c/d7b/d80/dfd/dad/fff 7526 1 2024-04-22T23:46:18.861 INFO:tasks.workunit.client.1.smithi044.stdout:4/835: rename d5/d42/fe0 to d5/f11a 0 2024-04-22T23:46:18.862 INFO:tasks.workunit.client.1.smithi044.stdout:3/809: symlink d9/dc8/ddd/l10e 0 2024-04-22T23:46:18.862 INFO:tasks.workunit.client.1.smithi044.stdout:4/836: chown d5/db/d16/d4d/dad/dc5/f100 704 1 2024-04-22T23:46:18.863 INFO:tasks.workunit.client.1.smithi044.stdout:1/800: creat d1/d8/d20/dae/f120 x:0 0 0 2024-04-22T23:46:18.864 INFO:tasks.workunit.client.1.smithi044.stdout:6/650: link d5/d3f/d4c/d40/d74/daf/lb3 d5/d3f/d4c/lcc 0 2024-04-22T23:46:18.864 INFO:tasks.workunit.client.1.smithi044.stdout:4/837: chown d5/d31/d44/d20/d93/fc4 961560 1 2024-04-22T23:46:18.872 INFO:tasks.workunit.client.1.smithi044.stdout:6/651: mkdir d5/dd/d28/d75/d88/dcd 0 2024-04-22T23:46:18.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/652: write f1 [8772088,12137] 0 2024-04-22T23:46:18.874 INFO:tasks.workunit.client.1.smithi044.stdout:1/801: symlink d1/d8/d1c/d7b/d80/l121 0 2024-04-22T23:46:18.875 INFO:tasks.workunit.client.1.smithi044.stdout:7/767: sync 2024-04-22T23:46:18.875 INFO:tasks.workunit.client.1.smithi044.stdout:8/801: sync 2024-04-22T23:46:18.875 INFO:tasks.workunit.client.1.smithi044.stdout:2/619: sync 2024-04-22T23:46:18.875 INFO:tasks.workunit.client.1.smithi044.stdout:9/646: sync 2024-04-22T23:46:18.876 INFO:tasks.workunit.client.1.smithi044.stdout:0/934: sync 2024-04-22T23:46:18.877 INFO:tasks.workunit.client.1.smithi044.stdout:2/620: chown d8/d19/d36/d93 4105 1 2024-04-22T23:46:18.877 INFO:tasks.workunit.client.1.smithi044.stdout:6/653: mknod d5/d3f/d4c/d92/cce 0 2024-04-22T23:46:18.878 INFO:tasks.workunit.client.1.smithi044.stdout:7/768: read dd/d1a/db7/dc8/fa1 [887789,44654] 0 2024-04-22T23:46:18.879 INFO:tasks.workunit.client.1.smithi044.stdout:8/802: dread - db/d14/d72/ffd zero size 2024-04-22T23:46:18.879 INFO:tasks.workunit.client.1.smithi044.stdout:4/838: creat d5/f11b x:0 0 0 2024-04-22T23:46:18.881 INFO:tasks.workunit.client.1.smithi044.stdout:3/810: dread d9/db/d15/d25/d4c/f4d [0,4194304] 0 2024-04-22T23:46:18.881 INFO:tasks.workunit.client.1.smithi044.stdout:1/802: creat d1/d8/d1c/d7b/d80/d46/f122 x:0 0 0 2024-04-22T23:46:18.884 INFO:tasks.workunit.client.1.smithi044.stdout:1/803: truncate d1/d8/d20/dae/f120 700512 0 2024-04-22T23:46:18.893 INFO:tasks.workunit.client.1.smithi044.stdout:3/811: truncate d9/d12/f78 1592878 0 2024-04-22T23:46:18.897 INFO:tasks.workunit.client.1.smithi044.stdout:6/654: creat d5/d3f/d4c/d23/d3d/fcf x:0 0 0 2024-04-22T23:46:18.901 INFO:tasks.workunit.client.1.smithi044.stdout:6/655: readlink d5/l1c 0 2024-04-22T23:46:18.901 INFO:tasks.workunit.client.1.smithi044.stdout:5/828: rename d15/d16/d1d/d22/d81 to d15/d16/dbc/d116 0 2024-04-22T23:46:18.902 INFO:tasks.workunit.client.1.smithi044.stdout:2/621: dwrite d8/d19/f47 [0,4194304] 0 2024-04-22T23:46:18.903 INFO:tasks.workunit.client.1.smithi044.stdout:0/935: getdents d0/d2/dd/d3e/d40/d43/de4 0 2024-04-22T23:46:18.906 INFO:tasks.workunit.client.1.smithi044.stdout:9/647: mkdir d2/dc/d1d/d34/d8e/dbd/dd1 0 2024-04-22T23:46:18.908 INFO:tasks.workunit.client.1.smithi044.stdout:2/622: fdatasync d8/d19/d36/d93/dbf/fd5 0 2024-04-22T23:46:18.910 INFO:tasks.workunit.client.1.smithi044.stdout:7/769: mkdir dd/df7/d112/d11c 0 2024-04-22T23:46:18.911 INFO:tasks.workunit.client.1.smithi044.stdout:4/839: dwrite d5/db/d16/d4d/dad/dc5/f100 [0,4194304] 0 2024-04-22T23:46:18.913 INFO:tasks.workunit.client.1.smithi044.stdout:6/656: unlink d5/dd/f3e 0 2024-04-22T23:46:18.914 INFO:tasks.workunit.client.1.smithi044.stdout:3/812: dread d9/f9b [0,4194304] 0 2024-04-22T23:46:18.918 INFO:tasks.workunit.client.1.smithi044.stdout:4/840: chown d5/d31/d44/d20/l7c 302 1 2024-04-22T23:46:18.920 INFO:tasks.workunit.client.1.smithi044.stdout:0/936: stat d0/d2/d4b/dad/f10a 0 2024-04-22T23:46:18.921 INFO:tasks.workunit.client.1.smithi044.stdout:8/803: write db/d14/d24/f59 [1301160,2024] 0 2024-04-22T23:46:18.924 INFO:tasks.workunit.client.1.smithi044.stdout:0/937: fsync d0/d2/dd/d3e/d40/d43/de4/d146/f32 0 2024-04-22T23:46:18.925 INFO:tasks.workunit.client.1.smithi044.stdout:2/623: creat d8/d19/d36/d46/ddb/ddd/fde x:0 0 0 2024-04-22T23:46:18.931 INFO:tasks.workunit.client.1.smithi044.stdout:4/841: dwrite d5/db/d16/d4d/dad/dc5/dd9/fed [0,4194304] 0 2024-04-22T23:46:18.931 INFO:tasks.workunit.client.1.smithi044.stdout:6/657: fdatasync d5/d3f/d50/d79/f81 0 2024-04-22T23:46:18.933 INFO:tasks.workunit.client.1.smithi044.stdout:6/658: dread - d5/dd/f7d zero size 2024-04-22T23:46:18.937 INFO:tasks.workunit.client.1.smithi044.stdout:5/829: mkdir d15/d16/dbc/d116/d115/d117 0 2024-04-22T23:46:18.944 INFO:tasks.workunit.client.1.smithi044.stdout:6/659: dwrite d5/dd/fa8 [0,4194304] 0 2024-04-22T23:46:18.953 INFO:tasks.workunit.client.1.smithi044.stdout:7/770: mkdir dd/df7/d112/d11c/d11d 0 2024-04-22T23:46:18.957 INFO:tasks.workunit.client.1.smithi044.stdout:4/842: creat d5/d31/d24/d69/f11c x:0 0 0 2024-04-22T23:46:18.958 INFO:tasks.workunit.client.1.smithi044.stdout:4/843: stat d5/d31/d24/l56 0 2024-04-22T23:46:18.959 INFO:tasks.workunit.client.1.smithi044.stdout:1/804: truncate d1/d8/d20/dae/f120 251011 0 2024-04-22T23:46:18.960 INFO:tasks.workunit.client.1.smithi044.stdout:3/813: dread d9/db/d15/d25/d5b/f7e [0,4194304] 0 2024-04-22T23:46:18.961 INFO:tasks.workunit.client.1.smithi044.stdout:5/830: mknod d15/d5d/db5/dc0/d38/d9f/c118 0 2024-04-22T23:46:18.962 INFO:tasks.workunit.client.1.smithi044.stdout:1/805: chown d1/d8/d1c/d7b/f65 174509 1 2024-04-22T23:46:18.963 INFO:tasks.workunit.client.1.smithi044.stdout:1/806: stat d1/d8/d34/ld6 0 2024-04-22T23:46:18.966 INFO:tasks.workunit.client.1.smithi044.stdout:2/624: unlink d8/d19/d1c/c75 0 2024-04-22T23:46:18.968 INFO:tasks.workunit.client.1.smithi044.stdout:7/771: unlink dd/ffd 0 2024-04-22T23:46:18.970 INFO:tasks.workunit.client.1.smithi044.stdout:0/938: creat d0/d2/dd/d3e/d40/d43/de4/d126/d127/f14c x:0 0 0 2024-04-22T23:46:18.971 INFO:tasks.workunit.client.1.smithi044.stdout:4/844: creat d5/d31/d44/d20/d22/d8d/f11d x:0 0 0 2024-04-22T23:46:18.971 INFO:tasks.workunit.client.1.smithi044.stdout:0/939: write d0/d4c/d59/fb3 [293502,63082] 0 2024-04-22T23:46:18.973 INFO:tasks.workunit.client.1.smithi044.stdout:9/648: getdents d2/dc/d1d 0 2024-04-22T23:46:18.976 INFO:tasks.workunit.client.1.smithi044.stdout:3/814: symlink d9/db/d15/d25/d4c/d86/l10f 0 2024-04-22T23:46:18.978 INFO:tasks.workunit.client.1.smithi044.stdout:8/804: creat db/d14/f11d x:0 0 0 2024-04-22T23:46:18.979 INFO:tasks.workunit.client.1.smithi044.stdout:1/807: unlink d1/d8/fa9 0 2024-04-22T23:46:18.979 INFO:tasks.workunit.client.1.smithi044.stdout:0/940: dwrite d0/d2/dd/d3e/d40/d43/de4/d146/f70 [0,4194304] 0 2024-04-22T23:46:18.980 INFO:tasks.workunit.client.1.smithi044.stdout:2/625: truncate d8/f92 365304 0 2024-04-22T23:46:18.981 INFO:tasks.workunit.client.1.smithi044.stdout:2/626: read f7 [3681650,7533] 0 2024-04-22T23:46:18.983 INFO:tasks.workunit.client.1.smithi044.stdout:5/831: symlink d15/d16/l119 0 2024-04-22T23:46:18.986 INFO:tasks.workunit.client.1.smithi044.stdout:4/845: creat d5/d31/d24/d69/f11e x:0 0 0 2024-04-22T23:46:18.993 INFO:tasks.workunit.client.1.smithi044.stdout:6/660: rename d5/d3f/d4c/d23/d3d/f51 to d5/dd/fd0 0 2024-04-22T23:46:18.995 INFO:tasks.workunit.client.1.smithi044.stdout:9/649: symlink d2/dc/d1d/d34/d38/d5c/d69/d7a/ld2 0 2024-04-22T23:46:19.002 INFO:tasks.workunit.client.1.smithi044.stdout:3/815: symlink d9/db/d15/d25/l110 0 2024-04-22T23:46:19.004 INFO:tasks.workunit.client.1.smithi044.stdout:1/808: readlink d1/d8/d20/d45/l64 0 2024-04-22T23:46:19.010 INFO:tasks.workunit.client.1.smithi044.stdout:2/627: mknod d8/d27/d5b/d74/d81/cdf 0 2024-04-22T23:46:19.012 INFO:tasks.workunit.client.1.smithi044.stdout:5/832: truncate d15/d16/d1d/d22/d2a/d3a/d43/d80/f8c 4567510 0 2024-04-22T23:46:19.013 INFO:tasks.workunit.client.1.smithi044.stdout:5/833: chown d15/d16/l119 407722 1 2024-04-22T23:46:19.014 INFO:tasks.workunit.client.1.smithi044.stdout:5/834: truncate d15/d3c/fec 717626 0 2024-04-22T23:46:19.018 INFO:tasks.workunit.client.1.smithi044.stdout:8/805: dread db/f27 [0,4194304] 0 2024-04-22T23:46:19.023 INFO:tasks.workunit.client.1.smithi044.stdout:6/661: unlink d5/d3f/d4c/d40/d53/f6c 0 2024-04-22T23:46:19.025 INFO:tasks.workunit.client.1.smithi044.stdout:8/806: dwrite db/d14/d24/f4e [0,4194304] 0 2024-04-22T23:46:19.027 INFO:tasks.workunit.client.1.smithi044.stdout:8/807: read db/d68/d74/fb0 [176404,65663] 0 2024-04-22T23:46:19.043 INFO:tasks.workunit.client.1.smithi044.stdout:6/662: dread d5/d3f/f54 [0,4194304] 0 2024-04-22T23:46:19.091 INFO:tasks.workunit.client.1.smithi044.stdout:1/809: readlink d1/d8/d1c/d7b/d80/dfd/dbf/dc2/lfc 0 2024-04-22T23:46:19.091 INFO:tasks.workunit.client.1.smithi044.stdout:7/772: link dd/ff4 dd/d114/f11e 0 2024-04-22T23:46:19.092 INFO:tasks.workunit.client.1.smithi044.stdout:1/810: chown d1/d8/d20/d45/f3d 358336356 1 2024-04-22T23:46:19.093 INFO:tasks.workunit.client.1.smithi044.stdout:7/773: fdatasync dd/d27/fb5 0 2024-04-22T23:46:19.093 INFO:tasks.workunit.client.1.smithi044.stdout:5/835: creat d15/d5d/db5/f11a x:0 0 0 2024-04-22T23:46:19.102 INFO:tasks.workunit.client.1.smithi044.stdout:9/650: mkdir d2/dc/d94/dd3 0 2024-04-22T23:46:19.103 INFO:tasks.workunit.client.1.smithi044.stdout:6/663: unlink d5/dd/f10 0 2024-04-22T23:46:19.108 INFO:tasks.workunit.client.1.smithi044.stdout:1/811: rmdir d1/d8/d34/da1/dab/de9/db9 39 2024-04-22T23:46:19.117 INFO:tasks.workunit.client.1.smithi044.stdout:4/846: creat d5/d31/d44/d20/f11f x:0 0 0 2024-04-22T23:46:19.119 INFO:tasks.workunit.client.1.smithi044.stdout:0/941: rename d0/d2/dd/d3e/d40/d43/d79/dbb/cfe to d0/d60/c14d 0 2024-04-22T23:46:19.123 INFO:tasks.workunit.client.1.smithi044.stdout:9/651: write d2/dc/d1d/d34/d38/d4a/d72/fb1 [1253415,74854] 0 2024-04-22T23:46:19.126 INFO:tasks.workunit.client.1.smithi044.stdout:3/816: creat d9/f111 x:0 0 0 2024-04-22T23:46:19.127 INFO:tasks.workunit.client.1.smithi044.stdout:1/812: mkdir d1/d8/d34/da1/d123 0 2024-04-22T23:46:19.128 INFO:tasks.workunit.client.1.smithi044.stdout:3/817: read d9/db/d15/d25/fe2 [481876,25990] 0 2024-04-22T23:46:19.129 INFO:tasks.workunit.client.1.smithi044.stdout:1/813: readlink d1/d8/d34/ld6 0 2024-04-22T23:46:19.130 INFO:tasks.workunit.client.1.smithi044.stdout:0/942: rmdir d0/d2/dd/d3e/d40/d43/de4/d146/dbf 39 2024-04-22T23:46:19.133 INFO:tasks.workunit.client.1.smithi044.stdout:9/652: dwrite d2/dc/d51/fce [0,4194304] 0 2024-04-22T23:46:19.138 INFO:tasks.workunit.client.1.smithi044.stdout:1/814: dread d1/d8/d20/d106/f109 [0,4194304] 0 2024-04-22T23:46:19.138 INFO:tasks.workunit.client.1.smithi044.stdout:6/664: rename d5/d3f/d50/d79/d96/fc6 to d5/d3f/d4c/d83/fd1 0 2024-04-22T23:46:19.143 INFO:tasks.workunit.client.1.smithi044.stdout:9/653: dwrite d2/dc/f18 [8388608,4194304] 0 2024-04-22T23:46:19.145 INFO:tasks.workunit.client.1.smithi044.stdout:8/808: getdents db/ddd 0 2024-04-22T23:46:19.147 INFO:tasks.workunit.client.1.smithi044.stdout:0/943: creat d0/d2/dd/f14e x:0 0 0 2024-04-22T23:46:19.149 INFO:tasks.workunit.client.1.smithi044.stdout:0/944: chown d0/d2/dd/d3e/d40/d43/de4/d146/d2f/f13a 2204 1 2024-04-22T23:46:19.152 INFO:tasks.workunit.client.1.smithi044.stdout:4/847: creat d5/d31/d44/d20/f120 x:0 0 0 2024-04-22T23:46:19.281 INFO:tasks.workunit.client.1.smithi044.stdout:1/815: truncate d1/d8/f1a 5152200 0 2024-04-22T23:46:19.283 INFO:tasks.workunit.client.1.smithi044.stdout:8/809: rmdir db/d14/d84/dbb 39 2024-04-22T23:46:19.283 INFO:tasks.workunit.client.1.smithi044.stdout:7/774: dwrite dd/d98/de8/f107 [0,4194304] 0 2024-04-22T23:46:19.283 INFO:tasks.workunit.client.1.smithi044.stdout:0/945: mkdir d0/d2/d4b/dad/d125/d14f 0 2024-04-22T23:46:19.291 INFO:tasks.workunit.client.1.smithi044.stdout:3/818: creat d9/db/d15/d25/d67/d6b/f112 x:0 0 0 2024-04-22T23:46:19.291 INFO:tasks.workunit.client.1.smithi044.stdout:8/810: fdatasync db/d68/d74/fac 0 2024-04-22T23:46:19.291 INFO:tasks.workunit.client.1.smithi044.stdout:1/816: truncate d1/d8/d20/d45/d75/f93 1265116 0 2024-04-22T23:46:19.293 INFO:tasks.workunit.client.1.smithi044.stdout:0/946: creat d0/d2/dd/d3e/d40/d43/d7b/f150 x:0 0 0 2024-04-22T23:46:19.299 INFO:tasks.workunit.client.1.smithi044.stdout:6/665: dwrite d5/d3f/f54 [0,4194304] 0 2024-04-22T23:46:19.301 INFO:tasks.workunit.client.1.smithi044.stdout:4/848: dwrite d5/d31/d24/f72 [0,4194304] 0 2024-04-22T23:46:19.301 INFO:tasks.workunit.client.1.smithi044.stdout:8/811: readlink db/d14/d24/lc8 0 2024-04-22T23:46:19.302 INFO:tasks.workunit.client.1.smithi044.stdout:7/775: dread dd/d55/d5e/d60/d68/f73 [0,4194304] 0 2024-04-22T23:46:19.306 INFO:tasks.workunit.client.1.smithi044.stdout:7/776: chown dd/d1f/d24/f1c 1404742 1 2024-04-22T23:46:19.308 INFO:tasks.workunit.client.1.smithi044.stdout:7/777: write dd/d55/f58 [3571415,51993] 0 2024-04-22T23:46:19.309 INFO:tasks.workunit.client.1.smithi044.stdout:4/849: rmdir d5/d42 39 2024-04-22T23:46:19.316 INFO:tasks.workunit.client.1.smithi044.stdout:0/947: write d0/d2/fcb [946027,114088] 0 2024-04-22T23:46:19.318 INFO:tasks.workunit.client.1.smithi044.stdout:0/948: readlink d0/l81 0 2024-04-22T23:46:19.318 INFO:tasks.workunit.client.1.smithi044.stdout:6/666: unlink d5/d3f/d50/d79/lc3 0 2024-04-22T23:46:19.319 INFO:tasks.workunit.client.1.smithi044.stdout:4/850: dwrite d5/d31/d44/d20/d22/d8d/f10a [0,4194304] 0 2024-04-22T23:46:19.322 INFO:tasks.workunit.client.1.smithi044.stdout:1/817: dread d1/d8/f1a [0,4194304] 0 2024-04-22T23:46:19.323 INFO:tasks.workunit.client.1.smithi044.stdout:3/819: rename d9/db/d15/d25/d5b/d52 to d9/db/d15/d25/d67/d6b/ded/d113 0 2024-04-22T23:46:19.325 INFO:tasks.workunit.client.1.smithi044.stdout:3/820: chown d9/db/d15/d2e/c5a 7501 1 2024-04-22T23:46:19.326 INFO:tasks.workunit.client.1.smithi044.stdout:0/949: fsync d0/d2/d4b/fa3 0 2024-04-22T23:46:19.332 INFO:tasks.workunit.client.1.smithi044.stdout:8/812: write db/d6c/fd9 [916193,29474] 0 2024-04-22T23:46:19.335 INFO:tasks.workunit.client.1.smithi044.stdout:7/778: dread dd/d1f/f2c [0,4194304] 0 2024-04-22T23:46:19.336 INFO:tasks.workunit.client.1.smithi044.stdout:7/779: stat dd/d55/d5e/d60/d7b/f7f 0 2024-04-22T23:46:19.338 INFO:tasks.workunit.client.1.smithi044.stdout:6/667: write d5/d3f/d4c/d23/f26 [64848,40149] 0 2024-04-22T23:46:19.339 INFO:tasks.workunit.client.1.smithi044.stdout:6/668: chown d5/d3f/d4c/d40/db9 0 1 2024-04-22T23:46:19.339 INFO:tasks.workunit.client.1.smithi044.stdout:2/628: sync 2024-04-22T23:46:19.339 INFO:tasks.workunit.client.1.smithi044.stdout:5/836: sync 2024-04-22T23:46:19.340 INFO:tasks.workunit.client.1.smithi044.stdout:3/821: rename d9/db/d15/d25/d67/d6b/ded/d113/dbb/cc0 to d9/d12/db1/df9/c114 0 2024-04-22T23:46:19.342 INFO:tasks.workunit.client.1.smithi044.stdout:4/851: creat d5/d31/d44/d7e/daa/f121 x:0 0 0 2024-04-22T23:46:19.343 INFO:tasks.workunit.client.1.smithi044.stdout:8/813: symlink db/d68/d8b/dca/dd6/l11e 0 2024-04-22T23:46:19.343 INFO:tasks.workunit.client.1.smithi044.stdout:4/852: truncate d5/d31/f102 18831 0 2024-04-22T23:46:19.344 INFO:tasks.workunit.client.1.smithi044.stdout:8/814: chown db/d14/d24/f26 6507 1 2024-04-22T23:46:19.349 INFO:tasks.workunit.client.1.smithi044.stdout:3/822: mkdir d9/db/d33/d5d/d64/d115 0 2024-04-22T23:46:19.351 INFO:tasks.workunit.client.1.smithi044.stdout:3/823: chown d9/db/d15/d2e/d48/f102 0 1 2024-04-22T23:46:19.352 INFO:tasks.workunit.client.1.smithi044.stdout:1/818: dread d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:19.353 INFO:tasks.workunit.client.1.smithi044.stdout:4/853: rmdir d5/d31/d44/d20 39 2024-04-22T23:46:19.356 INFO:tasks.workunit.client.1.smithi044.stdout:1/819: dread d1/d8/d1c/d7b/d80/dfd/dbf/ddb/fef [0,4194304] 0 2024-04-22T23:46:19.357 INFO:tasks.workunit.client.1.smithi044.stdout:7/780: truncate f8 255905 0 2024-04-22T23:46:19.359 INFO:tasks.workunit.client.1.smithi044.stdout:5/837: write d15/d16/d1d/d22/d2a/d3a/d43/d80/fda [587638,52770] 0 2024-04-22T23:46:19.362 INFO:tasks.workunit.client.1.smithi044.stdout:2/629: mkdir d8/d19/d36/d46/d63/de0 0 2024-04-22T23:46:19.363 INFO:tasks.workunit.client.1.smithi044.stdout:8/815: dread db/ddd/df2/fe1 [4194304,4194304] 0 2024-04-22T23:46:19.366 INFO:tasks.workunit.client.1.smithi044.stdout:0/950: getdents d0/d4c/dbd 0 2024-04-22T23:46:19.370 INFO:tasks.workunit.client.1.smithi044.stdout:3/824: dread d9/db/d33/f96 [0,4194304] 0 2024-04-22T23:46:19.372 INFO:tasks.workunit.client.1.smithi044.stdout:8/816: dwrite db/d14/d24/f59 [0,4194304] 0 2024-04-22T23:46:19.372 INFO:tasks.workunit.client.1.smithi044.stdout:3/825: fdatasync d9/db/d9f/f68 0 2024-04-22T23:46:19.372 INFO:tasks.workunit.client.1.smithi044.stdout:7/781: mkdir dd/df7/db8/d108/d11f 0 2024-04-22T23:46:19.375 INFO:tasks.workunit.client.1.smithi044.stdout:6/669: creat d5/dd/fd2 x:0 0 0 2024-04-22T23:46:19.378 INFO:tasks.workunit.client.1.smithi044.stdout:2/630: mkdir d8/d60/de1 0 2024-04-22T23:46:19.379 INFO:tasks.workunit.client.1.smithi044.stdout:4/854: dread d5/d52/f95 [0,4194304] 0 2024-04-22T23:46:19.380 INFO:tasks.workunit.client.1.smithi044.stdout:2/631: write d8/d19/d36/d46/d63/d65/d94/fb7 [669812,52051] 0 2024-04-22T23:46:19.381 INFO:tasks.workunit.client.1.smithi044.stdout:0/951: mknod d0/d2/dd/d3e/d40/d43/de4/d146/d2f/d77/c151 0 2024-04-22T23:46:19.382 INFO:tasks.workunit.client.1.smithi044.stdout:0/952: readlink d0/d4c/d59/d5a/l7f 0 2024-04-22T23:46:19.385 INFO:tasks.workunit.client.1.smithi044.stdout:9/654: sync 2024-04-22T23:46:19.388 INFO:tasks.workunit.client.1.smithi044.stdout:6/670: creat d5/dd/d67/fd3 x:0 0 0 2024-04-22T23:46:19.390 INFO:tasks.workunit.client.1.smithi044.stdout:8/817: dread db/fd [0,4194304] 0 2024-04-22T23:46:19.390 INFO:tasks.workunit.client.1.smithi044.stdout:6/671: write d5/d30/d73/fc1 [1323416,79926] 0 2024-04-22T23:46:19.391 INFO:tasks.workunit.client.1.smithi044.stdout:6/672: chown d5/dd/f16 6963 1 2024-04-22T23:46:19.391 INFO:tasks.workunit.client.1.smithi044.stdout:0/953: creat d0/d4c/dbd/f152 x:0 0 0 2024-04-22T23:46:19.392 INFO:tasks.workunit.client.1.smithi044.stdout:3/826: creat d9/d12/db1/d101/f116 x:0 0 0 2024-04-22T23:46:19.393 INFO:tasks.workunit.client.1.smithi044.stdout:7/782: symlink dd/d55/d5e/d60/d106/l120 0 2024-04-22T23:46:19.394 INFO:tasks.workunit.client.1.smithi044.stdout:5/838: link d15/d16/d1d/d22/d4f/dc9/d8d/f98 d15/d16/d1d/d22/df5/f11b 0 2024-04-22T23:46:19.399 INFO:tasks.workunit.client.1.smithi044.stdout:6/673: dwrite d5/dd/f68 [0,4194304] 0 2024-04-22T23:46:19.401 INFO:tasks.workunit.client.1.smithi044.stdout:8/818: chown db/d14/d24/c28 0 1 2024-04-22T23:46:19.403 INFO:tasks.workunit.client.1.smithi044.stdout:3/827: creat d9/d12/db1/d101/f117 x:0 0 0 2024-04-22T23:46:19.405 INFO:tasks.workunit.client.1.smithi044.stdout:0/954: creat d0/d2/dd/d3e/d40/d43/d7b/f153 x:0 0 0 2024-04-22T23:46:19.406 INFO:tasks.workunit.client.1.smithi044.stdout:7/783: dwrite dd/d1a/db7/dc8/d97/fe6 [0,4194304] 0 2024-04-22T23:46:19.408 INFO:tasks.workunit.client.1.smithi044.stdout:0/955: stat d0/d2/dd/c2c 0 2024-04-22T23:46:19.409 INFO:tasks.workunit.client.1.smithi044.stdout:5/839: rmdir d15/d16/d1d/d22/d2a/d3a/d43 39 2024-04-22T23:46:19.417 INFO:tasks.workunit.client.1.smithi044.stdout:6/674: creat d5/d3f/d50/d79/d96/fd4 x:0 0 0 2024-04-22T23:46:19.417 INFO:tasks.workunit.client.1.smithi044.stdout:5/840: dwrite d15/d5d/db5/dc0/f103 [4194304,4194304] 0 2024-04-22T23:46:19.418 INFO:tasks.workunit.client.1.smithi044.stdout:6/675: dread - d5/d3f/d50/f86 zero size 2024-04-22T23:46:19.419 INFO:tasks.workunit.client.1.smithi044.stdout:6/676: fsync d5/d3f/d4c/d92/fa5 0 2024-04-22T23:46:19.427 INFO:tasks.workunit.client.1.smithi044.stdout:0/956: unlink d0/d60/fe2 0 2024-04-22T23:46:19.430 INFO:tasks.workunit.client.1.smithi044.stdout:8/819: creat db/d68/d74/d112/f11f x:0 0 0 2024-04-22T23:46:19.431 INFO:tasks.workunit.client.1.smithi044.stdout:4/855: write d5/d31/d44/d20/f23 [1606414,118924] 0 2024-04-22T23:46:19.433 INFO:tasks.workunit.client.1.smithi044.stdout:6/677: readlink d5/d3f/d4c/lcc 0 2024-04-22T23:46:19.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/655: getdents d2/dc/d1d/d34/d38/d5c/d69/d75/d78 0 2024-04-22T23:46:19.436 INFO:tasks.workunit.client.1.smithi044.stdout:7/784: getdents dd/df7/d112/d11c/d11d 0 2024-04-22T23:46:19.438 INFO:tasks.workunit.client.1.smithi044.stdout:3/828: write d9/db/d15/f18 [9986227,118124] 0 2024-04-22T23:46:19.439 INFO:tasks.workunit.client.1.smithi044.stdout:0/957: creat d0/d2/d4b/dad/dc8/d10e/f154 x:0 0 0 2024-04-22T23:46:19.441 INFO:tasks.workunit.client.1.smithi044.stdout:5/841: mknod d15/d3c/df9/c11c 0 2024-04-22T23:46:19.442 INFO:tasks.workunit.client.1.smithi044.stdout:3/829: truncate d9/db/d9f/d38/d9e/ff2 89563 0 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:19.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:19.445 INFO:tasks.workunit.client.1.smithi044.stdout:8/820: rename db/f69 to db/d6c/f120 0 2024-04-22T23:46:19.446 INFO:tasks.workunit.client.1.smithi044.stdout:9/656: rename d2 to d2/dc/d1d/dd4 22 2024-04-22T23:46:19.449 INFO:tasks.workunit.client.1.smithi044.stdout:7/785: mknod dd/d1a/db7/c121 0 2024-04-22T23:46:19.451 INFO:tasks.workunit.client.1.smithi044.stdout:0/958: dwrite d0/d60/fd7 [0,4194304] 0 2024-04-22T23:46:19.451 INFO:tasks.workunit.client.1.smithi044.stdout:9/657: write d2/f7d [350738,31627] 0 2024-04-22T23:46:19.457 INFO:tasks.workunit.client.1.smithi044.stdout:5/842: dwrite d15/d16/d1d/d22/f97 [0,4194304] 0 2024-04-22T23:46:19.458 INFO:tasks.workunit.client.1.smithi044.stdout:4/856: dwrite d5/f11a [0,4194304] 0 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:19.461 INFO:tasks.workunit.client.1.smithi044.stdout:5/843: write d15/d5d/db5/dc0/f2e [4920858,6019] 0 2024-04-22T23:46:19.468 INFO:tasks.workunit.client.1.smithi044.stdout:4/857: dread d5/d31/d44/d20/f4b [4194304,4194304] 0 2024-04-22T23:46:19.473 INFO:tasks.workunit.client.1.smithi044.stdout:6/678: dwrite d5/d3f/f15 [0,4194304] 0 2024-04-22T23:46:19.482 INFO:tasks.workunit.client.1.smithi044.stdout:7/786: mkdir dd/d1a/d122 0 2024-04-22T23:46:19.485 INFO:tasks.workunit.client.1.smithi044.stdout:9/658: symlink d2/dc/d1d/d3f/d9d/ld5 0 2024-04-22T23:46:19.485 INFO:tasks.workunit.client.1.smithi044.stdout:3/830: write d9/d12/fdc [1221793,60306] 0 2024-04-22T23:46:19.488 INFO:tasks.workunit.client.1.smithi044.stdout:0/959: creat d0/d2/dd/d3e/d40/d43/d79/f155 x:0 0 0 2024-04-22T23:46:19.499 INFO:tasks.workunit.client.1.smithi044.stdout:4/858: dread d5/db/d34/fb9 [0,4194304] 0 2024-04-22T23:46:19.520 INFO:tasks.workunit.client.1.smithi044.stdout:5/844: chown d15/d16/d1d/d22/d2a/d3a/d43/da1 57719 1 2024-04-22T23:46:19.526 INFO:tasks.workunit.client.1.smithi044.stdout:7/787: read - dd/d1a/d61/fa6 zero size 2024-04-22T23:46:19.529 INFO:tasks.workunit.client.1.smithi044.stdout:4/859: mkdir d5/db/d16/d4d/dad/deb/d122 0 2024-04-22T23:46:19.531 INFO:tasks.workunit.client.1.smithi044.stdout:8/821: creat db/d14/d24/f121 x:0 0 0 2024-04-22T23:46:19.532 INFO:tasks.workunit.client.1.smithi044.stdout:5/845: mkdir d15/d16/d1d/d22/d2a/d3a/d43/d11d 0 2024-04-22T23:46:19.533 INFO:tasks.workunit.client.1.smithi044.stdout:5/846: fsync d15/d5d/d64/f10e 0 2024-04-22T23:46:19.534 INFO:tasks.workunit.client.1.smithi044.stdout:6/679: symlink d5/d3f/da7/ld5 0 2024-04-22T23:46:19.540 INFO:tasks.workunit.client.1.smithi044.stdout:3/831: creat d9/db/d15/d25/d67/d6b/ded/d113/dbb/d10c/f118 x:0 0 0 2024-04-22T23:46:19.540 INFO:tasks.workunit.client.1.smithi044.stdout:0/960: dread d0/d2/dd/d3e/d40/d43/f5f [0,4194304] 0 2024-04-22T23:46:19.541 INFO:tasks.workunit.client.1.smithi044.stdout:8/822: creat db/d14/dc7/f122 x:0 0 0 2024-04-22T23:46:19.543 INFO:tasks.workunit.client.1.smithi044.stdout:5/847: creat d15/d3c/df9/f11e x:0 0 0 2024-04-22T23:46:19.546 INFO:tasks.workunit.client.1.smithi044.stdout:9/659: truncate d2/dc/f18 9951546 0 2024-04-22T23:46:19.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/832: symlink d9/d14/dd2/l119 0 2024-04-22T23:46:19.548 INFO:tasks.workunit.client.1.smithi044.stdout:8/823: fsync db/d14/d84/dc0/ff5 0 2024-04-22T23:46:19.550 INFO:tasks.workunit.client.1.smithi044.stdout:3/833: write d9/d14/d7d/df8/dfd/d3c/d95/dcb/fff [354845,32408] 0 2024-04-22T23:46:19.554 INFO:tasks.workunit.client.1.smithi044.stdout:8/824: readlink db/d52/d9c/la4 0 2024-04-22T23:46:19.556 INFO:tasks.workunit.client.1.smithi044.stdout:7/788: getdents dd/d55/dcf 0 2024-04-22T23:46:19.558 INFO:tasks.workunit.client.1.smithi044.stdout:3/834: rmdir d9/db/d9f/d38/dd9 39 2024-04-22T23:46:19.562 INFO:tasks.workunit.client.1.smithi044.stdout:6/680: getdents d5/d3f/d4c/d40/d74 0 2024-04-22T23:46:19.565 INFO:tasks.workunit.client.1.smithi044.stdout:8/825: creat db/d68/d8b/dca/f123 x:0 0 0 2024-04-22T23:46:19.568 INFO:tasks.workunit.client.1.smithi044.stdout:9/660: link d2/dc/d1d/d3f/d9d/ld5 d2/dc/d1d/d34/d38/d4a/da6/ld6 0 2024-04-22T23:46:19.569 INFO:tasks.workunit.client.1.smithi044.stdout:6/681: dwrite d5/d3f/d50/fb5 [0,4194304] 0 2024-04-22T23:46:19.570 INFO:tasks.workunit.client.1.smithi044.stdout:5/848: getdents d15/d5d/d64/de0 0 2024-04-22T23:46:19.578 INFO:tasks.workunit.client.1.smithi044.stdout:8/826: mknod db/d52/d9c/dad/db8/c124 0 2024-04-22T23:46:19.581 INFO:tasks.workunit.client.1.smithi044.stdout:5/849: dwrite d15/d5d/db5/dc0/f2e [4194304,4194304] 0 2024-04-22T23:46:19.582 INFO:tasks.workunit.client.1.smithi044.stdout:7/789: dwrite dd/d1a/f90 [0,4194304] 0 2024-04-22T23:46:19.584 INFO:tasks.workunit.client.1.smithi044.stdout:9/661: readlink d2/dc/d1d/d3f/d9d/ld5 0 2024-04-22T23:46:19.585 INFO:tasks.workunit.client.1.smithi044.stdout:9/662: readlink d2/d6/l2f 0 2024-04-22T23:46:19.590 INFO:tasks.workunit.client.1.smithi044.stdout:3/835: creat d9/db/d9f/d38/f11a x:0 0 0 2024-04-22T23:46:19.592 INFO:tasks.workunit.client.1.smithi044.stdout:3/836: write d9/dc8/ff4 [607271,73178] 0 2024-04-22T23:46:19.593 INFO:tasks.workunit.client.1.smithi044.stdout:5/850: rmdir d15/d3c 39 2024-04-22T23:46:19.595 INFO:tasks.workunit.client.1.smithi044.stdout:3/837: write d9/db/d15/d25/d67/d6b/ded/d113/dbb/d10c/f118 [948354,23491] 0 2024-04-22T23:46:19.596 INFO:tasks.workunit.client.1.smithi044.stdout:7/790: mkdir dd/d55/d5e/d60/d68/d94/dbd/dc9/d123 0 2024-04-22T23:46:19.597 INFO:tasks.workunit.client.1.smithi044.stdout:7/791: fdatasync dd/d1a/f8e 0 2024-04-22T23:46:19.598 INFO:tasks.workunit.client.1.smithi044.stdout:6/682: write d5/d3f/d4c/d40/d53/fc9 [360331,98179] 0 2024-04-22T23:46:19.601 INFO:tasks.workunit.client.1.smithi044.stdout:7/792: fdatasync dd/d1f/d24/fa2 0 2024-04-22T23:46:19.602 INFO:tasks.workunit.client.1.smithi044.stdout:7/793: chown dd/d1a/db7/dc8 662 1 2024-04-22T23:46:19.604 INFO:tasks.workunit.client.1.smithi044.stdout:8/827: write db/d68/f77 [490649,65508] 0 2024-04-22T23:46:19.605 INFO:tasks.workunit.client.1.smithi044.stdout:5/851: rmdir d15/d16/d1d/d22/d4f/d68/d76 39 2024-04-22T23:46:19.609 INFO:tasks.workunit.client.1.smithi044.stdout:6/683: creat d5/d3f/d4c/d83/fd6 x:0 0 0 2024-04-22T23:46:19.611 INFO:tasks.workunit.client.1.smithi044.stdout:3/838: rename d9/db/d9f/d38/f41 to d9/d14/dd2/f11b 0 2024-04-22T23:46:19.618 INFO:tasks.workunit.client.1.smithi044.stdout:7/794: rename dd/d79/c100 to dd/df7/d112/c124 0 2024-04-22T23:46:19.620 INFO:tasks.workunit.client.1.smithi044.stdout:6/684: dwrite d5/d3f/d4c/d23/d3d/f4a [0,4194304] 0 2024-04-22T23:46:19.620 INFO:tasks.workunit.client.1.smithi044.stdout:9/663: dwrite d2/d6/d42/d8b/f98 [0,4194304] 0 2024-04-22T23:46:19.621 INFO:tasks.workunit.client.1.smithi044.stdout:3/839: rmdir d9/db 39 2024-04-22T23:46:19.624 INFO:tasks.workunit.client.1.smithi044.stdout:5/852: fdatasync d15/d16/d1d/d22/d2a/d3a/f4b 0 2024-04-22T23:46:19.629 INFO:tasks.workunit.client.1.smithi044.stdout:9/664: creat d2/d6/fd7 x:0 0 0 2024-04-22T23:46:19.629 INFO:tasks.workunit.client.1.smithi044.stdout:5/853: truncate d15/d5d/db5/dc0/d38/d9f/fc4 251914 0 2024-04-22T23:46:19.630 INFO:tasks.workunit.client.1.smithi044.stdout:5/854: stat d15/d16/d1d/d22/d2a/d3a/d43/fc6 0 2024-04-22T23:46:19.631 INFO:tasks.workunit.client.1.smithi044.stdout:6/685: unlink d5/dd/f27 0 2024-04-22T23:46:19.633 INFO:tasks.workunit.client.1.smithi044.stdout:7/795: symlink dd/d27/de7/d111/l125 0 2024-04-22T23:46:19.634 INFO:tasks.workunit.client.1.smithi044.stdout:7/796: write f5 [4329280,23632] 0 2024-04-22T23:46:19.636 INFO:tasks.workunit.client.1.smithi044.stdout:6/686: mknod d5/d3f/cd7 0 2024-04-22T23:46:19.637 INFO:tasks.workunit.client.1.smithi044.stdout:7/797: fsync dd/d55/d5e/d60/d68/d94/dbd/fea 0 2024-04-22T23:46:19.643 INFO:tasks.workunit.client.1.smithi044.stdout:5/855: creat d15/d16/d1d/d22/d2a/f11f x:0 0 0 2024-04-22T23:46:19.644 INFO:tasks.workunit.client.1.smithi044.stdout:7/798: rename dd/d1a/db7/dc8/laa to dd/d98/l126 0 2024-04-22T23:46:19.646 INFO:tasks.workunit.client.1.smithi044.stdout:7/799: write dd/d1a/d61/fb0 [5787350,83211] 0 2024-04-22T23:46:19.646 INFO:tasks.workunit.client.1.smithi044.stdout:5/856: read d15/d5d/d64/d6f/f8a [389699,54636] 0 2024-04-22T23:46:19.653 INFO:tasks.workunit.client.1.smithi044.stdout:3/840: dread d9/db/d15/f21 [0,4194304] 0 2024-04-22T23:46:19.653 INFO:tasks.workunit.client.1.smithi044.stdout:7/800: dwrite dd/d99/f86 [0,4194304] 0 2024-04-22T23:46:19.660 INFO:tasks.workunit.client.1.smithi044.stdout:3/841: mknod d9/d14/d7d/df8/c11c 0 2024-04-22T23:46:19.661 INFO:tasks.workunit.client.1.smithi044.stdout:7/801: mkdir dd/d55/d5e/d60/d68/d94/d127 0 2024-04-22T23:46:19.664 INFO:tasks.workunit.client.1.smithi044.stdout:5/857: creat d15/d16/d1d/d22/d4f/d68/f120 x:0 0 0 2024-04-22T23:46:19.667 INFO:tasks.workunit.client.1.smithi044.stdout:6/687: mknod d5/d3f/d4c/d23/cd8 0 2024-04-22T23:46:19.670 INFO:tasks.workunit.client.1.smithi044.stdout:5/858: read d15/d5d/d64/d6f/fbe [752075,124431] 0 2024-04-22T23:46:19.672 INFO:tasks.workunit.client.1.smithi044.stdout:6/688: creat d5/d30/d73/fd9 x:0 0 0 2024-04-22T23:46:19.674 INFO:tasks.workunit.client.1.smithi044.stdout:5/859: mkdir d15/d16/d1d/da2/d121 0 2024-04-22T23:46:19.675 INFO:tasks.workunit.client.1.smithi044.stdout:5/860: fdatasync d15/d5d/d64/d6f/ffd 0 2024-04-22T23:46:19.676 INFO:tasks.workunit.client.1.smithi044.stdout:5/861: truncate d15/d5d/d64/f10e 203072 0 2024-04-22T23:46:19.676 INFO:tasks.workunit.client.1.smithi044.stdout:5/862: dread - d15/d5d/db5/f11a zero size 2024-04-22T23:46:19.677 INFO:tasks.workunit.client.1.smithi044.stdout:6/689: chown d5/d3f/lb 9 1 2024-04-22T23:46:19.678 INFO:tasks.workunit.client.1.smithi044.stdout:6/690: chown d5/d3f/d4c/d83/la0 2 1 2024-04-22T23:46:19.683 INFO:tasks.workunit.client.1.smithi044.stdout:6/691: rename d5/d3f/d4c/f57 to d5/d3f/d4c/d23/d3d/d64/fda 0 2024-04-22T23:46:19.683 INFO:tasks.workunit.client.1.smithi044.stdout:6/692: readlink d5/l1c 0 2024-04-22T23:46:19.686 INFO:tasks.workunit.client.1.smithi044.stdout:3/842: truncate d9/db/d15/d2e/d48/fa1 7134037 0 2024-04-22T23:46:19.690 INFO:tasks.workunit.client.1.smithi044.stdout:5/863: creat d15/d16/d1d/d22/d2a/d3a/f122 x:0 0 0 2024-04-22T23:46:19.698 INFO:tasks.workunit.client.1.smithi044.stdout:7/802: dwrite dd/d1a/f7e [0,4194304] 0 2024-04-22T23:46:19.699 INFO:tasks.workunit.client.1.smithi044.stdout:5/864: truncate d15/d16/d1d/d22/d4f/d68/f120 919325 0 2024-04-22T23:46:19.700 INFO:tasks.workunit.client.1.smithi044.stdout:3/843: dwrite d9/d14/d7d/df8/dfd/d3c/d95/dcb/fff [0,4194304] 0 2024-04-22T23:46:19.703 INFO:tasks.workunit.client.1.smithi044.stdout:6/693: dread d5/dd/d28/f2e [0,4194304] 0 2024-04-22T23:46:19.706 INFO:tasks.workunit.client.1.smithi044.stdout:6/694: readlink d5/dd/d28/d75/d88/l8a 0 2024-04-22T23:46:19.712 INFO:tasks.workunit.client.1.smithi044.stdout:2/632: sync 2024-04-22T23:46:19.712 INFO:tasks.workunit.client.1.smithi044.stdout:1/820: sync 2024-04-22T23:46:19.713 INFO:tasks.workunit.client.1.smithi044.stdout:3/844: creat d9/d14/d7d/f11d x:0 0 0 2024-04-22T23:46:19.718 INFO:tasks.workunit.client.1.smithi044.stdout:6/695: dwrite d5/d30/d73/fd9 [0,4194304] 0 2024-04-22T23:46:19.718 INFO:tasks.workunit.client.1.smithi044.stdout:3/845: dread - d9/db/d15/d25/d4c/fee zero size 2024-04-22T23:46:19.719 INFO:tasks.workunit.client.1.smithi044.stdout:6/696: dread - d5/dd/d67/fd3 zero size 2024-04-22T23:46:19.719 INFO:tasks.workunit.client.1.smithi044.stdout:7/803: write dd/d1f/f2c [2826873,11342] 0 2024-04-22T23:46:19.721 INFO:tasks.workunit.client.1.smithi044.stdout:1/821: mkdir d1/d8/d20/dae/deb/d124 0 2024-04-22T23:46:19.721 INFO:tasks.workunit.client.1.smithi044.stdout:6/697: stat d5/d3f/d4c/d40 0 2024-04-22T23:46:19.724 INFO:tasks.workunit.client.1.smithi044.stdout:3/846: dread - d9/d14/d7d/df8/dfd/d3c/fb7 zero size 2024-04-22T23:46:19.726 INFO:tasks.workunit.client.1.smithi044.stdout:1/822: creat d1/d8/d1c/d7b/d80/dfd/dbf/ddb/f125 x:0 0 0 2024-04-22T23:46:19.729 INFO:tasks.workunit.client.1.smithi044.stdout:6/698: mkdir d5/d3f/d4c/d23/ddb 0 2024-04-22T23:46:19.735 INFO:tasks.workunit.client.1.smithi044.stdout:1/823: dwrite d1/d8/d1c/d23/f42 [0,4194304] 0 2024-04-22T23:46:19.738 INFO:tasks.workunit.client.1.smithi044.stdout:1/824: rmdir d1/d8/d20/d45/d75 39 2024-04-22T23:46:19.738 INFO:tasks.workunit.client.1.smithi044.stdout:7/804: write dd/d99/dbb/fd0 [428073,101395] 0 2024-04-22T23:46:19.742 INFO:tasks.workunit.client.1.smithi044.stdout:5/865: dread d15/d3c/f53 [0,4194304] 0 2024-04-22T23:46:19.749 INFO:tasks.workunit.client.1.smithi044.stdout:6/699: rename d5/d3f/d50/d79/l7b to d5/d3f/da7/ldc 0 2024-04-22T23:46:19.753 INFO:tasks.workunit.client.1.smithi044.stdout:2/633: dwrite d8/d19/d36/d46/ddb/f9c [0,4194304] 0 2024-04-22T23:46:19.758 INFO:tasks.workunit.client.1.smithi044.stdout:7/805: dwrite dd/d1f/fe5 [0,4194304] 0 2024-04-22T23:46:19.759 INFO:tasks.workunit.client.1.smithi044.stdout:6/700: write d5/dd/f9d [1907339,89166] 0 2024-04-22T23:46:19.766 INFO:tasks.workunit.client.1.smithi044.stdout:3/847: dwrite d9/db/d9f/f98 [0,4194304] 0 2024-04-22T23:46:19.768 INFO:tasks.workunit.client.1.smithi044.stdout:1/825: dwrite d1/d8/d1c/d7b/d80/dfd/dbf/fec [0,4194304] 0 2024-04-22T23:46:19.775 INFO:tasks.workunit.client.1.smithi044.stdout:5/866: dread d15/d5d/db5/f109 [0,4194304] 0 2024-04-22T23:46:19.779 INFO:tasks.workunit.client.1.smithi044.stdout:6/701: dwrite d5/dd/f9d [4194304,4194304] 0 2024-04-22T23:46:19.782 INFO:tasks.workunit.client.1.smithi044.stdout:1/826: dread - d1/d8/d1c/d74/f8e zero size 2024-04-22T23:46:19.785 INFO:tasks.workunit.client.1.smithi044.stdout:5/867: fsync d15/d16/d1d/f51 0 2024-04-22T23:46:19.786 INFO:tasks.workunit.client.1.smithi044.stdout:6/702: rmdir d5/d3f/d4c/d40/db9 39 2024-04-22T23:46:19.787 INFO:tasks.workunit.client.1.smithi044.stdout:6/703: write d5/f4d [223584,92335] 0 2024-04-22T23:46:19.788 INFO:tasks.workunit.client.1.smithi044.stdout:1/827: mknod d1/d8/d20/d45/d5d/c126 0 2024-04-22T23:46:19.792 INFO:tasks.workunit.client.1.smithi044.stdout:6/704: dread - d5/d3f/d4c/d4f/f59 zero size 2024-04-22T23:46:19.794 INFO:tasks.workunit.client.1.smithi044.stdout:2/634: dread d8/d19/d1c/f2f [0,4194304] 0 2024-04-22T23:46:19.794 INFO:tasks.workunit.client.1.smithi044.stdout:2/635: chown d8/d19/d36/d46 0 1 2024-04-22T23:46:19.794 INFO:tasks.workunit.client.1.smithi044.stdout:5/868: symlink d15/l123 0 2024-04-22T23:46:19.795 INFO:tasks.workunit.client.1.smithi044.stdout:6/705: symlink d5/d30/d73/ldd 0 2024-04-22T23:46:19.797 INFO:tasks.workunit.client.1.smithi044.stdout:6/706: write d5/d3f/d4c/d92/f98 [255152,126893] 0 2024-04-22T23:46:19.798 INFO:tasks.workunit.client.1.smithi044.stdout:2/636: mkdir d8/d19/d36/d46/d63/d65/de2 0 2024-04-22T23:46:19.799 INFO:tasks.workunit.client.1.smithi044.stdout:2/637: fsync d8/d19/f20 0 2024-04-22T23:46:19.802 INFO:tasks.workunit.client.1.smithi044.stdout:5/869: dread d15/d16/d1d/d22/d2a/d3a/d43/fa8 [0,4194304] 0 2024-04-22T23:46:19.805 INFO:tasks.workunit.client.1.smithi044.stdout:2/638: mknod d8/d60/dc1/ce3 0 2024-04-22T23:46:19.806 INFO:tasks.workunit.client.1.smithi044.stdout:3/848: dread d9/db/d15/f63 [0,4194304] 0 2024-04-22T23:46:19.807 INFO:tasks.workunit.client.1.smithi044.stdout:6/707: dwrite d5/d3f/d4c/d23/d3d/d64/fda [0,4194304] 0 2024-04-22T23:46:19.811 INFO:tasks.workunit.client.1.smithi044.stdout:2/639: creat d8/d19/d36/d46/fe4 x:0 0 0 2024-04-22T23:46:19.813 INFO:tasks.workunit.client.1.smithi044.stdout:7/806: dread dd/d1f/d24/f13 [0,4194304] 0 2024-04-22T23:46:19.819 INFO:tasks.workunit.client.1.smithi044.stdout:6/708: rename d5/c19 to d5/dd/d67/cde 0 2024-04-22T23:46:19.823 INFO:tasks.workunit.client.1.smithi044.stdout:3/849: dread f8 [0,4194304] 0 2024-04-22T23:46:19.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/709: mknod d5/cdf 0 2024-04-22T23:46:19.826 INFO:tasks.workunit.client.1.smithi044.stdout:5/870: dread f12 [0,4194304] 0 2024-04-22T23:46:19.831 INFO:tasks.workunit.client.1.smithi044.stdout:7/807: getdents dd/df7/d9c 0 2024-04-22T23:46:19.834 INFO:tasks.workunit.client.1.smithi044.stdout:2/640: dread d8/fd [0,4194304] 0 2024-04-22T23:46:19.839 INFO:tasks.workunit.client.1.smithi044.stdout:3/850: creat d9/db/d33/d5d/d70/dd8/f11e x:0 0 0 2024-04-22T23:46:19.840 INFO:tasks.workunit.client.1.smithi044.stdout:3/851: stat d9/d14/c4e 0 2024-04-22T23:46:19.841 INFO:tasks.workunit.client.1.smithi044.stdout:3/852: stat d9/db/d33/d5d/d70/dd8 0 2024-04-22T23:46:19.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/641: truncate d8/d27/d5b/d74/f79 71217 0 2024-04-22T23:46:19.841 INFO:tasks.workunit.client.1.smithi044.stdout:3/853: chown d9/db/d9f/ce7 5247 1 2024-04-22T23:46:19.842 INFO:tasks.workunit.client.1.smithi044.stdout:3/854: chown d9/db/d33/l77 138281 1 2024-04-22T23:46:19.846 INFO:tasks.workunit.client.1.smithi044.stdout:3/855: creat d9/db/d33/d5d/d64/f11f x:0 0 0 2024-04-22T23:46:19.847 INFO:tasks.workunit.client.1.smithi044.stdout:2/642: truncate d8/d60/f69 117090 0 2024-04-22T23:46:19.848 INFO:tasks.workunit.client.1.smithi044.stdout:2/643: write d8/d27/d5b/d74/fab [647577,91165] 0 2024-04-22T23:46:19.848 INFO:tasks.workunit.client.1.smithi044.stdout:3/856: symlink d9/db/d15/d2e/d48/l120 0 2024-04-22T23:46:19.850 INFO:tasks.workunit.client.1.smithi044.stdout:2/644: mkdir d8/d27/de5 0 2024-04-22T23:46:19.852 INFO:tasks.workunit.client.1.smithi044.stdout:3/857: dread - d9/db/d33/d5d/d64/dbe/fe9 zero size 2024-04-22T23:46:19.856 INFO:tasks.workunit.client.1.smithi044.stdout:3/858: mkdir d9/db/d15/d2e/d48/d121 0 2024-04-22T23:46:19.868 INFO:tasks.workunit.client.1.smithi044.stdout:5/871: dwrite d15/d5d/db5/f109 [4194304,4194304] 0 2024-04-22T23:46:19.869 INFO:tasks.workunit.client.1.smithi044.stdout:3/859: dwrite d9/db/d15/d2e/d48/f102 [0,4194304] 0 2024-04-22T23:46:19.872 INFO:tasks.workunit.client.1.smithi044.stdout:6/710: dwrite d5/dd/d28/f2c [0,4194304] 0 2024-04-22T23:46:19.877 INFO:tasks.workunit.client.1.smithi044.stdout:3/860: chown d9/d14/d7d/df8/dfd/d3c/fb7 32 1 2024-04-22T23:46:19.877 INFO:tasks.workunit.client.1.smithi044.stdout:7/808: dwrite dd/d1f/d24/f3f [0,4194304] 0 2024-04-22T23:46:19.878 INFO:tasks.workunit.client.1.smithi044.stdout:3/861: truncate d9/dce/fef 1474015 0 2024-04-22T23:46:19.881 INFO:tasks.workunit.client.1.smithi044.stdout:3/862: dread - d9/db/d9f/d38/f11a zero size 2024-04-22T23:46:19.885 INFO:tasks.workunit.client.1.smithi044.stdout:5/872: creat d15/d16/d1d/daf/f124 x:0 0 0 2024-04-22T23:46:19.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/711: mknod d5/ce0 0 2024-04-22T23:46:19.892 INFO:tasks.workunit.client.1.smithi044.stdout:6/712: write d5/d30/d73/fd9 [2650806,55701] 0 2024-04-22T23:46:19.897 INFO:tasks.workunit.client.1.smithi044.stdout:6/713: dread d5/d3f/d4c/d92/f98 [0,4194304] 0 2024-04-22T23:46:19.900 INFO:tasks.workunit.client.1.smithi044.stdout:7/809: rename dd/d55/d5e/d60/d68/d94/dbd/dc9/fed to dd/d55/d5e/d60/d68/d94/dbd/dd8/f128 0 2024-04-22T23:46:19.902 INFO:tasks.workunit.client.1.smithi044.stdout:3/863: mknod d9/db/d33/d5d/c122 0 2024-04-22T23:46:19.905 INFO:tasks.workunit.client.1.smithi044.stdout:5/873: dread d15/f94 [0,4194304] 0 2024-04-22T23:46:19.919 INFO:tasks.workunit.client.1.smithi044.stdout:7/810: rmdir dd/d1a/db7 39 2024-04-22T23:46:19.921 INFO:tasks.workunit.client.1.smithi044.stdout:2/645: dwrite d8/d19/d36/d46/d63/d65/d94/fa8 [0,4194304] 0 2024-04-22T23:46:19.932 INFO:tasks.workunit.client.1.smithi044.stdout:7/811: chown dd/d1a/db7/dc8/d97/fe6 484442791 1 2024-04-22T23:46:19.933 INFO:tasks.workunit.client.1.smithi044.stdout:7/812: stat dd/d1a/c41 0 2024-04-22T23:46:19.937 INFO:tasks.workunit.client.1.smithi044.stdout:2/646: mknod d8/ce6 0 2024-04-22T23:46:19.945 INFO:tasks.workunit.client.1.smithi044.stdout:6/714: getdents d5/dd/d28/d75/d88 0 2024-04-22T23:46:19.947 INFO:tasks.workunit.client.1.smithi044.stdout:7/813: symlink dd/d1a/db7/dc8/d97/db9/l129 0 2024-04-22T23:46:19.948 INFO:tasks.workunit.client.1.smithi044.stdout:7/814: chown dd/d1a/f7e 0 1 2024-04-22T23:46:19.948 INFO:tasks.workunit.client.1.smithi044.stdout:7/815: stat dd/d27/fb5 0 2024-04-22T23:46:19.950 INFO:tasks.workunit.client.1.smithi044.stdout:7/816: write dd/d1f/f6a [214008,124766] 0 2024-04-22T23:46:19.952 INFO:tasks.workunit.client.1.smithi044.stdout:2/647: unlink d8/d19/d36/d46/d63/d65/d72/f8e 0 2024-04-22T23:46:19.954 INFO:tasks.workunit.client.1.smithi044.stdout:5/874: link d15/d16/d1d/d22/d2a/d3a/c56 d15/c125 0 2024-04-22T23:46:19.956 INFO:tasks.workunit.client.1.smithi044.stdout:6/715: stat d5/dd/c37 0 2024-04-22T23:46:19.958 INFO:tasks.workunit.client.1.smithi044.stdout:3/864: dread d9/db/d9f/d38/f4a [0,4194304] 0 2024-04-22T23:46:19.960 INFO:tasks.workunit.client.1.smithi044.stdout:7/817: rmdir dd/d1f 39 2024-04-22T23:46:19.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/648: mknod d8/d19/d5d/ce7 0 2024-04-22T23:46:19.963 INFO:tasks.workunit.client.1.smithi044.stdout:5/875: symlink d15/d16/d1d/d22/df5/l126 0 2024-04-22T23:46:19.964 INFO:tasks.workunit.client.1.smithi044.stdout:3/865: rmdir d9/d14/d7d/df8 39 2024-04-22T23:46:19.967 INFO:tasks.workunit.client.1.smithi044.stdout:2/649: read - d8/d19/d5d/f7e zero size 2024-04-22T23:46:19.967 INFO:tasks.workunit.client.1.smithi044.stdout:7/818: write dd/d1f/d24/f3f [1262900,36935] 0 2024-04-22T23:46:19.968 INFO:tasks.workunit.client.1.smithi044.stdout:7/819: dread - dd/d98/de8/ff0 zero size 2024-04-22T23:46:19.968 INFO:tasks.workunit.client.1.smithi044.stdout:4/860: sync 2024-04-22T23:46:19.976 INFO:tasks.workunit.client.1.smithi044.stdout:2/650: dwrite d8/d19/d36/d46/d63/d65/f84 [0,4194304] 0 2024-04-22T23:46:19.976 INFO:tasks.workunit.client.1.smithi044.stdout:5/876: mkdir d15/d16/dbc/d116/d115/d117/d127 0 2024-04-22T23:46:19.979 INFO:tasks.workunit.client.1.smithi044.stdout:7/820: creat dd/df7/d9c/f12a x:0 0 0 2024-04-22T23:46:19.979 INFO:tasks.workunit.client.1.smithi044.stdout:5/877: truncate d15/d16/d1d/d22/df5/f107 725310 0 2024-04-22T23:46:19.980 INFO:tasks.workunit.client.1.smithi044.stdout:7/821: readlink dd/d55/d5e/d60/d75/ldf 0 2024-04-22T23:46:19.981 INFO:tasks.workunit.client.1.smithi044.stdout:4/861: mknod d5/d31/d24/d38/d84/c123 0 2024-04-22T23:46:19.985 INFO:tasks.workunit.client.1.smithi044.stdout:3/866: mknod d9/d14/d7d/df8/dfd/d3c/d95/c123 0 2024-04-22T23:46:19.986 INFO:tasks.workunit.client.1.smithi044.stdout:6/716: getdents d5/dd/d28 0 2024-04-22T23:46:19.990 INFO:tasks.workunit.client.1.smithi044.stdout:2/651: mkdir d8/d19/d36/d93/dae/de8 0 2024-04-22T23:46:19.991 INFO:tasks.workunit.client.1.smithi044.stdout:2/652: fdatasync d8/d19/d36/d93/dbf/fd5 0 2024-04-22T23:46:19.992 INFO:tasks.workunit.client.1.smithi044.stdout:5/878: creat d15/d16/d1d/daf/f128 x:0 0 0 2024-04-22T23:46:19.993 INFO:tasks.workunit.client.1.smithi044.stdout:7/822: symlink dd/d55/d5e/d60/d7b/l12b 0 2024-04-22T23:46:19.993 INFO:tasks.workunit.client.1.smithi044.stdout:7/823: fdatasync fc 0 2024-04-22T23:46:19.999 INFO:tasks.workunit.client.1.smithi044.stdout:6/717: symlink d5/d3f/d4c/d40/d74/le1 0 2024-04-22T23:46:20.000 INFO:tasks.workunit.client.1.smithi044.stdout:4/862: dwrite d5/d31/d44/d20/d22/f67 [4194304,4194304] 0 2024-04-22T23:46:20.009 INFO:tasks.workunit.client.1.smithi044.stdout:5/879: fsync d15/d5d/d82/f83 0 2024-04-22T23:46:20.010 INFO:tasks.workunit.client.1.smithi044.stdout:3/867: truncate d9/db/d15/d25/d4c/f75 1554727 0 2024-04-22T23:46:20.010 INFO:tasks.workunit.client.1.smithi044.stdout:5/880: readlink d15/d16/l87 0 2024-04-22T23:46:20.014 INFO:tasks.workunit.client.1.smithi044.stdout:3/868: write d9/db/d15/d2e/d48/f102 [2913836,27377] 0 2024-04-22T23:46:20.014 INFO:tasks.workunit.client.1.smithi044.stdout:4/863: creat d5/db/d34/f124 x:0 0 0 2024-04-22T23:46:20.015 INFO:tasks.workunit.client.1.smithi044.stdout:0/961: sync 2024-04-22T23:46:20.015 INFO:tasks.workunit.client.1.smithi044.stdout:5/881: read d15/d5d/db5/dc0/d38/fa5 [3671579,123379] 0 2024-04-22T23:46:20.018 INFO:tasks.workunit.client.1.smithi044.stdout:3/869: truncate d9/db/d15/d25/d4c/fee 214742 0 2024-04-22T23:46:20.034 INFO:tasks.workunit.client.1.smithi044.stdout:3/870: dread f5 [0,4194304] 0 2024-04-22T23:46:20.034 INFO:tasks.workunit.client.1.smithi044.stdout:5/882: dread f11 [0,4194304] 0 2024-04-22T23:46:20.051 INFO:tasks.workunit.client.1.smithi044.stdout:7/824: symlink dd/d55/d5e/d60/d68/d94/dbd/dc9/d123/l12c 0 2024-04-22T23:46:20.053 INFO:tasks.workunit.client.1.smithi044.stdout:7/825: write dd/d1a/db7/dc8/d97/db9/f105 [898080,23586] 0 2024-04-22T23:46:20.053 INFO:tasks.workunit.client.1.smithi044.stdout:6/718: creat d5/d3f/d4c/d40/d53/d56/db7/fe2 x:0 0 0 2024-04-22T23:46:20.057 INFO:tasks.workunit.client.1.smithi044.stdout:2/653: truncate d8/d19/d36/d44/f97 35262 0 2024-04-22T23:46:20.059 INFO:tasks.workunit.client.1.smithi044.stdout:0/962: mkdir d0/d4c/dbd/d156 0 2024-04-22T23:46:20.061 INFO:tasks.workunit.client.1.smithi044.stdout:6/719: dwrite d5/d3f/d50/d79/f8c [0,4194304] 0 2024-04-22T23:46:20.062 INFO:tasks.workunit.client.1.smithi044.stdout:3/871: mknod d9/d12/db1/c124 0 2024-04-22T23:46:20.064 INFO:tasks.workunit.client.1.smithi044.stdout:3/872: chown d9/db/d15/d2e/d48/f73 1143 1 2024-04-22T23:46:20.064 INFO:tasks.workunit.client.1.smithi044.stdout:8/828: sync 2024-04-22T23:46:20.064 INFO:tasks.workunit.client.1.smithi044.stdout:9/665: sync 2024-04-22T23:46:20.064 INFO:tasks.workunit.client.1.smithi044.stdout:1/828: sync 2024-04-22T23:46:20.065 INFO:tasks.workunit.client.1.smithi044.stdout:8/829: chown c2 37 1 2024-04-22T23:46:20.073 INFO:tasks.workunit.client.1.smithi044.stdout:3/873: dwrite d9/db/d9f/f10b [0,4194304] 0 2024-04-22T23:46:20.074 INFO:tasks.workunit.client.1.smithi044.stdout:2/654: rename d8/d60/dc1/la0 to d8/d19/d1c/le9 0 2024-04-22T23:46:20.079 INFO:tasks.workunit.client.1.smithi044.stdout:4/864: dread d5/d42/fbe [0,4194304] 0 2024-04-22T23:46:20.156 INFO:tasks.workunit.client.1.smithi044.stdout:4/865: symlink d5/d31/d24/d69/d90/l125 0 2024-04-22T23:46:20.157 INFO:tasks.workunit.client.1.smithi044.stdout:6/720: mkdir d5/dd/d28/d75/d88/dcd/de3 0 2024-04-22T23:46:20.159 INFO:tasks.workunit.client.1.smithi044.stdout:5/883: creat d15/d16/d1d/d22/d4f/d68/f129 x:0 0 0 2024-04-22T23:46:20.164 INFO:tasks.workunit.client.1.smithi044.stdout:9/666: creat d2/dc/d1d/d34/d38/d4a/dc7/fd8 x:0 0 0 2024-04-22T23:46:20.164 INFO:tasks.workunit.client.1.smithi044.stdout:9/667: truncate d2/dc/d94/fc9 180323 0 2024-04-22T23:46:20.167 INFO:tasks.workunit.client.1.smithi044.stdout:0/963: rename d0/d2/l29 to d0/d2/d4b/dad/dc8/d10e/l157 0 2024-04-22T23:46:20.168 INFO:tasks.workunit.client.1.smithi044.stdout:0/964: chown d0/d2/d4b/dad/dc8/d10e 799 1 2024-04-22T23:46:20.171 INFO:tasks.workunit.client.1.smithi044.stdout:3/874: fsync d9/db/f10 0 2024-04-22T23:46:20.176 INFO:tasks.workunit.client.1.smithi044.stdout:4/866: rmdir d5/d31/d44/d20/d93 39 2024-04-22T23:46:20.187 INFO:tasks.workunit.client.1.smithi044.stdout:9/668: creat d2/dc/d1d/d3f/fd9 x:0 0 0 2024-04-22T23:46:20.188 INFO:tasks.workunit.client.1.smithi044.stdout:7/826: link dd/d1f/d24/f1c dd/f12d 0 2024-04-22T23:46:20.196 INFO:tasks.workunit.client.1.smithi044.stdout:2/655: creat d8/d19/d36/d44/dd8/fea x:0 0 0 2024-04-22T23:46:20.198 INFO:tasks.workunit.client.1.smithi044.stdout:3/875: creat d9/d12/db1/d101/f125 x:0 0 0 2024-04-22T23:46:20.209 INFO:tasks.workunit.client.1.smithi044.stdout:1/829: getdents d1/d8/d1c/d7b 0 2024-04-22T23:46:20.212 INFO:tasks.workunit.client.1.smithi044.stdout:9/669: unlink d2/d9/f6a 0 2024-04-22T23:46:20.214 INFO:tasks.workunit.client.1.smithi044.stdout:4/867: dread d5/d31/d24/fde [0,4194304] 0 2024-04-22T23:46:20.216 INFO:tasks.workunit.client.1.smithi044.stdout:7/827: mkdir dd/d98/de8/d12e 0 2024-04-22T23:46:20.233 INFO:tasks.workunit.client.1.smithi044.stdout:1/830: chown d1/d8/d20/d45/d75/df2/l107 1523 1 2024-04-22T23:46:20.238 INFO:tasks.workunit.client.1.smithi044.stdout:4/868: rmdir d5/d31/d24/d38/d6a 39 2024-04-22T23:46:20.239 INFO:tasks.workunit.client.1.smithi044.stdout:1/831: dwrite d1/d8/d1c/d7b/f8b [0,4194304] 0 2024-04-22T23:46:20.242 INFO:tasks.workunit.client.1.smithi044.stdout:7/828: creat dd/d55/d5e/d60/d68/d94/dbd/dc9/d123/f12f x:0 0 0 2024-04-22T23:46:20.249 INFO:tasks.workunit.client.1.smithi044.stdout:7/829: truncate dd/d55/d5e/d60/d75/f92 435589 0 2024-04-22T23:46:20.255 INFO:tasks.workunit.client.1.smithi044.stdout:7/830: dwrite dd/d1f/fe5 [0,4194304] 0 2024-04-22T23:46:20.255 INFO:tasks.workunit.client.1.smithi044.stdout:9/670: link d2/dc/d1d/d3f/f48 d2/dc/d1d/d34/d38/d5c/d69/d9f/fda 0 2024-04-22T23:46:20.256 INFO:tasks.workunit.client.1.smithi044.stdout:9/671: readlink d2/d6/lc6 0 2024-04-22T23:46:20.258 INFO:tasks.workunit.client.1.smithi044.stdout:1/832: creat d1/f127 x:0 0 0 2024-04-22T23:46:20.260 INFO:tasks.workunit.client.1.smithi044.stdout:9/672: mknod d2/d6/d42/daa/cdb 0 2024-04-22T23:46:20.261 INFO:tasks.workunit.client.1.smithi044.stdout:1/833: symlink d1/d8/d1c/d7b/d80/de3/l128 0 2024-04-22T23:46:20.262 INFO:tasks.workunit.client.1.smithi044.stdout:9/673: write d2/dc/d1d/d34/d38/d5c/d69/d75/fb5 [764707,31901] 0 2024-04-22T23:46:20.267 INFO:tasks.workunit.client.1.smithi044.stdout:9/674: mkdir d2/d6/d42/d8b/ddc 0 2024-04-22T23:46:20.283 INFO:tasks.workunit.client.1.smithi044.stdout:5/884: symlink d15/d16/d1d/d22/d2a/d3a/l12a 0 2024-04-22T23:46:20.289 INFO:tasks.workunit.client.1.smithi044.stdout:7/831: dread dd/d27/fb5 [0,4194304] 0 2024-04-22T23:46:20.290 INFO:tasks.workunit.client.1.smithi044.stdout:8/830: dwrite db/d68/d8b/da7/fbc [0,4194304] 0 2024-04-22T23:46:20.296 INFO:tasks.workunit.client.1.smithi044.stdout:7/832: mknod dd/d55/d5e/d60/d68/d94/dbd/dc9/d123/c130 0 2024-04-22T23:46:20.297 INFO:tasks.workunit.client.1.smithi044.stdout:8/831: getdents db/ddd/df2 0 2024-04-22T23:46:20.298 INFO:tasks.workunit.client.1.smithi044.stdout:8/832: truncate db/d14/f11d 748832 0 2024-04-22T23:46:20.303 INFO:tasks.workunit.client.1.smithi044.stdout:7/833: dwrite dd/df7/db8/fee [0,4194304] 0 2024-04-22T23:46:20.306 INFO:tasks.workunit.client.1.smithi044.stdout:8/833: getdents db/d14/d9e 0 2024-04-22T23:46:20.309 INFO:tasks.workunit.client.1.smithi044.stdout:8/834: write db/d14/dc7/f122 [455246,40175] 0 2024-04-22T23:46:20.310 INFO:tasks.workunit.client.1.smithi044.stdout:8/835: stat db/d14/d72/cb2 0 2024-04-22T23:46:20.311 INFO:tasks.workunit.client.1.smithi044.stdout:7/834: dwrite dd/d55/d5e/d60/ffa [0,4194304] 0 2024-04-22T23:46:20.318 INFO:tasks.workunit.client.1.smithi044.stdout:8/836: mknod db/d52/d9c/dad/dc4/d10e/c125 0 2024-04-22T23:46:20.322 INFO:tasks.workunit.client.1.smithi044.stdout:8/837: dread - db/d68/d8b/da7/fae zero size 2024-04-22T23:46:20.329 INFO:tasks.workunit.client.1.smithi044.stdout:8/838: rename db/d68/d8b/dca/f11c to db/d52/d9c/f126 0 2024-04-22T23:46:20.332 INFO:tasks.workunit.client.1.smithi044.stdout:8/839: fdatasync db/f38 0 2024-04-22T23:46:20.333 INFO:tasks.workunit.client.1.smithi044.stdout:6/721: write d5/d3f/d4c/db4/fcb [1254752,65373] 0 2024-04-22T23:46:20.343 INFO:tasks.workunit.client.1.smithi044.stdout:7/835: dread dd/d55/f58 [0,4194304] 0 2024-04-22T23:46:20.345 INFO:tasks.workunit.client.1.smithi044.stdout:6/722: dread d5/d3f/d4c/d23/f26 [0,4194304] 0 2024-04-22T23:46:20.345 INFO:tasks.workunit.client.1.smithi044.stdout:7/836: write dd/d99/deb/f104 [664771,8359] 0 2024-04-22T23:46:20.346 INFO:tasks.workunit.client.1.smithi044.stdout:6/723: stat d5/dd/f68 0 2024-04-22T23:46:20.347 INFO:tasks.workunit.client.1.smithi044.stdout:7/837: chown dd/d55/d5e/d60/d106/l120 0 1 2024-04-22T23:46:20.351 INFO:tasks.workunit.client.1.smithi044.stdout:2/656: dwrite d8/f1a [0,4194304] 0 2024-04-22T23:46:20.354 INFO:tasks.workunit.client.1.smithi044.stdout:3/876: write d9/d14/d7d/df8/dfd/d3c/fb7 [93154,40463] 0 2024-04-22T23:46:20.361 INFO:tasks.workunit.client.1.smithi044.stdout:2/657: creat d8/d19/d36/d93/dae/de8/feb x:0 0 0 2024-04-22T23:46:20.365 INFO:tasks.workunit.client.1.smithi044.stdout:7/838: getdents dd/d55/d5e/d60/d68/d94/dbd/dd8 0 2024-04-22T23:46:20.367 INFO:tasks.workunit.client.1.smithi044.stdout:2/658: creat d8/d19/d5d/fec x:0 0 0 2024-04-22T23:46:20.403 INFO:tasks.workunit.client.1.smithi044.stdout:2/659: rename d8/d19/d5d/f83 to d8/d19/fed 0 2024-04-22T23:46:20.403 INFO:tasks.workunit.client.1.smithi044.stdout:1/834: write d1/d8/d20/d45/d75/df2/f94 [431550,103993] 0 2024-04-22T23:46:20.403 INFO:tasks.workunit.client.1.smithi044.stdout:9/675: creat d2/dc/fdd x:0 0 0 2024-04-22T23:46:20.404 INFO:tasks.workunit.client.1.smithi044.stdout:0/965: creat d0/f158 x:0 0 0 2024-04-22T23:46:20.406 INFO:tasks.workunit.client.1.smithi044.stdout:3/877: creat d9/db/d15/d2e/f126 x:0 0 0 2024-04-22T23:46:20.407 INFO:tasks.workunit.client.1.smithi044.stdout:8/840: write db/d14/d24/f5c [3880949,1596] 0 2024-04-22T23:46:20.407 INFO:tasks.workunit.client.1.smithi044.stdout:6/724: write d5/d3f/d4c/f3a [3781127,75140] 0 2024-04-22T23:46:20.411 INFO:tasks.workunit.client.1.smithi044.stdout:6/725: stat d5/d3f/d4c/d4f/fad 0 2024-04-22T23:46:20.412 INFO:tasks.workunit.client.1.smithi044.stdout:9/676: creat d2/dc/d1d/d34/d38/d5c/d7b/fde x:0 0 0 2024-04-22T23:46:20.413 INFO:tasks.workunit.client.1.smithi044.stdout:5/885: link d15/d16/d1d/d22/d2a/d3a/c56 d15/d16/d1d/d22/c12b 0 2024-04-22T23:46:20.414 INFO:tasks.workunit.client.1.smithi044.stdout:7/839: dread dd/d55/d5e/d60/d7b/f7f [0,4194304] 0 2024-04-22T23:46:20.418 INFO:tasks.workunit.client.1.smithi044.stdout:0/966: link d0/d60/de6/d109/fd8 d0/d4c/d123/d142/f159 0 2024-04-22T23:46:20.419 INFO:tasks.workunit.client.1.smithi044.stdout:1/835: dwrite d1/d8/d20/d45/d5d/fdf [0,4194304] 0 2024-04-22T23:46:20.422 INFO:tasks.workunit.client.1.smithi044.stdout:6/726: read d5/d3f/f63 [1856187,34376] 0 2024-04-22T23:46:20.424 INFO:tasks.workunit.client.1.smithi044.stdout:7/840: symlink dd/d98/de8/l131 0 2024-04-22T23:46:20.425 INFO:tasks.workunit.client.1.smithi044.stdout:9/677: dread - d2/dc/d1d/d34/d38/d5c/d69/d9f/fa0 zero size 2024-04-22T23:46:20.426 INFO:tasks.workunit.client.1.smithi044.stdout:8/841: dwrite db/d14/d72/ffd [0,4194304] 0 2024-04-22T23:46:20.429 INFO:tasks.workunit.client.1.smithi044.stdout:5/886: symlink d15/d16/d1d/d22/d4f/dc9/d8d/l12c 0 2024-04-22T23:46:20.431 INFO:tasks.workunit.client.1.smithi044.stdout:5/887: read f12 [3903910,10734] 0 2024-04-22T23:46:20.432 INFO:tasks.workunit.client.1.smithi044.stdout:0/967: chown d0/d2/dd/d3e/ca5 7 1 2024-04-22T23:46:20.434 INFO:tasks.workunit.client.1.smithi044.stdout:3/878: rename d9/db/d15/d25/d67/d6b/la4 to d9/d12/l127 0 2024-04-22T23:46:20.434 INFO:tasks.workunit.client.1.smithi044.stdout:2/660: dwrite d8/d27/d5b/d74/fab [0,4194304] 0 2024-04-22T23:46:20.435 INFO:tasks.workunit.client.1.smithi044.stdout:3/879: chown d9/db/d33 0 1 2024-04-22T23:46:20.438 INFO:tasks.workunit.client.1.smithi044.stdout:7/841: symlink dd/d1a/db7/l132 0 2024-04-22T23:46:20.442 INFO:tasks.workunit.client.1.smithi044.stdout:9/678: rmdir d2/dc/d1d/d34/d38/d5c/d69/d7a/da7 39 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: pgmap v13: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 37 MiB/s rd, 122 MiB/s wr, 271 op/s 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:20 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:20.446 INFO:tasks.workunit.client.1.smithi044.stdout:0/968: unlink d0/l18 0 2024-04-22T23:46:20.449 INFO:tasks.workunit.client.1.smithi044.stdout:2/661: creat d8/d19/d1c/daf/dd0/fee x:0 0 0 2024-04-22T23:46:20.458 INFO:tasks.workunit.client.1.smithi044.stdout:6/727: creat d5/d3f/d4c/d40/db9/fe4 x:0 0 0 2024-04-22T23:46:20.459 INFO:tasks.workunit.client.1.smithi044.stdout:6/728: chown d5/d3f/d4c/d23/f26 70 1 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: pgmap v13: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 37 MiB/s rd, 122 MiB/s wr, 271 op/s 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:20.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:20 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:20.463 INFO:tasks.workunit.client.1.smithi044.stdout:7/842: write dd/d27/f5b [624899,50343] 0 2024-04-22T23:46:20.467 INFO:tasks.workunit.client.1.smithi044.stdout:8/842: write db/d52/d9c/fc6 [785984,13271] 0 2024-04-22T23:46:20.467 INFO:tasks.workunit.client.1.smithi044.stdout:0/969: chown d0/d2/d4b/dad/dc8/d10e/d11b/f140 1 1 2024-04-22T23:46:20.469 INFO:tasks.workunit.client.1.smithi044.stdout:3/880: dwrite d9/db/d9f/d38/f4a [4194304,4194304] 0 2024-04-22T23:46:20.471 INFO:tasks.workunit.client.1.smithi044.stdout:2/662: unlink d8/d19/d36/d46/d63/d65/d94/fa7 0 2024-04-22T23:46:20.473 INFO:tasks.workunit.client.1.smithi044.stdout:6/729: write d5/d3f/d4c/d23/f26 [736030,82176] 0 2024-04-22T23:46:20.477 INFO:tasks.workunit.client.1.smithi044.stdout:5/888: creat d15/d16/d1d/d22/f12d x:0 0 0 2024-04-22T23:46:20.478 INFO:tasks.workunit.client.1.smithi044.stdout:7/843: rename dd/d1a/d61/fa6 to dd/d27/de7/d111/f133 0 2024-04-22T23:46:20.479 INFO:tasks.workunit.client.1.smithi044.stdout:3/881: dwrite d9/d14/d7d/df8/dfd/d3c/fb7 [0,4194304] 0 2024-04-22T23:46:20.481 INFO:tasks.workunit.client.1.smithi044.stdout:1/836: getdents d1/d8/d1c/d23 0 2024-04-22T23:46:20.487 INFO:tasks.workunit.client.1.smithi044.stdout:3/882: dread d9/db/d9f/f24 [0,4194304] 0 2024-04-22T23:46:20.491 INFO:tasks.workunit.client.1.smithi044.stdout:0/970: rmdir d0/d2/dd/d3e/d40/d43/de4/d146/d2f 39 2024-04-22T23:46:20.494 INFO:tasks.workunit.client.1.smithi044.stdout:2/663: dread - d8/d19/d36/d46/f89 zero size 2024-04-22T23:46:20.499 INFO:tasks.workunit.client.1.smithi044.stdout:6/730: mkdir d5/d3f/d4c/d92/d97/de5 0 2024-04-22T23:46:20.500 INFO:tasks.workunit.client.1.smithi044.stdout:5/889: fsync d15/d16/d1d/d22/d4f/d68/fc3 0 2024-04-22T23:46:20.505 INFO:tasks.workunit.client.1.smithi044.stdout:5/890: dread - d15/d5d/d64/d6f/f106 zero size 2024-04-22T23:46:20.509 INFO:tasks.workunit.client.1.smithi044.stdout:8/843: mkdir db/d14/d24/d127 0 2024-04-22T23:46:20.513 INFO:tasks.workunit.client.1.smithi044.stdout:7/844: dwrite dd/d1a/f7d [0,4194304] 0 2024-04-22T23:46:20.514 INFO:tasks.workunit.client.1.smithi044.stdout:9/679: getdents d2/d6/d42/d8b 0 2024-04-22T23:46:20.515 INFO:tasks.workunit.client.1.smithi044.stdout:6/731: mkdir d5/d3f/d50/de6 0 2024-04-22T23:46:20.518 INFO:tasks.workunit.client.1.smithi044.stdout:7/845: chown dd/d55/d5e/d60/d68/d94/cae 20 1 2024-04-22T23:46:20.520 INFO:tasks.workunit.client.1.smithi044.stdout:8/844: unlink db/d14/l6d 0 2024-04-22T23:46:20.523 INFO:tasks.workunit.client.1.smithi044.stdout:2/664: dwrite d8/d27/f2a [0,4194304] 0 2024-04-22T23:46:20.525 INFO:tasks.workunit.client.1.smithi044.stdout:3/883: dwrite d9/db/d15/d25/d4c/d86/fca [0,4194304] 0 2024-04-22T23:46:20.528 INFO:tasks.workunit.client.1.smithi044.stdout:1/837: rename d1/d8/d20/d45/d75/df2/d99/f10e to d1/d8/d20/d47/d4a/f129 0 2024-04-22T23:46:20.533 INFO:tasks.workunit.client.1.smithi044.stdout:9/680: creat d2/dc/d1d/d34/d38/d5c/fdf x:0 0 0 2024-04-22T23:46:20.534 INFO:tasks.workunit.client.1.smithi044.stdout:7/846: dwrite dd/df7/db8/f109 [0,4194304] 0 2024-04-22T23:46:20.540 INFO:tasks.workunit.client.1.smithi044.stdout:7/847: dread dd/d1f/f6a [0,4194304] 0 2024-04-22T23:46:20.541 INFO:tasks.workunit.client.1.smithi044.stdout:2/665: dread - d8/d27/f4f zero size 2024-04-22T23:46:20.544 INFO:tasks.workunit.client.1.smithi044.stdout:9/681: dwrite d2/dc/d94/fc3 [0,4194304] 0 2024-04-22T23:46:20.546 INFO:tasks.workunit.client.1.smithi044.stdout:5/891: creat d15/f12e x:0 0 0 2024-04-22T23:46:20.547 INFO:tasks.workunit.client.1.smithi044.stdout:3/884: mkdir d9/d14/d128 0 2024-04-22T23:46:20.555 INFO:tasks.workunit.client.1.smithi044.stdout:0/971: rename d0/d2/dd/d3e/d40/d43/d7b/c24 to d0/d2/dd/c15a 0 2024-04-22T23:46:20.556 INFO:tasks.workunit.client.1.smithi044.stdout:7/848: dread dd/d1f/fa0 [0,4194304] 0 2024-04-22T23:46:20.557 INFO:tasks.workunit.client.1.smithi044.stdout:1/838: creat d1/d8/d1c/d7b/d80/dfd/f12a x:0 0 0 2024-04-22T23:46:20.558 INFO:tasks.workunit.client.1.smithi044.stdout:7/849: write dd/d55/d5e/fe4 [591055,24278] 0 2024-04-22T23:46:20.559 INFO:tasks.workunit.client.1.smithi044.stdout:1/839: truncate d1/d8/d20/d47/d4a/f129 21853 0 2024-04-22T23:46:20.568 INFO:tasks.workunit.client.1.smithi044.stdout:3/885: rmdir d9/db/d15/d25/d4c/d86 39 2024-04-22T23:46:20.571 INFO:tasks.workunit.client.1.smithi044.stdout:2/666: dwrite d8/d19/d1c/f2f [0,4194304] 0 2024-04-22T23:46:20.578 INFO:tasks.workunit.client.1.smithi044.stdout:7/850: stat dd/d98/de8/df2/c110 0 2024-04-22T23:46:20.578 INFO:tasks.workunit.client.1.smithi044.stdout:3/886: dwrite f7 [0,4194304] 0 2024-04-22T23:46:20.580 INFO:tasks.workunit.client.1.smithi044.stdout:3/887: chown d9/db/d15/d25/c4f 4 1 2024-04-22T23:46:20.586 INFO:tasks.workunit.client.1.smithi044.stdout:5/892: creat d15/d5d/d111/f12f x:0 0 0 2024-04-22T23:46:20.591 INFO:tasks.workunit.client.1.smithi044.stdout:2/667: unlink d8/d19/d1c/f98 0 2024-04-22T23:46:20.591 INFO:tasks.workunit.client.1.smithi044.stdout:6/732: getdents d5/d3f/d4c/d40/d74/daf 0 2024-04-22T23:46:20.592 INFO:tasks.workunit.client.1.smithi044.stdout:9/682: write d2/dc/d1d/d34/d38/d5c/f76 [4568121,78852] 0 2024-04-22T23:46:20.593 INFO:tasks.workunit.client.1.smithi044.stdout:9/683: fdatasync d2/dc/f87 0 2024-04-22T23:46:20.595 INFO:tasks.workunit.client.1.smithi044.stdout:9/684: truncate d2/d6/d42/d8b/fc5 309246 0 2024-04-22T23:46:20.596 INFO:tasks.workunit.client.1.smithi044.stdout:8/845: link db/c82 db/d68/d8b/ded/c128 0 2024-04-22T23:46:20.598 INFO:tasks.workunit.client.1.smithi044.stdout:5/893: creat d15/d5d/d82/f130 x:0 0 0 2024-04-22T23:46:20.601 INFO:tasks.workunit.client.1.smithi044.stdout:2/668: creat d8/d19/d96/fef x:0 0 0 2024-04-22T23:46:20.603 INFO:tasks.workunit.client.1.smithi044.stdout:3/888: dread d9/d12/fdc [0,4194304] 0 2024-04-22T23:46:20.605 INFO:tasks.workunit.client.1.smithi044.stdout:6/733: creat d5/dd/d28/fe7 x:0 0 0 2024-04-22T23:46:20.607 INFO:tasks.workunit.client.1.smithi044.stdout:1/840: rename d1/d8/d1c/d7b/d80/dfd/f89 to d1/d8/d20/d47/f12b 0 2024-04-22T23:46:20.610 INFO:tasks.workunit.client.1.smithi044.stdout:6/734: dread d5/d3f/d4c/d23/d3d/d64/f94 [0,4194304] 0 2024-04-22T23:46:20.616 INFO:tasks.workunit.client.1.smithi044.stdout:9/685: symlink d2/dc/d1d/d34/d38/d4a/le0 0 2024-04-22T23:46:20.618 INFO:tasks.workunit.client.1.smithi044.stdout:0/972: getdents d0/d2/dd/d3e/d40/d43/d79 0 2024-04-22T23:46:20.624 INFO:tasks.workunit.client.1.smithi044.stdout:7/851: link dd/d99/dbb/ff6 dd/d1f/f134 0 2024-04-22T23:46:20.625 INFO:tasks.workunit.client.1.smithi044.stdout:7/852: chown dd/d1f/d24/f70 3015 1 2024-04-22T23:46:20.625 INFO:tasks.workunit.client.1.smithi044.stdout:7/853: readlink dd/d1f/l5a 0 2024-04-22T23:46:20.626 INFO:tasks.workunit.client.1.smithi044.stdout:7/854: chown dd/d98 10 1 2024-04-22T23:46:20.627 INFO:tasks.workunit.client.1.smithi044.stdout:7/855: write dd/d1f/d24/fcc [1094978,47098] 0 2024-04-22T23:46:20.632 INFO:tasks.workunit.client.1.smithi044.stdout:4/869: sync 2024-04-22T23:46:20.635 INFO:tasks.workunit.client.1.smithi044.stdout:8/846: stat db/d14/d24 0 2024-04-22T23:46:20.636 INFO:tasks.workunit.client.1.smithi044.stdout:2/669: dwrite d8/d19/d1c/f33 [0,4194304] 0 2024-04-22T23:46:20.636 INFO:tasks.workunit.client.1.smithi044.stdout:6/735: read d5/d3f/d4c/d23/f9b [513711,120136] 0 2024-04-22T23:46:20.637 INFO:tasks.workunit.client.1.smithi044.stdout:1/841: dread d1/d8/d20/d47/f85 [0,4194304] 0 2024-04-22T23:46:20.639 INFO:tasks.workunit.client.1.smithi044.stdout:0/973: fsync d0/d2/dd/d3e/d40/d43/de4/d146/d2f/f4d 0 2024-04-22T23:46:20.644 INFO:tasks.workunit.client.1.smithi044.stdout:1/842: dread d1/d8/d20/d45/d75/df2/f94 [0,4194304] 0 2024-04-22T23:46:20.645 INFO:tasks.workunit.client.1.smithi044.stdout:1/843: stat d1/d8/d20/d45 0 2024-04-22T23:46:20.645 INFO:tasks.workunit.client.1.smithi044.stdout:7/856: mkdir dd/d99/d135 0 2024-04-22T23:46:20.646 INFO:tasks.workunit.client.1.smithi044.stdout:1/844: fdatasync d1/d8/d20/d47/d92/fa2 0 2024-04-22T23:46:20.651 INFO:tasks.workunit.client.1.smithi044.stdout:2/670: mkdir d8/d19/d36/d46/df0 0 2024-04-22T23:46:20.652 INFO:tasks.workunit.client.1.smithi044.stdout:2/671: chown d8/d19/d36/d46/d63/f71 33 1 2024-04-22T23:46:20.653 INFO:tasks.workunit.client.1.smithi044.stdout:9/686: mknod d2/dc/d1d/d34/d38/d4a/db7/ce1 0 2024-04-22T23:46:20.656 INFO:tasks.workunit.client.1.smithi044.stdout:8/847: truncate db/d52/d9c/dad/db8/fe8 4809027 0 2024-04-22T23:46:20.657 INFO:tasks.workunit.client.1.smithi044.stdout:5/894: link d15/d16/d1d/d22/d2a/d3a/f4b d15/d5d/d111/f131 0 2024-04-22T23:46:20.660 INFO:tasks.workunit.client.1.smithi044.stdout:3/889: truncate d9/db/d9f/fd7 419153 0 2024-04-22T23:46:20.663 INFO:tasks.workunit.client.1.smithi044.stdout:2/672: dwrite d8/d19/d36/d93/dbf/fd5 [0,4194304] 0 2024-04-22T23:46:20.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/687: dwrite d2/dc/d1d/d34/d38/d5c/d7b/fde [0,4194304] 0 2024-04-22T23:46:20.667 INFO:tasks.workunit.client.1.smithi044.stdout:7/857: mknod dd/d55/d5e/d60/d68/d94/dbd/dc9/d123/c136 0 2024-04-22T23:46:20.667 INFO:tasks.workunit.client.1.smithi044.stdout:6/736: write d5/d3f/d4c/d23/f9b [1213305,57043] 0 2024-04-22T23:46:20.669 INFO:tasks.workunit.client.1.smithi044.stdout:6/737: write d5/d3f/d4c/d23/d3d/d64/fda [4423078,33780] 0 2024-04-22T23:46:20.678 INFO:tasks.workunit.client.1.smithi044.stdout:3/890: write f8 [5694480,71676] 0 2024-04-22T23:46:20.683 INFO:tasks.workunit.client.1.smithi044.stdout:1/845: write d1/d8/d20/d45/d5d/f61 [391976,34696] 0 2024-04-22T23:46:20.684 INFO:tasks.workunit.client.1.smithi044.stdout:3/891: read d9/d14/d7d/df8/dfd/f58 [5700337,104405] 0 2024-04-22T23:46:20.684 INFO:tasks.workunit.client.1.smithi044.stdout:8/848: rename db/d68/d8b/da7/cd8 to db/d14/d24/d127/c129 0 2024-04-22T23:46:20.691 INFO:tasks.workunit.client.1.smithi044.stdout:1/846: dread d1/d8/d34/da1/fbd [0,4194304] 0 2024-04-22T23:46:20.703 INFO:tasks.workunit.client.1.smithi044.stdout:2/673: rename d8/d60/l8a to d8/d19/d36/d93/lf1 0 2024-04-22T23:46:20.705 INFO:tasks.workunit.client.1.smithi044.stdout:1/847: dread d1/d8/d1c/d7b/fd9 [0,4194304] 0 2024-04-22T23:46:20.709 INFO:tasks.workunit.client.1.smithi044.stdout:8/849: mknod db/d68/d8b/dca/dd6/c12a 0 2024-04-22T23:46:20.710 INFO:tasks.workunit.client.1.smithi044.stdout:3/892: creat d9/db/d15/d25/d67/d6b/ded/d113/dbb/d10c/f129 x:0 0 0 2024-04-22T23:46:20.710 INFO:tasks.workunit.client.1.smithi044.stdout:4/870: link d5/d31/f1b d5/db/d34/f126 0 2024-04-22T23:46:20.710 INFO:tasks.workunit.client.1.smithi044.stdout:4/871: chown d5/d31/d44/d20/d22/d8d/fd6 106 1 2024-04-22T23:46:20.712 INFO:tasks.workunit.client.1.smithi044.stdout:9/688: mknod d2/dc/d1d/d34/ce2 0 2024-04-22T23:46:20.713 INFO:tasks.workunit.client.1.smithi044.stdout:7/858: mkdir dd/d98/de8/d12e/d137 0 2024-04-22T23:46:20.714 INFO:tasks.workunit.client.1.smithi044.stdout:9/689: dread d2/d6/fa1 [0,4194304] 0 2024-04-22T23:46:20.717 INFO:tasks.workunit.client.1.smithi044.stdout:5/895: dwrite d15/d16/d1d/d22/d4f/dc9/d8d/f98 [0,4194304] 0 2024-04-22T23:46:20.719 INFO:tasks.workunit.client.1.smithi044.stdout:2/674: mknod d8/d19/d5d/db0/cf2 0 2024-04-22T23:46:20.721 INFO:tasks.workunit.client.1.smithi044.stdout:1/848: mkdir d1/d8/d1c/d7b/d80/d4b/d12c 0 2024-04-22T23:46:20.722 INFO:tasks.workunit.client.1.smithi044.stdout:1/849: write d1/d8/fa0 [561167,87538] 0 2024-04-22T23:46:20.726 INFO:tasks.workunit.client.1.smithi044.stdout:7/859: rename dd/d1a/d122 to dd/d98/d138 0 2024-04-22T23:46:20.726 INFO:tasks.workunit.client.1.smithi044.stdout:7/860: readlink dd/l43 0 2024-04-22T23:46:20.729 INFO:tasks.workunit.client.1.smithi044.stdout:9/690: mkdir d2/dc/d1d/d3f/de3 0 2024-04-22T23:46:20.729 INFO:tasks.workunit.client.1.smithi044.stdout:6/738: link d5/dd/c37 d5/d3f/d4c/d92/ce8 0 2024-04-22T23:46:20.731 INFO:tasks.workunit.client.1.smithi044.stdout:6/739: write d5/dd/fa8 [3844408,10259] 0 2024-04-22T23:46:20.732 INFO:tasks.workunit.client.1.smithi044.stdout:7/861: dread dd/d55/d5e/d60/d7b/f7f [0,4194304] 0 2024-04-22T23:46:20.734 INFO:tasks.workunit.client.1.smithi044.stdout:2/675: creat d8/d19/d96/ff3 x:0 0 0 2024-04-22T23:46:20.735 INFO:tasks.workunit.client.1.smithi044.stdout:2/676: dread - d8/fd9 zero size 2024-04-22T23:46:20.737 INFO:tasks.workunit.client.1.smithi044.stdout:8/850: fsync db/d14/d72/f8e 0 2024-04-22T23:46:20.745 INFO:tasks.workunit.client.1.smithi044.stdout:6/740: symlink d5/d3f/d4c/d40/d53/d56/db7/le9 0 2024-04-22T23:46:20.746 INFO:tasks.workunit.client.1.smithi044.stdout:6/741: stat d5/d3f/d4c/d40/d53/d56/c7c 0 2024-04-22T23:46:20.747 INFO:tasks.workunit.client.1.smithi044.stdout:7/862: symlink dd/d55/d5e/d60/d106/l139 0 2024-04-22T23:46:20.753 INFO:tasks.workunit.client.1.smithi044.stdout:8/851: fdatasync db/d6c/fcc 0 2024-04-22T23:46:20.757 INFO:tasks.workunit.client.1.smithi044.stdout:1/850: dwrite d1/d8/d1c/d7b/d80/dfd/dbf/ddb/fef [0,4194304] 0 2024-04-22T23:46:20.759 INFO:tasks.workunit.client.1.smithi044.stdout:9/691: creat d2/d6/d42/d8b/ddc/fe4 x:0 0 0 2024-04-22T23:46:20.761 INFO:tasks.workunit.client.1.smithi044.stdout:5/896: creat d15/d5d/d64/f132 x:0 0 0 2024-04-22T23:46:20.768 INFO:tasks.workunit.client.1.smithi044.stdout:3/893: getdents d9/db/d9f/d38/d74 0 2024-04-22T23:46:20.769 INFO:tasks.workunit.client.1.smithi044.stdout:4/872: getdents d5/db/d16/d4d 0 2024-04-22T23:46:20.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/851: symlink d1/d8/d20/d45/d75/dfb/l12d 0 2024-04-22T23:46:20.773 INFO:tasks.workunit.client.1.smithi044.stdout:5/897: mkdir d15/d3c/df9/d133 0 2024-04-22T23:46:20.775 INFO:tasks.workunit.client.1.smithi044.stdout:2/677: link d8/caa d8/d60/de1/cf4 0 2024-04-22T23:46:20.778 INFO:tasks.workunit.client.1.smithi044.stdout:3/894: symlink d9/db/d15/d25/l12a 0 2024-04-22T23:46:20.781 INFO:tasks.workunit.client.1.smithi044.stdout:6/742: dwrite d5/d3f/d4c/d23/f29 [0,4194304] 0 2024-04-22T23:46:20.790 INFO:tasks.workunit.client.1.smithi044.stdout:4/873: rmdir d5 39 2024-04-22T23:46:20.793 INFO:tasks.workunit.client.1.smithi044.stdout:9/692: unlink d2/dc/d1d/d3f/d9d/cd0 0 2024-04-22T23:46:20.794 INFO:tasks.workunit.client.1.smithi044.stdout:3/895: dread d9/db/d15/d25/f6d [0,4194304] 0 2024-04-22T23:46:20.796 INFO:tasks.workunit.client.1.smithi044.stdout:1/852: creat d1/d8/d20/d106/f12e x:0 0 0 2024-04-22T23:46:20.800 INFO:tasks.workunit.client.1.smithi044.stdout:2/678: mkdir d8/d19/d36/d46/ddb/ddd/df5 0 2024-04-22T23:46:20.801 INFO:tasks.workunit.client.1.smithi044.stdout:8/852: creat db/d68/f12b x:0 0 0 2024-04-22T23:46:20.808 INFO:tasks.workunit.client.1.smithi044.stdout:9/693: creat d2/dc/d1d/d3f/d9d/fe5 x:0 0 0 2024-04-22T23:46:20.808 INFO:tasks.workunit.client.1.smithi044.stdout:9/694: chown d2/dc/d1d/d34/d8e/dbd/dd1 964611858 1 2024-04-22T23:46:20.811 INFO:tasks.workunit.client.1.smithi044.stdout:5/898: mkdir d15/d16/d1d/d22/d4f/d68/d134 0 2024-04-22T23:46:20.814 INFO:tasks.workunit.client.1.smithi044.stdout:1/853: unlink d1/d8/d34/da1/dab/de9/c100 0 2024-04-22T23:46:20.815 INFO:tasks.workunit.client.1.smithi044.stdout:7/863: getdents dd/d55 0 2024-04-22T23:46:20.815 INFO:tasks.workunit.client.1.smithi044.stdout:1/854: chown d1/d8/d20/d45/d75/f84 239950 1 2024-04-22T23:46:20.818 INFO:tasks.workunit.client.1.smithi044.stdout:2/679: creat d8/d27/d5b/d74/ff6 x:0 0 0 2024-04-22T23:46:20.819 INFO:tasks.workunit.client.1.smithi044.stdout:2/680: stat d8/d60/dc1/fa6 0 2024-04-22T23:46:20.819 INFO:tasks.workunit.client.1.smithi044.stdout:2/681: readlink d8/d19/lc3 0 2024-04-22T23:46:20.820 INFO:tasks.workunit.client.1.smithi044.stdout:2/682: chown d8/d19/d96/fad 1 1 2024-04-22T23:46:20.824 INFO:tasks.workunit.client.1.smithi044.stdout:8/853: dwrite db/ddd/df2/fe1 [4194304,4194304] 0 2024-04-22T23:46:20.825 INFO:tasks.workunit.client.1.smithi044.stdout:6/743: write d5/d3f/d4c/d23/d3d/d64/f94 [106814,61846] 0 2024-04-22T23:46:20.826 INFO:tasks.workunit.client.1.smithi044.stdout:6/744: fdatasync d5/dd/d67/f5d 0 2024-04-22T23:46:20.834 INFO:tasks.workunit.client.1.smithi044.stdout:2/683: dread d8/d27/f56 [0,4194304] 0 2024-04-22T23:46:20.834 INFO:tasks.workunit.client.1.smithi044.stdout:9/695: symlink d2/dc/d1d/d34/d8e/dbd/dd1/le6 0 2024-04-22T23:46:20.836 INFO:tasks.workunit.client.1.smithi044.stdout:9/696: fsync d2/dc/d1d/d3f/fb4 0 2024-04-22T23:46:20.837 INFO:tasks.workunit.client.1.smithi044.stdout:9/697: chown d2/d6/c52 5898126 1 2024-04-22T23:46:20.842 INFO:tasks.workunit.client.1.smithi044.stdout:6/745: creat d5/d3f/d4c/d92/fea x:0 0 0 2024-04-22T23:46:20.844 INFO:tasks.workunit.client.1.smithi044.stdout:7/864: creat dd/df7/db8/d108/d11f/f13a x:0 0 0 2024-04-22T23:46:20.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/698: chown d2/d6/d42/ccd 25 1 2024-04-22T23:46:20.855 INFO:tasks.workunit.client.1.smithi044.stdout:6/746: symlink d5/d3f/d4c/d4f/leb 0 2024-04-22T23:46:20.858 INFO:tasks.workunit.client.1.smithi044.stdout:1/855: link d1/d8/d1c/d23/c82 d1/d8/d20/d45/d5d/c12f 0 2024-04-22T23:46:20.859 INFO:tasks.workunit.client.1.smithi044.stdout:3/896: write d9/db/f71 [158904,24270] 0 2024-04-22T23:46:20.863 INFO:tasks.workunit.client.1.smithi044.stdout:5/899: getdents d15/d16/d1d/d22/d2a/d3a/d43 0 2024-04-22T23:46:20.865 INFO:tasks.workunit.client.1.smithi044.stdout:6/747: mknod d5/d3f/d4c/d23/cec 0 2024-04-22T23:46:20.869 INFO:tasks.workunit.client.1.smithi044.stdout:3/897: dwrite d9/db/d33/d5d/fe3 [0,4194304] 0 2024-04-22T23:46:20.871 INFO:tasks.workunit.client.1.smithi044.stdout:7/865: mknod dd/d55/d5e/d60/d68/d94/dbd/c13b 0 2024-04-22T23:46:20.873 INFO:tasks.workunit.client.1.smithi044.stdout:4/874: write d5/db/d3c/fdb [854955,77104] 0 2024-04-22T23:46:20.878 INFO:tasks.workunit.client.1.smithi044.stdout:4/875: read d5/d31/d44/d20/d22/f62 [2070854,127998] 0 2024-04-22T23:46:20.878 INFO:tasks.workunit.client.1.smithi044.stdout:1/856: rmdir d1/d8/d1c/d23/d3e 39 2024-04-22T23:46:20.881 INFO:tasks.workunit.client.1.smithi044.stdout:9/699: mknod d2/dc/ce7 0 2024-04-22T23:46:20.882 INFO:tasks.workunit.client.1.smithi044.stdout:9/700: stat d2/dc/d94/dd3 0 2024-04-22T23:46:20.898 INFO:tasks.workunit.client.1.smithi044.stdout:9/701: mknod d2/dc/d1d/d34/d38/d5c/d69/ce8 0 2024-04-22T23:46:20.898 INFO:tasks.workunit.client.1.smithi044.stdout:7/866: dread dd/d1a/db7/dc8/f49 [0,4194304] 0 2024-04-22T23:46:20.899 INFO:tasks.workunit.client.1.smithi044.stdout:7/867: chown dd/d99/fc2 10950819 1 2024-04-22T23:46:20.901 INFO:tasks.workunit.client.1.smithi044.stdout:1/857: dread d1/d8/d1c/d7b/d80/d4b/f9a [0,4194304] 0 2024-04-22T23:46:20.902 INFO:tasks.workunit.client.1.smithi044.stdout:3/898: symlink d9/l12b 0 2024-04-22T23:46:20.902 INFO:tasks.workunit.client.1.smithi044.stdout:1/858: readlink d1/d8/d1c/d74/l7a 0 2024-04-22T23:46:20.906 INFO:tasks.workunit.client.1.smithi044.stdout:5/900: creat d15/d16/d1d/d22/d2a/d3a/f135 x:0 0 0 2024-04-22T23:46:20.907 INFO:tasks.workunit.client.1.smithi044.stdout:5/901: stat d15/d16/d1d/d22/d4f/dc9/d8d/f8f 0 2024-04-22T23:46:20.908 INFO:tasks.workunit.client.1.smithi044.stdout:2/684: write d8/fe [1143566,54983] 0 2024-04-22T23:46:20.908 INFO:tasks.workunit.client.1.smithi044.stdout:7/868: write dd/d55/f58 [1024316,89534] 0 2024-04-22T23:46:20.912 INFO:tasks.workunit.client.1.smithi044.stdout:4/876: creat d5/d52/f127 x:0 0 0 2024-04-22T23:46:20.913 INFO:tasks.workunit.client.1.smithi044.stdout:9/702: creat d2/dc/d1d/d34/d38/d4a/dca/fe9 x:0 0 0 2024-04-22T23:46:20.914 INFO:tasks.workunit.client.1.smithi044.stdout:7/869: fsync dd/fe3 0 2024-04-22T23:46:20.916 INFO:tasks.workunit.client.1.smithi044.stdout:0/974: sync 2024-04-22T23:46:20.916 INFO:tasks.workunit.client.1.smithi044.stdout:8/854: dwrite db/f4f [0,4194304] 0 2024-04-22T23:46:20.919 INFO:tasks.workunit.client.1.smithi044.stdout:7/870: read dd/d1a/f90 [1289878,37565] 0 2024-04-22T23:46:20.920 INFO:tasks.workunit.client.1.smithi044.stdout:4/877: symlink d5/d31/d44/dd2/l128 0 2024-04-22T23:46:20.920 INFO:tasks.workunit.client.1.smithi044.stdout:8/855: dread - db/d68/d8b/da7/fae zero size 2024-04-22T23:46:20.926 INFO:tasks.workunit.client.1.smithi044.stdout:0/975: dwrite d0/d2/dd/d3e/f3f [4194304,4194304] 0 2024-04-22T23:46:20.927 INFO:tasks.workunit.client.1.smithi044.stdout:3/899: rename d9/db/d9f/f10b to d9/f12c 0 2024-04-22T23:46:20.937 INFO:tasks.workunit.client.1.smithi044.stdout:6/748: dwrite d5/d3f/f22 [0,4194304] 0 2024-04-22T23:46:20.938 INFO:tasks.workunit.client.1.smithi044.stdout:1/859: write d1/f7 [4718516,83862] 0 2024-04-22T23:46:20.942 INFO:tasks.workunit.client.1.smithi044.stdout:9/703: mknod d2/dc/d1d/d34/d38/d4a/da6/cea 0 2024-04-22T23:46:20.944 INFO:tasks.workunit.client.1.smithi044.stdout:2/685: write d8/d27/f28 [478335,57217] 0 2024-04-22T23:46:20.948 INFO:tasks.workunit.client.1.smithi044.stdout:5/902: rename d15/d5d/db5/dc0/f5a to d15/d5d/db5/dc0/d38/f136 0 2024-04-22T23:46:20.950 INFO:tasks.workunit.client.1.smithi044.stdout:3/900: chown d9/d12/db1/df9/c114 2501 1 2024-04-22T23:46:20.950 INFO:tasks.workunit.client.1.smithi044.stdout:5/903: write d15/d5d/d82/f130 [650698,16784] 0 2024-04-22T23:46:20.955 INFO:tasks.workunit.client.1.smithi044.stdout:1/860: fdatasync d1/d8/d1c/d7b/d80/d4b/f9a 0 2024-04-22T23:46:20.956 INFO:tasks.workunit.client.1.smithi044.stdout:2/686: truncate d8/d27/d5b/d74/d80/fb2 737583 0 2024-04-22T23:46:20.961 INFO:tasks.workunit.client.1.smithi044.stdout:8/856: creat db/f12c x:0 0 0 2024-04-22T23:46:20.965 INFO:tasks.workunit.client.1.smithi044.stdout:0/976: dread d0/d4c/d59/fb2 [0,4194304] 0 2024-04-22T23:46:20.966 INFO:tasks.workunit.client.1.smithi044.stdout:6/749: dread d5/d3f/d4c/d40/f71 [0,4194304] 0 2024-04-22T23:46:20.968 INFO:tasks.workunit.client.1.smithi044.stdout:7/871: getdents dd 0 2024-04-22T23:46:20.969 INFO:tasks.workunit.client.1.smithi044.stdout:5/904: symlink d15/d16/d1d/d22/d2a/d3a/d43/d11d/l137 0 2024-04-22T23:46:20.971 INFO:tasks.workunit.client.1.smithi044.stdout:8/857: creat db/d68/d8b/ded/f12d x:0 0 0 2024-04-22T23:46:20.971 INFO:tasks.workunit.client.1.smithi044.stdout:4/878: link d5/d31/d24/d38/d84/ce7 d5/db/d16/c129 0 2024-04-22T23:46:20.971 INFO:tasks.workunit.client.1.smithi044.stdout:6/750: dread - d5/dd/fd2 zero size 2024-04-22T23:46:20.973 INFO:tasks.workunit.client.1.smithi044.stdout:9/704: rename d2/dc/d1d/d34/d38/d4a/dc7 to d2/dc/d1d/d34/d38/d5c/d69/d7a/deb 0 2024-04-22T23:46:20.974 INFO:tasks.workunit.client.1.smithi044.stdout:7/872: chown dd/d1a/d61/l101 3661593 1 2024-04-22T23:46:20.974 INFO:tasks.workunit.client.1.smithi044.stdout:3/901: dwrite d9/d12/fdc [0,4194304] 0 2024-04-22T23:46:20.978 INFO:tasks.workunit.client.1.smithi044.stdout:7/873: write dd/d1f/f66 [3816340,129048] 0 2024-04-22T23:46:20.978 INFO:tasks.workunit.client.1.smithi044.stdout:6/751: chown d5/dd/f6a 1313 1 2024-04-22T23:46:20.978 INFO:tasks.workunit.client.1.smithi044.stdout:0/977: creat d0/d2/dd/f15b x:0 0 0 2024-04-22T23:46:20.978 INFO:tasks.workunit.client.1.smithi044.stdout:4/879: chown d5/d31/d44/d20/d22/d8d 5888 1 2024-04-22T23:46:20.983 INFO:tasks.workunit.client.1.smithi044.stdout:2/687: dread f7 [0,4194304] 0 2024-04-22T23:46:20.985 INFO:tasks.workunit.client.1.smithi044.stdout:7/874: stat dd/d55/d5e/d60/d68/d94/dab 0 2024-04-22T23:46:20.986 INFO:tasks.workunit.client.1.smithi044.stdout:1/861: dwrite d1/d8/d20/d47/f12b [0,4194304] 0 2024-04-22T23:46:20.990 INFO:tasks.workunit.client.1.smithi044.stdout:7/875: chown dd/df7/db8/d108/l10d 7068 1 2024-04-22T23:46:20.991 INFO:tasks.workunit.client.1.smithi044.stdout:1/862: chown d1/d8/d20/d45/d5d/l91 51365196 1 2024-04-22T23:46:20.993 INFO:tasks.workunit.client.1.smithi044.stdout:1/863: chown d1/d8/d20/d45/d75/df2/l107 129098958 1 2024-04-22T23:46:20.995 INFO:tasks.workunit.client.1.smithi044.stdout:9/705: dread d2/dc/d1d/d34/d38/d5c/d69/fb8 [0,4194304] 0 2024-04-22T23:46:20.996 INFO:tasks.workunit.client.1.smithi044.stdout:9/706: chown d2/dc/d1d/d34/d8e 23200 1 2024-04-22T23:46:20.997 INFO:tasks.workunit.client.1.smithi044.stdout:1/864: read d1/d8/d20/fea [883296,119821] 0 2024-04-22T23:46:20.997 INFO:tasks.workunit.client.1.smithi044.stdout:3/902: rmdir d9/db/d15/d25/d67/d6b 39 2024-04-22T23:46:21.000 INFO:tasks.workunit.client.1.smithi044.stdout:3/903: write d9/db/d33/d5d/d64/dbe/f106 [458910,1160] 0 2024-04-22T23:46:21.004 INFO:tasks.workunit.client.1.smithi044.stdout:9/707: dwrite d2/dc/d1d/d3f/fd9 [0,4194304] 0 2024-04-22T23:46:21.005 INFO:tasks.workunit.client.1.smithi044.stdout:9/708: chown d2/dc/d1d/d34/d38/d4a/d72/f8f 543 1 2024-04-22T23:46:21.006 INFO:tasks.workunit.client.1.smithi044.stdout:0/978: rename d0/d4c/d8b to d0/d2/dd/d3e/d40/d43/d79/dbb/d15c 0 2024-04-22T23:46:21.008 INFO:tasks.workunit.client.1.smithi044.stdout:0/979: fdatasync d0/d2/dd/d3e/d40/d43/d7b/f153 0 2024-04-22T23:46:21.017 INFO:tasks.workunit.client.1.smithi044.stdout:4/880: truncate d5/d31/d44/d20/d22/d8d/fa8 989984 0 2024-04-22T23:46:21.018 INFO:tasks.workunit.client.1.smithi044.stdout:5/905: write d15/d16/f1a [600112,11600] 0 2024-04-22T23:46:21.018 INFO:tasks.workunit.client.1.smithi044.stdout:0/980: read - d0/d2/dd/d3e/d40/d43/de4/d146/dbc/f13f zero size 2024-04-22T23:46:21.018 INFO:tasks.workunit.client.1.smithi044.stdout:2/688: dread d8/d19/d36/d44/f97 [0,4194304] 0 2024-04-22T23:46:21.019 INFO:tasks.workunit.client.1.smithi044.stdout:2/689: fdatasync f6 0 2024-04-22T23:46:21.023 INFO:tasks.workunit.client.1.smithi044.stdout:1/865: dread d1/d8/d1c/f59 [0,4194304] 0 2024-04-22T23:46:21.026 INFO:tasks.workunit.client.1.smithi044.stdout:8/858: dwrite db/f48 [0,4194304] 0 2024-04-22T23:46:21.033 INFO:tasks.workunit.client.1.smithi044.stdout:6/752: creat d5/d3f/d4c/d92/d97/de5/fed x:0 0 0 2024-04-22T23:46:21.035 INFO:tasks.workunit.client.1.smithi044.stdout:6/753: truncate d5/dd/d28/fe7 839306 0 2024-04-22T23:46:21.036 INFO:tasks.workunit.client.1.smithi044.stdout:9/709: rmdir d2/dc/d1d/d34/d8e/dbd/dd1 39 2024-04-22T23:46:21.036 INFO:tasks.workunit.client.1.smithi044.stdout:6/754: dread - d5/dd/fc5 zero size 2024-04-22T23:46:21.036 INFO:tasks.workunit.client.1.smithi044.stdout:7/876: dread dd/d99/f10e [0,4194304] 0 2024-04-22T23:46:21.037 INFO:tasks.workunit.client.1.smithi044.stdout:7/877: fsync dd/d27/f46 0 2024-04-22T23:46:21.078 INFO:tasks.workunit.client.1.smithi044.stdout:0/981: mkdir d0/d2/dd/d3e/d40/d43/dde/d15d 0 2024-04-22T23:46:21.087 INFO:tasks.workunit.client.1.smithi044.stdout:1/866: dwrite d1/d8/d1c/d23/d3e/dde/f104 [0,4194304] 0 2024-04-22T23:46:21.094 INFO:tasks.workunit.client.1.smithi044.stdout:9/710: creat d2/dc/d1d/d34/d38/d5c/d69/d7a/deb/fec x:0 0 0 2024-04-22T23:46:21.095 INFO:tasks.workunit.client.1.smithi044.stdout:7/878: mkdir dd/d1a/db7/dc8/d97/d13c 0 2024-04-22T23:46:21.098 INFO:tasks.workunit.client.1.smithi044.stdout:6/755: rename d5/d3f/d4c/d23/d3d/d64/fda to d5/dd/d28/d75/fee 0 2024-04-22T23:46:21.099 INFO:tasks.workunit.client.1.smithi044.stdout:6/756: readlink d5/d3f/d4c/d40/d74/lc8 0 2024-04-22T23:46:21.107 INFO:tasks.workunit.client.1.smithi044.stdout:0/982: unlink d0/d2/d4b/d6b/l9c 0 2024-04-22T23:46:21.107 INFO:tasks.workunit.client.1.smithi044.stdout:6/757: dwrite d5/d3f/d50/d79/fae [0,4194304] 0 2024-04-22T23:46:21.107 INFO:tasks.workunit.client.1.smithi044.stdout:4/881: dwrite d5/db/d34/f59 [0,4194304] 0 2024-04-22T23:46:21.113 INFO:tasks.workunit.client.1.smithi044.stdout:0/983: stat d0/d2/dd/d3e/d40/d43 0 2024-04-22T23:46:21.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/859: write db/f38 [1358827,57088] 0 2024-04-22T23:46:21.117 INFO:tasks.workunit.client.1.smithi044.stdout:1/867: chown d1/d8/d1c/d7b/d80/dfd/dbf/dc2/lc3 30 1 2024-04-22T23:46:21.117 INFO:tasks.workunit.client.1.smithi044.stdout:5/906: dwrite d15/d5d/db5/dc0/d38/f50 [0,4194304] 0 2024-04-22T23:46:21.123 INFO:tasks.workunit.client.1.smithi044.stdout:7/879: fsync dd/d27/fb6 0 2024-04-22T23:46:21.128 INFO:tasks.workunit.client.1.smithi044.stdout:4/882: rmdir d5/d31/d24 39 2024-04-22T23:46:21.132 INFO:tasks.workunit.client.1.smithi044.stdout:6/758: rename d5/d3f/d4c/d40/d74/daf/cba to d5/dd/d28/d75/d88/cef 0 2024-04-22T23:46:21.134 INFO:tasks.workunit.client.1.smithi044.stdout:6/759: write d5/d3f/d50/fb5 [5067456,102199] 0 2024-04-22T23:46:21.136 INFO:tasks.workunit.client.1.smithi044.stdout:2/690: dwrite d8/d19/d36/d44/f70 [0,4194304] 0 2024-04-22T23:46:21.136 INFO:tasks.workunit.client.1.smithi044.stdout:3/904: getdents d9/db/d15/d25/d67 0 2024-04-22T23:46:21.144 INFO:tasks.workunit.client.1.smithi044.stdout:6/760: dread d5/d3f/d4c/d40/d53/f8e [0,4194304] 0 2024-04-22T23:46:21.146 INFO:tasks.workunit.client.1.smithi044.stdout:6/761: stat d5/d3f/d50/d79/d96/fa1 0 2024-04-22T23:46:21.146 INFO:tasks.workunit.client.1.smithi044.stdout:6/762: chown d5 230846 1 2024-04-22T23:46:21.147 INFO:tasks.workunit.client.1.smithi044.stdout:7/880: mkdir dd/d99/d13d 0 2024-04-22T23:46:21.147 INFO:tasks.workunit.client.1.smithi044.stdout:9/711: truncate d2/dc/d94/f50 2495378 0 2024-04-22T23:46:21.149 INFO:tasks.workunit.client.1.smithi044.stdout:5/907: write d15/d16/dbc/d116/fdb [100959,103767] 0 2024-04-22T23:46:21.153 INFO:tasks.workunit.client.1.smithi044.stdout:0/984: dwrite d0/d2/d4b/d6b/fc2 [0,4194304] 0 2024-04-22T23:46:21.153 INFO:tasks.workunit.client.1.smithi044.stdout:1/868: rename d1/d8/d20/ccd to d1/d8/d20/dae/deb/d124/c130 0 2024-04-22T23:46:21.163 INFO:tasks.workunit.client.1.smithi044.stdout:0/985: dwrite d0/d60/de6/d109/f113 [0,4194304] 0 2024-04-22T23:46:21.164 INFO:tasks.workunit.client.1.smithi044.stdout:8/860: fsync db/d14/d24/f58 0 2024-04-22T23:46:21.166 INFO:tasks.workunit.client.1.smithi044.stdout:6/763: mkdir d5/dd/d28/d75/df0 0 2024-04-22T23:46:21.167 INFO:tasks.workunit.client.1.smithi044.stdout:6/764: dread - d5/d3f/d4c/d40/d53/d56/db7/fe2 zero size 2024-04-22T23:46:21.167 INFO:tasks.workunit.client.1.smithi044.stdout:7/881: symlink dd/l13e 0 2024-04-22T23:46:21.170 INFO:tasks.workunit.client.1.smithi044.stdout:4/883: link d5/d31/d44/d20/d22/fd7 d5/db/d16/d4d/dad/dc5/f12a 0 2024-04-22T23:46:21.172 INFO:tasks.workunit.client.1.smithi044.stdout:4/884: chown d5/d31/d44/d20/d22/c66 12 1 2024-04-22T23:46:21.173 INFO:tasks.workunit.client.1.smithi044.stdout:3/905: write d9/fb9 [302309,5878] 0 2024-04-22T23:46:21.177 INFO:tasks.workunit.client.1.smithi044.stdout:9/712: creat d2/dc/d1d/d34/d38/d5c/d69/d75/d78/fed x:0 0 0 2024-04-22T23:46:21.179 INFO:tasks.workunit.client.1.smithi044.stdout:9/713: chown d2/dc/d1d/d3f/fb4 859717 1 2024-04-22T23:46:21.186 INFO:tasks.workunit.client.1.smithi044.stdout:5/908: write d15/d5d/db5/dc0/fd0 [66752,12531] 0 2024-04-22T23:46:21.187 INFO:tasks.workunit.client.1.smithi044.stdout:6/765: dread d5/d3f/d50/d79/f8c [0,4194304] 0 2024-04-22T23:46:21.188 INFO:tasks.workunit.client.1.smithi044.stdout:6/766: fdatasync d5/d3f/f22 0 2024-04-22T23:46:21.189 INFO:tasks.workunit.client.1.smithi044.stdout:0/986: symlink d0/d2/dd/d3e/d40/d43/d79/d12a/l15e 0 2024-04-22T23:46:21.193 INFO:tasks.workunit.client.1.smithi044.stdout:4/885: mknod d5/d31/d44/d20/d22/c12b 0 2024-04-22T23:46:21.195 INFO:tasks.workunit.client.1.smithi044.stdout:9/714: truncate d2/dc/d1d/d3f/f63 540054 0 2024-04-22T23:46:21.204 INFO:tasks.workunit.client.1.smithi044.stdout:1/869: rmdir d1/d8/d20/d45/d75/df2/d99 39 2024-04-22T23:46:21.207 INFO:tasks.workunit.client.1.smithi044.stdout:9/715: rmdir d2/dc/d1d/d34 39 2024-04-22T23:46:21.210 INFO:tasks.workunit.client.1.smithi044.stdout:6/767: mkdir d5/d3f/d4c/df1 0 2024-04-22T23:46:21.211 INFO:tasks.workunit.client.1.smithi044.stdout:0/987: mknod d0/d2/c15f 0 2024-04-22T23:46:21.211 INFO:tasks.workunit.client.1.smithi044.stdout:6/768: chown d5/d3f/d4c/d23/d3d/d64/c85 1092 1 2024-04-22T23:46:21.212 INFO:tasks.workunit.client.1.smithi044.stdout:6/769: chown d5/d3f/d4c/d40/db9 818017 1 2024-04-22T23:46:21.212 INFO:tasks.workunit.client.1.smithi044.stdout:0/988: write d0/d2/dd/d3e/d40/d43/f120 [768731,61124] 0 2024-04-22T23:46:21.214 INFO:tasks.workunit.client.1.smithi044.stdout:6/770: write d5/d3f/d50/f5a [2747744,69840] 0 2024-04-22T23:46:21.215 INFO:tasks.workunit.client.1.smithi044.stdout:0/989: write d0/d2/d4b/dad/f117 [2898872,4775] 0 2024-04-22T23:46:21.217 INFO:tasks.workunit.client.1.smithi044.stdout:3/906: rename d9/l12b to d9/db/d15/d25/d67/d6b/l12d 0 2024-04-22T23:46:21.217 INFO:tasks.workunit.client.1.smithi044.stdout:1/870: creat d1/d8/d1c/d7b/d80/dfd/dbf/dc2/f131 x:0 0 0 2024-04-22T23:46:21.218 INFO:tasks.workunit.client.1.smithi044.stdout:0/990: fdatasync d0/d2/f1e 0 2024-04-22T23:46:21.220 INFO:tasks.workunit.client.1.smithi044.stdout:8/861: getdents db/d14/d24 0 2024-04-22T23:46:21.221 INFO:tasks.workunit.client.1.smithi044.stdout:7/882: mkdir dd/d55/dcb/d13f 0 2024-04-22T23:46:21.222 INFO:tasks.workunit.client.1.smithi044.stdout:7/883: chown dd/d55/d5e/d60/d75/ldf 51421894 1 2024-04-22T23:46:21.228 INFO:tasks.workunit.client.1.smithi044.stdout:4/886: rename d5/d31/d24/d69/f11c to d5/d31/d24/d69/d90/d116/f12c 0 2024-04-22T23:46:21.236 INFO:tasks.workunit.client.1.smithi044.stdout:8/862: mknod db/d68/d8b/dca/c12e 0 2024-04-22T23:46:21.236 INFO:tasks.workunit.client.1.smithi044.stdout:8/863: readlink db/d52/l5f 0 2024-04-22T23:46:21.238 INFO:tasks.workunit.client.1.smithi044.stdout:7/884: write dd/d1a/db7/dc8/f8f [1847883,29412] 0 2024-04-22T23:46:21.238 INFO:tasks.workunit.client.1.smithi044.stdout:1/871: dread d1/d8/d20/d45/d75/dfb/f52 [0,4194304] 0 2024-04-22T23:46:21.240 INFO:tasks.workunit.client.1.smithi044.stdout:6/771: mknod d5/d3f/cf2 0 2024-04-22T23:46:21.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/772: truncate d5/dd/d67/fd3 294883 0 2024-04-22T23:46:21.243 INFO:tasks.workunit.client.1.smithi044.stdout:0/991: rename d0/d2/dd/d3e/d40/d43/d79/d12a/c13b to d0/dd6/df4/c160 0 2024-04-22T23:46:21.244 INFO:tasks.workunit.client.1.smithi044.stdout:4/887: symlink d5/d31/d24/l12d 0 2024-04-22T23:46:21.246 INFO:tasks.workunit.client.1.smithi044.stdout:3/907: symlink d9/d14/d7d/df8/dfd/d3c/dd1/l12e 0 2024-04-22T23:46:21.247 INFO:tasks.workunit.client.1.smithi044.stdout:1/872: dwrite d1/d8/f111 [0,4194304] 0 2024-04-22T23:46:21.249 INFO:tasks.workunit.client.1.smithi044.stdout:1/873: read d1/d8/f1e [2340487,51344] 0 2024-04-22T23:46:21.255 INFO:tasks.workunit.client.1.smithi044.stdout:7/885: rename dd/d55/d5e/d60/d68/d94/fbe to dd/d55/dcf/f140 0 2024-04-22T23:46:21.258 INFO:tasks.workunit.client.1.smithi044.stdout:6/773: dwrite d5/d3f/d4c/d23/d3d/f4a [0,4194304] 0 2024-04-22T23:46:21.260 INFO:tasks.workunit.client.1.smithi044.stdout:9/716: creat d2/dc/d1d/d34/d8e/dbd/fee x:0 0 0 2024-04-22T23:46:21.262 INFO:tasks.workunit.client.1.smithi044.stdout:9/717: chown d2/dc/d1d/d34/d38/c46 13 1 2024-04-22T23:46:21.263 INFO:tasks.workunit.client.1.smithi044.stdout:0/992: creat d0/d2/dd/d3e/d40/d43/d7b/f161 x:0 0 0 2024-04-22T23:46:21.266 INFO:tasks.workunit.client.1.smithi044.stdout:8/864: chown db/d14/d84/dbb/cbf 987 1 2024-04-22T23:46:21.270 INFO:tasks.workunit.client.1.smithi044.stdout:1/874: mkdir d1/d8/d1c/d74/d132 0 2024-04-22T23:46:21.272 INFO:tasks.workunit.client.1.smithi044.stdout:7/886: write dd/d1f/d24/fcc [1911407,119356] 0 2024-04-22T23:46:21.273 INFO:tasks.workunit.client.1.smithi044.stdout:4/888: dread - d5/d31/d24/d38/d6a/d75/fdf zero size 2024-04-22T23:46:21.275 INFO:tasks.workunit.client.1.smithi044.stdout:8/865: dread db/d14/d24/f51 [0,4194304] 0 2024-04-22T23:46:21.280 INFO:tasks.workunit.client.1.smithi044.stdout:6/774: dread d5/dd/d28/f2c [0,4194304] 0 2024-04-22T23:46:21.281 INFO:tasks.workunit.client.1.smithi044.stdout:4/889: dwrite d5/d31/d24/d38/d84/fcb [0,4194304] 0 2024-04-22T23:46:21.283 INFO:tasks.workunit.client.1.smithi044.stdout:6/775: write d5/d3f/d4c/d23/f9b [1979010,122295] 0 2024-04-22T23:46:21.285 INFO:tasks.workunit.client.1.smithi044.stdout:6/776: read - d5/d3f/d4c/d40/d53/d56/db7/fe2 zero size 2024-04-22T23:46:21.294 INFO:tasks.workunit.client.1.smithi044.stdout:7/887: dwrite dd/d27/f46 [0,4194304] 0 2024-04-22T23:46:21.301 INFO:tasks.workunit.client.1.smithi044.stdout:4/890: truncate d5/d31/f54 3716235 0 2024-04-22T23:46:21.301 INFO:tasks.workunit.client.1.smithi044.stdout:0/993: write d0/d2/dd/d3e/d40/d43/f4a [2118743,33977] 0 2024-04-22T23:46:21.301 INFO:tasks.workunit.client.1.smithi044.stdout:4/891: stat d5/d31/d44/cb2 0 2024-04-22T23:46:21.303 INFO:tasks.workunit.client.1.smithi044.stdout:4/892: chown d5/db/lc8 4075760 1 2024-04-22T23:46:21.304 INFO:tasks.workunit.client.1.smithi044.stdout:0/994: fsync d0/d2/dd/d3e/d40/d43/f4a 0 2024-04-22T23:46:21.305 INFO:tasks.workunit.client.1.smithi044.stdout:1/875: truncate d1/d8/d20/d106/f109 1190277 0 2024-04-22T23:46:21.306 INFO:tasks.workunit.client.1.smithi044.stdout:9/718: write d2/dc/f66 [4287563,11374] 0 2024-04-22T23:46:21.309 INFO:tasks.workunit.client.1.smithi044.stdout:3/908: getdents d9/d14/d7d/df8/dfd/d3c 0 2024-04-22T23:46:21.310 INFO:tasks.workunit.client.1.smithi044.stdout:3/909: readlink d9/d14/d7d/df8/dfd/d3c/dd1/l12e 0 2024-04-22T23:46:21.311 INFO:tasks.workunit.client.1.smithi044.stdout:3/910: chown d9/d14/d7d/df8/dfd/d3c/f81 5 1 2024-04-22T23:46:21.313 INFO:tasks.workunit.client.1.smithi044.stdout:3/911: read d9/db/d33/d5d/f104 [621191,11100] 0 2024-04-22T23:46:21.314 INFO:tasks.workunit.client.1.smithi044.stdout:3/912: chown d9/d14/d7d/df8/dfd/d3c/fb7 2424 1 2024-04-22T23:46:21.321 INFO:tasks.workunit.client.1.smithi044.stdout:1/876: fdatasync d1/d8/d20/dae/fe2 0 2024-04-22T23:46:21.322 INFO:tasks.workunit.client.1.smithi044.stdout:0/995: dwrite d0/d2/dd/d3e/d14b/f3a [4194304,4194304] 0 2024-04-22T23:46:21.323 INFO:tasks.workunit.client.1.smithi044.stdout:1/877: chown d1/c5 33 1 2024-04-22T23:46:21.330 INFO:tasks.workunit.client.1.smithi044.stdout:0/996: read d0/d2/dd/d3e/d40/d43/dde/ff3 [112489,40845] 0 2024-04-22T23:46:21.331 INFO:tasks.workunit.client.1.smithi044.stdout:0/997: write d0/d2/dd/d3e/d14b/ffb [1458910,126947] 0 2024-04-22T23:46:21.337 INFO:tasks.workunit.client.1.smithi044.stdout:4/893: fsync d5/d52/fc1 0 2024-04-22T23:46:21.338 INFO:tasks.workunit.client.1.smithi044.stdout:6/777: truncate d5/d3f/d4c/d23/f26 521385 0 2024-04-22T23:46:21.390 INFO:tasks.workunit.client.1.smithi044.stdout:8/866: link db/d6c/f91 db/d68/f12f 0 2024-04-22T23:46:21.400 INFO:tasks.workunit.client.1.smithi044.stdout:0/998: rmdir d0 39 2024-04-22T23:46:21.403 INFO:tasks.workunit.client.1.smithi044.stdout:4/894: mknod d5/d31/d44/d7e/daa/c12e 0 2024-04-22T23:46:21.404 INFO:tasks.workunit.client.1.smithi044.stdout:6/778: creat d5/dd/d28/d75/d88/ff3 x:0 0 0 2024-04-22T23:46:21.410 INFO:tasks.workunit.client.1.smithi044.stdout:1/878: symlink d1/d8/d34/da1/d123/l133 0 2024-04-22T23:46:21.412 INFO:tasks.workunit.client.1.smithi044.stdout:8/867: dread db/d14/f88 [0,4194304] 0 2024-04-22T23:46:21.417 INFO:tasks.workunit.client.1.smithi044.stdout:9/719: getdents d2/dc/d1d/d34/d38/d4a 0 2024-04-22T23:46:21.421 INFO:tasks.workunit.client.1.smithi044.stdout:9/720: truncate d2/dc/d1d/d34/d38/d5c/d69/d7a/deb/fec 40254 0 2024-04-22T23:46:21.422 INFO:tasks.workunit.client.1.smithi044.stdout:9/721: read - d2/dc/d94/fc2 zero size 2024-04-22T23:46:21.422 INFO:tasks.workunit.client.1.smithi044.stdout:8/868: chown db/d14/dc7/f122 0 1 2024-04-22T23:46:21.424 INFO:tasks.workunit.client.1.smithi044.stdout:6/779: rename d5/d3f/d4c/db4/fbb to d5/d3f/d50/d79/d96/ff4 0 2024-04-22T23:46:21.425 INFO:tasks.workunit.client.1.smithi044.stdout:3/913: getdents d9/db/d15/d2e 0 2024-04-22T23:46:21.427 INFO:tasks.workunit.client.1.smithi044.stdout:4/895: dread d5/db/d34/fb1 [0,4194304] 0 2024-04-22T23:46:21.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/914: dwrite d9/db/d33/d5d/f66 [0,4194304] 0 2024-04-22T23:46:21.432 INFO:tasks.workunit.client.1.smithi044.stdout:1/879: write d1/d8/d1c/d7b/d80/de3/ff3 [502934,2145] 0 2024-04-22T23:46:21.434 INFO:tasks.workunit.client.1.smithi044.stdout:9/722: read d2/dc/d94/f30 [121548,63506] 0 2024-04-22T23:46:21.435 INFO:tasks.workunit.client.1.smithi044.stdout:0/999: write d0/d2/dd/d3e/d14b/ffd [12583,92880] 0 2024-04-22T23:46:21.436 INFO:tasks.workunit.client.1.smithi044.stdout:6/780: dread - d5/d30/d73/f9f zero size 2024-04-22T23:46:21.441 INFO:tasks.workunit.client.1.smithi044.stdout:8/869: dwrite db/d14/d9e/fe0 [0,4194304] 0 2024-04-22T23:46:21.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:21 smithi031 ceph-mon[26721]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-22T23:46:21.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:21 smithi031 ceph-mon[26721]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-22T23:46:21.444 INFO:tasks.workunit.client.1.smithi044.stdout:1/880: mkdir d1/d8/d1c/d7b/d80/dfd/dbf/ddb/d134 0 2024-04-22T23:46:21.446 INFO:tasks.workunit.client.1.smithi044.stdout:6/781: creat d5/d3f/d50/d79/d96/ff5 x:0 0 0 2024-04-22T23:46:21.454 INFO:tasks.workunit.client.1.smithi044.stdout:9/723: creat d2/dc/d1d/d34/d38/d5c/d69/d75/fef x:0 0 0 2024-04-22T23:46:21.455 INFO:tasks.workunit.client.1.smithi044.stdout:1/881: creat d1/d8/d34/da1/dab/de9/d116/f135 x:0 0 0 2024-04-22T23:46:21.455 INFO:tasks.workunit.client.1.smithi044.stdout:4/896: link d5/db/d16/lc0 d5/d31/d44/d7e/l12f 0 2024-04-22T23:46:21.456 INFO:tasks.workunit.client.1.smithi044.stdout:8/870: creat db/f130 x:0 0 0 2024-04-22T23:46:21.457 INFO:tasks.workunit.client.1.smithi044.stdout:9/724: mkdir d2/dc/d1d/d34/d38/d5c/d7b/df0 0 2024-04-22T23:46:21.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:21 smithi044 ceph-mon[34463]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-22T23:46:21.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:21 smithi044 ceph-mon[34463]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-22T23:46:21.463 INFO:tasks.workunit.client.1.smithi044.stdout:9/725: dread - d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/fcf zero size 2024-04-22T23:46:21.466 INFO:tasks.workunit.client.1.smithi044.stdout:9/726: truncate d2/dc/d1d/d3f/d9d/fe5 239310 0 2024-04-22T23:46:21.467 INFO:tasks.workunit.client.1.smithi044.stdout:6/782: write d5/dd/f1e [3658888,129704] 0 2024-04-22T23:46:21.467 INFO:tasks.workunit.client.1.smithi044.stdout:3/915: dwrite d9/db/d9f/f56 [4194304,4194304] 0 2024-04-22T23:46:21.468 INFO:tasks.workunit.client.1.smithi044.stdout:9/727: stat d2/dc/d1d/d34/d38/d4a 0 2024-04-22T23:46:21.472 INFO:tasks.workunit.client.1.smithi044.stdout:1/882: getdents d1/d8/d20/d45/d75/dfb 0 2024-04-22T23:46:21.474 INFO:tasks.workunit.client.1.smithi044.stdout:8/871: getdents db/d68 0 2024-04-22T23:46:21.475 INFO:tasks.workunit.client.1.smithi044.stdout:4/897: write d5/fd3 [114456,84228] 0 2024-04-22T23:46:21.476 INFO:tasks.workunit.client.1.smithi044.stdout:8/872: chown db/f12c 344962 1 2024-04-22T23:46:21.478 INFO:tasks.workunit.client.1.smithi044.stdout:8/873: dread db/d52/d9c/dad/f116 [0,4194304] 0 2024-04-22T23:46:21.480 INFO:tasks.workunit.client.1.smithi044.stdout:6/783: link d5/d30/d73/ldd d5/d30/d73/lf6 0 2024-04-22T23:46:21.481 INFO:tasks.workunit.client.1.smithi044.stdout:9/728: dread d2/f7d [0,4194304] 0 2024-04-22T23:46:21.481 INFO:tasks.workunit.client.1.smithi044.stdout:6/784: readlink d5/d3f/d4c/d40/l95 0 2024-04-22T23:46:21.482 INFO:tasks.workunit.client.1.smithi044.stdout:8/874: mkdir db/d68/d8b/ded/d131 0 2024-04-22T23:46:21.483 INFO:tasks.workunit.client.1.smithi044.stdout:6/785: fsync d5/d3f/d4c/d40/f78 0 2024-04-22T23:46:21.484 INFO:tasks.workunit.client.1.smithi044.stdout:3/916: getdents d9/db/d9f/d38/d74/dd5 0 2024-04-22T23:46:21.485 INFO:tasks.workunit.client.1.smithi044.stdout:4/898: getdents d5/db/d3c 0 2024-04-22T23:46:21.487 INFO:tasks.workunit.client.1.smithi044.stdout:9/729: rename d2/dc/d94/fc3 to d2/dc/d1d/d34/d38/d5c/d69/d75/d78/ff1 0 2024-04-22T23:46:21.489 INFO:tasks.workunit.client.1.smithi044.stdout:3/917: dread d9/db/d33/d5d/fe3 [0,4194304] 0 2024-04-22T23:46:21.491 INFO:tasks.workunit.client.1.smithi044.stdout:9/730: mkdir d2/d6/d42/df2 0 2024-04-22T23:46:21.492 INFO:tasks.workunit.client.1.smithi044.stdout:3/918: fsync d9/db/d9f/d38/d9e/ff2 0 2024-04-22T23:46:21.493 INFO:tasks.workunit.client.1.smithi044.stdout:3/919: fsync d9/db/d15/d25/d5b/feb 0 2024-04-22T23:46:21.495 INFO:tasks.workunit.client.1.smithi044.stdout:6/786: rename d5/d3f/d4c/d92/d97/de5/fed to d5/d3f/ff7 0 2024-04-22T23:46:21.497 INFO:tasks.workunit.client.1.smithi044.stdout:9/731: rmdir d2/dc/d1d/d34/d38/d5c/d69/d9f 39 2024-04-22T23:46:21.500 INFO:tasks.workunit.client.1.smithi044.stdout:3/920: dwrite d9/dce/fef [0,4194304] 0 2024-04-22T23:46:21.501 INFO:tasks.workunit.client.1.smithi044.stdout:6/787: fdatasync d5/d3f/d50/d79/d96/fac 0 2024-04-22T23:46:21.502 INFO:tasks.workunit.client.1.smithi044.stdout:3/921: fsync d9/db/d33/d5d/fe3 0 2024-04-22T23:46:21.505 INFO:tasks.workunit.client.1.smithi044.stdout:3/922: write d9/db/d33/d5d/d64/f11f [113634,89289] 0 2024-04-22T23:46:21.512 INFO:tasks.workunit.client.1.smithi044.stdout:6/788: truncate d5/d3f/d4c/d23/f3b 1379255 0 2024-04-22T23:46:21.512 INFO:tasks.workunit.client.1.smithi044.stdout:3/923: dwrite f7 [4194304,4194304] 0 2024-04-22T23:46:21.518 INFO:tasks.workunit.client.1.smithi044.stdout:3/924: unlink d9/dce/fef 0 2024-04-22T23:46:21.521 INFO:tasks.workunit.client.1.smithi044.stdout:6/789: mkdir d5/d3f/d50/d79/df8 0 2024-04-22T23:46:21.521 INFO:tasks.workunit.client.1.smithi044.stdout:6/790: stat d5/d3f/d50/fb5 0 2024-04-22T23:46:21.525 INFO:tasks.workunit.client.1.smithi044.stdout:3/925: symlink d9/dc8/l12f 0 2024-04-22T23:46:21.526 INFO:tasks.workunit.client.1.smithi044.stdout:3/926: stat d9/d14/d7d/df8/dfd/d3c 0 2024-04-22T23:46:21.531 INFO:tasks.workunit.client.1.smithi044.stdout:9/732: dread d2/d6/f33 [4194304,4194304] 0 2024-04-22T23:46:21.533 INFO:tasks.workunit.client.1.smithi044.stdout:3/927: getdents d9/d12 0 2024-04-22T23:46:21.533 INFO:tasks.workunit.client.1.smithi044.stdout:9/733: write d2/dc/d1d/d3f/d9d/fe5 [918729,32935] 0 2024-04-22T23:46:21.534 INFO:tasks.workunit.client.1.smithi044.stdout:3/928: write d9/db/d9f/f98 [5197108,98509] 0 2024-04-22T23:46:21.539 INFO:tasks.workunit.client.1.smithi044.stdout:9/734: rmdir d2/dc/d1d/d34/d8e/dbd 39 2024-04-22T23:46:21.541 INFO:tasks.workunit.client.1.smithi044.stdout:6/791: dread d5/dd/f1b [4194304,4194304] 0 2024-04-22T23:46:21.605 INFO:tasks.workunit.client.1.smithi044.stdout:3/929: rmdir d9/d14/d7d/df8/dfd/d3c/d95 39 2024-04-22T23:46:21.611 INFO:tasks.workunit.client.1.smithi044.stdout:9/735: write d2/dc/d1d/fad [565933,53564] 0 2024-04-22T23:46:21.615 INFO:tasks.workunit.client.1.smithi044.stdout:3/930: unlink d9/d14/d7d/df8/dfd/d3c/fbc 0 2024-04-22T23:46:21.616 INFO:tasks.workunit.client.1.smithi044.stdout:9/736: dread d2/dc/d1d/d3f/fd9 [0,4194304] 0 2024-04-22T23:46:21.617 INFO:tasks.workunit.client.1.smithi044.stdout:6/792: mkdir d5/d3f/d4c/d40/df9 0 2024-04-22T23:46:21.617 INFO:tasks.workunit.client.1.smithi044.stdout:9/737: chown d2/fa3 324607050 1 2024-04-22T23:46:21.618 INFO:tasks.workunit.client.1.smithi044.stdout:6/793: write d5/f90 [568154,53652] 0 2024-04-22T23:46:21.626 INFO:tasks.workunit.client.1.smithi044.stdout:9/738: dwrite d2/dc/d1d/d34/d38/d4a/d72/fb1 [0,4194304] 0 2024-04-22T23:46:21.627 INFO:tasks.workunit.client.1.smithi044.stdout:9/739: chown d2/lba 0 1 2024-04-22T23:46:21.628 INFO:tasks.workunit.client.1.smithi044.stdout:9/740: readlink d2/d9/l23 0 2024-04-22T23:46:21.633 INFO:tasks.workunit.client.1.smithi044.stdout:3/931: dread d9/db/fa2 [0,4194304] 0 2024-04-22T23:46:21.640 INFO:tasks.workunit.client.1.smithi044.stdout:3/932: dread d9/db/d15/d25/d67/d6b/ded/d113/fda [0,4194304] 0 2024-04-22T23:46:21.643 INFO:tasks.workunit.client.1.smithi044.stdout:6/794: dwrite d5/dd/f16 [0,4194304] 0 2024-04-22T23:46:21.646 INFO:tasks.workunit.client.1.smithi044.stdout:6/795: write d5/d3f/d4c/d40/d53/fc9 [1063941,22147] 0 2024-04-22T23:46:21.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/741: dread d2/dc/d94/f50 [0,4194304] 0 2024-04-22T23:46:21.663 INFO:tasks.workunit.client.1.smithi044.stdout:9/742: rename d2/d6/d42/l4e to d2/dc/d94/dd3/lf3 0 2024-04-22T23:46:21.665 INFO:tasks.workunit.client.1.smithi044.stdout:6/796: write d5/d3f/d50/d79/d96/fa2 [879506,64971] 0 2024-04-22T23:46:21.669 INFO:tasks.workunit.client.1.smithi044.stdout:9/743: creat d2/dc/d1d/d34/d38/d5c/d69/dcb/ff4 x:0 0 0 2024-04-22T23:46:21.671 INFO:tasks.workunit.client.1.smithi044.stdout:9/744: chown d2/dc/d1d/d34/d38/d5c/d69/d75/fef 133 1 2024-04-22T23:46:21.671 INFO:tasks.workunit.client.1.smithi044.stdout:6/797: creat d5/d3f/d4c/d40/d74/daf/ffa x:0 0 0 2024-04-22T23:46:21.674 INFO:tasks.workunit.client.1.smithi044.stdout:3/933: dread d9/db/d9f/d38/dd9/fa3 [0,4194304] 0 2024-04-22T23:46:21.674 INFO:tasks.workunit.client.1.smithi044.stdout:6/798: read d5/d3f/f99 [184130,124410] 0 2024-04-22T23:46:21.675 INFO:tasks.workunit.client.1.smithi044.stdout:9/745: creat d2/d6/d4d/ff5 x:0 0 0 2024-04-22T23:46:21.676 INFO:tasks.workunit.client.1.smithi044.stdout:6/799: write d5/d3f/d50/d79/f81 [3466539,61190] 0 2024-04-22T23:46:21.677 INFO:tasks.workunit.client.1.smithi044.stdout:6/800: read d5/dd/f16 [2160036,34332] 0 2024-04-22T23:46:21.678 INFO:tasks.workunit.client.1.smithi044.stdout:6/801: write d5/dd/f1e [2879448,51174] 0 2024-04-22T23:46:21.679 INFO:tasks.workunit.client.1.smithi044.stdout:6/802: fsync d5/d3f/d50/fb5 0 2024-04-22T23:46:21.680 INFO:tasks.workunit.client.1.smithi044.stdout:3/934: mkdir d9/d12/d130 0 2024-04-22T23:46:21.680 INFO:tasks.workunit.client.1.smithi044.stdout:6/803: write d5/dd/fc5 [1010283,36490] 0 2024-04-22T23:46:21.690 INFO:tasks.workunit.client.1.smithi044.stdout:3/935: mknod d9/db/d15/d25/d67/d6b/ded/c131 0 2024-04-22T23:46:21.691 INFO:tasks.workunit.client.1.smithi044.stdout:3/936: chown d9/d14/d7d/df8/dfd/d3c/dd1 31225 1 2024-04-22T23:46:21.692 INFO:tasks.workunit.client.1.smithi044.stdout:6/804: mkdir d5/d3f/d4c/df1/dfb 0 2024-04-22T23:46:21.693 INFO:tasks.workunit.client.1.smithi044.stdout:9/746: write d2/dc/f3a [3154881,22] 0 2024-04-22T23:46:21.695 INFO:tasks.workunit.client.1.smithi044.stdout:9/747: fdatasync d2/dc/d1d/d34/d38/d5c/d69/d75/fb5 0 2024-04-22T23:46:21.696 INFO:tasks.workunit.client.1.smithi044.stdout:3/937: mknod d9/db/d15/d2e/d48/c132 0 2024-04-22T23:46:21.696 INFO:tasks.workunit.client.1.smithi044.stdout:9/748: chown d2/d6/d42/c9a 16070048 1 2024-04-22T23:46:21.697 INFO:tasks.workunit.client.1.smithi044.stdout:6/805: rename d5/d3f/d50/d79/d96/fd4 to d5/d3f/d4c/d40/df9/ffc 0 2024-04-22T23:46:21.698 INFO:tasks.workunit.client.1.smithi044.stdout:6/806: stat d5/d3f/d50/d79/d96/ff5 0 2024-04-22T23:46:21.700 INFO:tasks.workunit.client.1.smithi044.stdout:2/691: sync 2024-04-22T23:46:21.700 INFO:tasks.workunit.client.1.smithi044.stdout:7/888: sync 2024-04-22T23:46:21.700 INFO:tasks.workunit.client.1.smithi044.stdout:5/909: sync 2024-04-22T23:46:21.701 INFO:tasks.workunit.client.1.smithi044.stdout:3/938: rmdir d9/db/d9f/d38/d74/dd5 39 2024-04-22T23:46:21.711 INFO:tasks.workunit.client.1.smithi044.stdout:6/807: symlink d5/d3f/lfd 0 2024-04-22T23:46:21.715 INFO:tasks.workunit.client.1.smithi044.stdout:7/889: creat dd/df7/db8/f141 x:0 0 0 2024-04-22T23:46:21.715 INFO:tasks.workunit.client.1.smithi044.stdout:9/749: dwrite d2/d6/fa [0,4194304] 0 2024-04-22T23:46:21.717 INFO:tasks.workunit.client.1.smithi044.stdout:5/910: creat d15/d5d/db5/d10b/f138 x:0 0 0 2024-04-22T23:46:21.723 INFO:tasks.workunit.client.1.smithi044.stdout:3/939: symlink d9/db/d15/d25/d4c/d86/l133 0 2024-04-22T23:46:21.729 INFO:tasks.workunit.client.1.smithi044.stdout:5/911: unlink d15/d16/d1d/d22/d4f/d99/cd3 0 2024-04-22T23:46:21.731 INFO:tasks.workunit.client.1.smithi044.stdout:2/692: link d8/d19/d36/fa5 d8/d27/de5/ff7 0 2024-04-22T23:46:21.733 INFO:tasks.workunit.client.1.smithi044.stdout:3/940: symlink d9/db/d33/d5d/d70/dd8/l134 0 2024-04-22T23:46:21.737 INFO:tasks.workunit.client.1.smithi044.stdout:2/693: creat d8/d19/d1c/ff8 x:0 0 0 2024-04-22T23:46:21.738 INFO:tasks.workunit.client.1.smithi044.stdout:3/941: mknod d9/db/d15/c135 0 2024-04-22T23:46:21.738 INFO:tasks.workunit.client.1.smithi044.stdout:9/750: link d2/dc/d1d/l58 d2/d6/d42/df2/lf6 0 2024-04-22T23:46:21.739 INFO:tasks.workunit.client.1.smithi044.stdout:9/751: fdatasync d2/dc/d1d/d3f/fc1 0 2024-04-22T23:46:21.740 INFO:tasks.workunit.client.1.smithi044.stdout:6/808: link d5/d3f/d4c/l52 d5/d3f/d4c/d23/d3d/lfe 0 2024-04-22T23:46:21.742 INFO:tasks.workunit.client.1.smithi044.stdout:9/752: write d2/dc/d1d/d34/d38/d4a/d72/fb1 [3993277,49029] 0 2024-04-22T23:46:21.746 INFO:tasks.workunit.client.1.smithi044.stdout:5/912: dread f12 [0,4194304] 0 2024-04-22T23:46:21.748 INFO:tasks.workunit.client.1.smithi044.stdout:5/913: write d15/d5d/d111/f12f [203187,54224] 0 2024-04-22T23:46:21.749 INFO:tasks.workunit.client.1.smithi044.stdout:6/809: creat d5/d3f/d4c/db4/dc4/fff x:0 0 0 2024-04-22T23:46:21.749 INFO:tasks.workunit.client.1.smithi044.stdout:9/753: truncate d2/d6/f33 8620898 0 2024-04-22T23:46:21.750 INFO:tasks.workunit.client.1.smithi044.stdout:7/890: dread dd/d98/fca [0,4194304] 0 2024-04-22T23:46:21.750 INFO:tasks.workunit.client.1.smithi044.stdout:8/875: sync 2024-04-22T23:46:21.750 INFO:tasks.workunit.client.1.smithi044.stdout:1/883: sync 2024-04-22T23:46:21.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/899: sync 2024-04-22T23:46:21.754 INFO:tasks.workunit.client.1.smithi044.stdout:1/884: dread - d1/d8/d1c/d7b/d80/dfd/f12a zero size 2024-04-22T23:46:21.756 INFO:tasks.workunit.client.1.smithi044.stdout:2/694: rmdir d8/d19/d36/d46/d63/d65/de2 0 2024-04-22T23:46:21.757 INFO:tasks.workunit.client.1.smithi044.stdout:2/695: stat d8/ce6 0 2024-04-22T23:46:21.758 INFO:tasks.workunit.client.1.smithi044.stdout:1/885: readlink d1/d8/d20/d45/d75/dfb/l12d 0 2024-04-22T23:46:21.759 INFO:tasks.workunit.client.1.smithi044.stdout:2/696: stat d8/ff 0 2024-04-22T23:46:21.759 INFO:tasks.workunit.client.1.smithi044.stdout:5/914: read d15/d5d/d82/f83 [1624435,106208] 0 2024-04-22T23:46:21.761 INFO:tasks.workunit.client.1.smithi044.stdout:6/810: dwrite d5/dd/d28/d75/d88/ff3 [0,4194304] 0 2024-04-22T23:46:21.763 INFO:tasks.workunit.client.1.smithi044.stdout:7/891: dwrite dd/d1a/f7d [4194304,4194304] 0 2024-04-22T23:46:21.781 INFO:tasks.workunit.client.1.smithi044.stdout:1/886: truncate d1/d8/d34/f40 1374259 0 2024-04-22T23:46:21.783 INFO:tasks.workunit.client.1.smithi044.stdout:2/697: dread d8/d19/f25 [0,4194304] 0 2024-04-22T23:46:21.783 INFO:tasks.workunit.client.1.smithi044.stdout:1/887: chown d1/d8/d34/da1/dab/cce 161 1 2024-04-22T23:46:21.786 INFO:tasks.workunit.client.1.smithi044.stdout:7/892: dread dd/d99/f10e [0,4194304] 0 2024-04-22T23:46:21.789 INFO:tasks.workunit.client.1.smithi044.stdout:3/942: write d9/d12/db1/fbd [492027,85839] 0 2024-04-22T23:46:21.790 INFO:tasks.workunit.client.1.smithi044.stdout:5/915: symlink d15/d5d/db5/dc0/d38/d9f/l139 0 2024-04-22T23:46:21.795 INFO:tasks.workunit.client.1.smithi044.stdout:8/876: write db/d14/f25 [754021,44530] 0 2024-04-22T23:46:21.795 INFO:tasks.workunit.client.1.smithi044.stdout:4/900: mknod d5/d31/d44/d20/c130 0 2024-04-22T23:46:21.796 INFO:tasks.workunit.client.1.smithi044.stdout:4/901: stat d5/d42/ld0 0 2024-04-22T23:46:21.804 INFO:tasks.workunit.client.1.smithi044.stdout:7/893: write dd/d55/d5e/d60/d68/f73 [1398976,30843] 0 2024-04-22T23:46:21.804 INFO:tasks.workunit.client.1.smithi044.stdout:6/811: write d5/d3f/d4c/d4f/f59 [498725,81411] 0 2024-04-22T23:46:21.804 INFO:tasks.workunit.client.1.smithi044.stdout:6/812: fdatasync f1 0 2024-04-22T23:46:21.805 INFO:tasks.workunit.client.1.smithi044.stdout:5/916: mknod d15/d16/d1d/d22/d4f/dc9/c13a 0 2024-04-22T23:46:21.809 INFO:tasks.workunit.client.1.smithi044.stdout:3/943: creat d9/db/d15/d25/d67/d6b/ded/d113/dbb/d10c/f136 x:0 0 0 2024-04-22T23:46:21.810 INFO:tasks.workunit.client.1.smithi044.stdout:9/754: truncate d2/f7d 197794 0 2024-04-22T23:46:21.811 INFO:tasks.workunit.client.1.smithi044.stdout:8/877: mknod db/d68/d8b/dca/dd6/c132 0 2024-04-22T23:46:21.812 INFO:tasks.workunit.client.1.smithi044.stdout:7/894: dwrite dd/d1f/d24/fa2 [0,4194304] 0 2024-04-22T23:46:21.812 INFO:tasks.workunit.client.1.smithi044.stdout:9/755: write d2/dc/d1d/d3f/f4c [3599279,72936] 0 2024-04-22T23:46:21.815 INFO:tasks.workunit.client.1.smithi044.stdout:2/698: symlink d8/d19/d36/d46/ddb/ddd/df5/lf9 0 2024-04-22T23:46:21.818 INFO:tasks.workunit.client.1.smithi044.stdout:1/888: mknod d1/d8/d20/d47/d92/c136 0 2024-04-22T23:46:21.823 INFO:tasks.workunit.client.1.smithi044.stdout:4/902: link d5/d31/d44/d7e/daa/f121 d5/db/d16/d4d/dad/dc5/dd9/f131 0 2024-04-22T23:46:21.824 INFO:tasks.workunit.client.1.smithi044.stdout:4/903: readlink d5/d31/d44/d20/d22/l8b 0 2024-04-22T23:46:21.828 INFO:tasks.workunit.client.1.smithi044.stdout:9/756: unlink d2/dc/d1d/d34/d38/d4a/dca/fe9 0 2024-04-22T23:46:21.828 INFO:tasks.workunit.client.1.smithi044.stdout:3/944: dread d9/db/f10 [4194304,4194304] 0 2024-04-22T23:46:21.829 INFO:tasks.workunit.client.1.smithi044.stdout:2/699: creat d8/d60/ffa x:0 0 0 2024-04-22T23:46:21.829 INFO:tasks.workunit.client.1.smithi044.stdout:3/945: dread - d9/db/ff6 zero size 2024-04-22T23:46:21.834 INFO:tasks.workunit.client.1.smithi044.stdout:1/889: rename d1/d8/d20/d45/f70 to d1/d8/d1c/d23/d3e/dde/f137 0 2024-04-22T23:46:21.835 INFO:tasks.workunit.client.1.smithi044.stdout:3/946: dwrite d9/db/d15/f18 [4194304,4194304] 0 2024-04-22T23:46:21.839 INFO:tasks.workunit.client.1.smithi044.stdout:4/904: creat d5/db/d16/d4d/f132 x:0 0 0 2024-04-22T23:46:21.842 INFO:tasks.workunit.client.1.smithi044.stdout:4/905: truncate d5/d31/d44/d20/f120 1007971 0 2024-04-22T23:46:21.845 INFO:tasks.workunit.client.1.smithi044.stdout:9/757: mknod d2/d6/d42/daa/cf7 0 2024-04-22T23:46:21.846 INFO:tasks.workunit.client.1.smithi044.stdout:8/878: dwrite db/f11 [8388608,4194304] 0 2024-04-22T23:46:21.848 INFO:tasks.workunit.client.1.smithi044.stdout:8/879: dread - db/d68/f12b zero size 2024-04-22T23:46:21.851 INFO:tasks.workunit.client.1.smithi044.stdout:2/700: rmdir d8/d60/dc1 39 2024-04-22T23:46:21.853 INFO:tasks.workunit.client.1.smithi044.stdout:2/701: chown d8/d19/d36/fa5 1635 1 2024-04-22T23:46:21.856 INFO:tasks.workunit.client.1.smithi044.stdout:5/917: link d15/d16/d1d/l5b d15/d16/d1d/d22/d4f/l13b 0 2024-04-22T23:46:21.860 INFO:tasks.workunit.client.1.smithi044.stdout:1/890: dread d1/d8/d1c/d7b/d80/fe4 [0,4194304] 0 2024-04-22T23:46:21.860 INFO:tasks.workunit.client.1.smithi044.stdout:3/947: mknod d9/db/d15/d25/d67/d6b/ded/d113/dbb/c137 0 2024-04-22T23:46:21.863 INFO:tasks.workunit.client.1.smithi044.stdout:7/895: mkdir dd/d114/d142 0 2024-04-22T23:46:21.866 INFO:tasks.workunit.client.1.smithi044.stdout:9/758: creat d2/dc/d1d/d34/d38/d4a/db7/ff8 x:0 0 0 2024-04-22T23:46:21.867 INFO:tasks.workunit.client.1.smithi044.stdout:8/880: symlink db/d68/d8b/dca/dd6/l133 0 2024-04-22T23:46:21.869 INFO:tasks.workunit.client.1.smithi044.stdout:2/702: rename d8/d19/d36/d46/d63/f71 to d8/d19/d36/d46/d63/de0/ffb 0 2024-04-22T23:46:21.873 INFO:tasks.workunit.client.1.smithi044.stdout:1/891: symlink d1/d8/d34/da1/dab/l138 0 2024-04-22T23:46:21.876 INFO:tasks.workunit.client.1.smithi044.stdout:3/948: rmdir d9/db/d15/d25/d67/d6b/ded/d113 39 2024-04-22T23:46:21.878 INFO:tasks.workunit.client.1.smithi044.stdout:7/896: symlink dd/df7/d112/l143 0 2024-04-22T23:46:21.878 INFO:tasks.workunit.client.1.smithi044.stdout:7/897: stat fa 0 2024-04-22T23:46:21.879 INFO:tasks.workunit.client.1.smithi044.stdout:9/759: creat d2/d6/d42/daa/ff9 x:0 0 0 2024-04-22T23:46:21.880 INFO:tasks.workunit.client.1.smithi044.stdout:8/881: mkdir db/d14/d84/dbb/d134 0 2024-04-22T23:46:21.884 INFO:tasks.workunit.client.1.smithi044.stdout:6/813: truncate d5/dd/fa8 731603 0 2024-04-22T23:46:21.885 INFO:tasks.workunit.client.1.smithi044.stdout:1/892: mknod d1/d8/d20/d47/d92/c139 0 2024-04-22T23:46:21.887 INFO:tasks.workunit.client.1.smithi044.stdout:3/949: mkdir d9/db/d33/d5d/d64/d138 0 2024-04-22T23:46:21.890 INFO:tasks.workunit.client.1.smithi044.stdout:9/760: write d2/dc/d1d/d34/d8e/dbd/fee [422495,121933] 0 2024-04-22T23:46:21.891 INFO:tasks.workunit.client.1.smithi044.stdout:6/814: dwrite d5/d3f/d4c/d92/fa5 [0,4194304] 0 2024-04-22T23:46:21.894 INFO:tasks.workunit.client.1.smithi044.stdout:4/906: getdents d5/db/d34 0 2024-04-22T23:46:21.899 INFO:tasks.workunit.client.1.smithi044.stdout:3/950: symlink d9/db/d15/d2e/d48/l139 0 2024-04-22T23:46:21.900 INFO:tasks.workunit.client.1.smithi044.stdout:7/898: write dd/d1f/d24/f1e [1084825,118631] 0 2024-04-22T23:46:21.904 INFO:tasks.workunit.client.1.smithi044.stdout:6/815: fdatasync d5/d3f/d50/d79/d96/ff4 0 2024-04-22T23:46:21.908 INFO:tasks.workunit.client.1.smithi044.stdout:1/893: truncate d1/d8/d20/d45/d75/df2/d99/fc4 1752819 0 2024-04-22T23:46:21.908 INFO:tasks.workunit.client.1.smithi044.stdout:1/894: readlink d1/l2b 0 2024-04-22T23:46:21.909 INFO:tasks.workunit.client.1.smithi044.stdout:3/951: mknod d9/d14/d7d/df8/c13a 0 2024-04-22T23:46:21.911 INFO:tasks.workunit.client.1.smithi044.stdout:7/899: symlink dd/d1a/db7/dc8/d97/db9/l144 0 2024-04-22T23:46:21.913 INFO:tasks.workunit.client.1.smithi044.stdout:4/907: symlink d5/d31/l133 0 2024-04-22T23:46:21.913 INFO:tasks.workunit.client.1.smithi044.stdout:3/952: dread d9/fb9 [0,4194304] 0 2024-04-22T23:46:21.921 INFO:tasks.workunit.client.1.smithi044.stdout:1/895: mknod d1/d8/d1c/d7b/d80/dfd/dbf/c13a 0 2024-04-22T23:46:21.932 INFO:tasks.workunit.client.1.smithi044.stdout:1/896: rmdir d1/d8/d34 39 2024-04-22T23:46:21.935 INFO:tasks.workunit.client.1.smithi044.stdout:3/953: rename d9/db/d15/d25/d67 to d9/db/d15/d2e/d13b 0 2024-04-22T23:46:21.937 INFO:tasks.workunit.client.1.smithi044.stdout:3/954: truncate d9/db/ff6 261593 0 2024-04-22T23:46:21.938 INFO:tasks.workunit.client.1.smithi044.stdout:3/955: chown d9/db/d33/faf 1 1 2024-04-22T23:46:21.945 INFO:tasks.workunit.client.1.smithi044.stdout:3/956: mknod d9/db/d9f/dcd/c13c 0 2024-04-22T23:46:21.946 INFO:tasks.workunit.client.1.smithi044.stdout:3/957: chown d9/db/cb0 655 1 2024-04-22T23:46:21.946 INFO:tasks.workunit.client.1.smithi044.stdout:1/897: truncate d1/d8/d34/da1/fbd 369655 0 2024-04-22T23:46:21.954 INFO:tasks.workunit.client.1.smithi044.stdout:1/898: dwrite d1/d8/d1c/d7b/d80/dfd/dbf/dc2/ff0 [0,4194304] 0 2024-04-22T23:46:21.955 INFO:tasks.workunit.client.1.smithi044.stdout:4/908: dread d5/d31/d44/d20/d22/f4c [0,4194304] 0 2024-04-22T23:46:21.956 INFO:tasks.workunit.client.1.smithi044.stdout:4/909: chown d5/db/d16/d4d/dad/dc5 89502 1 2024-04-22T23:46:21.958 INFO:tasks.workunit.client.1.smithi044.stdout:4/910: write d5/d31/d44/d20/d22/fbf [29702,119007] 0 2024-04-22T23:46:21.960 INFO:tasks.workunit.client.1.smithi044.stdout:4/911: readlink d5/d42/l74 0 2024-04-22T23:46:21.961 INFO:tasks.workunit.client.1.smithi044.stdout:9/761: fsync d2/d6/d42/daa/ff9 0 2024-04-22T23:46:21.963 INFO:tasks.workunit.client.1.smithi044.stdout:9/762: fdatasync d2/d6/d42/d8b/fc5 0 2024-04-22T23:46:21.966 INFO:tasks.workunit.client.1.smithi044.stdout:1/899: dwrite d1/d8/d1c/d23/d3e/f90 [0,4194304] 0 2024-04-22T23:46:21.968 INFO:tasks.workunit.client.1.smithi044.stdout:5/918: truncate d15/d5d/db5/dc0/f66 2533117 0 2024-04-22T23:46:21.969 INFO:tasks.workunit.client.1.smithi044.stdout:5/919: dread - d15/d16/d1d/d22/d4f/d68/f129 zero size 2024-04-22T23:46:21.970 INFO:tasks.workunit.client.1.smithi044.stdout:2/703: write d8/d19/f37 [4455406,45149] 0 2024-04-22T23:46:21.970 INFO:tasks.workunit.client.1.smithi044.stdout:5/920: stat d15/d5d/d64/l6e 0 2024-04-22T23:46:21.973 INFO:tasks.workunit.client.1.smithi044.stdout:9/763: rmdir d2/d6 39 2024-04-22T23:46:21.987 INFO:tasks.workunit.client.1.smithi044.stdout:9/764: creat d2/dc/d94/ffa x:0 0 0 2024-04-22T23:46:21.988 INFO:tasks.workunit.client.1.smithi044.stdout:5/921: dread d15/d16/d1d/f89 [0,4194304] 0 2024-04-22T23:46:21.989 INFO:tasks.workunit.client.1.smithi044.stdout:9/765: read - d2/d6/d42/d8b/ddc/fe4 zero size 2024-04-22T23:46:21.990 INFO:tasks.workunit.client.1.smithi044.stdout:8/882: write db/d14/f2b [1168087,92246] 0 2024-04-22T23:46:21.991 INFO:tasks.workunit.client.1.smithi044.stdout:9/766: stat d2/d9/c3c 0 2024-04-22T23:46:21.992 INFO:tasks.workunit.client.1.smithi044.stdout:9/767: chown d2/dc/f66 120174932 1 2024-04-22T23:46:21.994 INFO:tasks.workunit.client.1.smithi044.stdout:9/768: chown d2/dc/d1d/d3f/l47 17 1 2024-04-22T23:46:21.995 INFO:tasks.workunit.client.1.smithi044.stdout:5/922: dread d15/d16/d1d/d22/d2a/d3a/f96 [0,4194304] 0 2024-04-22T23:46:22.004 INFO:tasks.workunit.client.1.smithi044.stdout:9/769: link d2/dc/d1d/d34/d38/d4a/d72/l91 d2/dc/d1d/d34/d38/lfb 0 2024-04-22T23:46:22.004 INFO:tasks.workunit.client.1.smithi044.stdout:6/816: dwrite d5/dd/d67/f47 [4194304,4194304] 0 2024-04-22T23:46:22.005 INFO:tasks.workunit.client.1.smithi044.stdout:5/923: dwrite d15/d16/d1d/f42 [0,4194304] 0 2024-04-22T23:46:22.007 INFO:tasks.workunit.client.1.smithi044.stdout:7/900: truncate dd/d1a/f7e 4142289 0 2024-04-22T23:46:22.008 INFO:tasks.workunit.client.1.smithi044.stdout:9/770: write d2/d6/d42/daa/fae [1026056,98439] 0 2024-04-22T23:46:22.015 INFO:tasks.workunit.client.1.smithi044.stdout:3/958: mknod d9/db/d15/d2e/d48/c13d 0 2024-04-22T23:46:22.021 INFO:tasks.workunit.client.1.smithi044.stdout:7/901: rename dd/d55/d5e/d60/d68/d94/dbd/dc9/d123/l12c to dd/d55/dcb/l145 0 2024-04-22T23:46:22.022 INFO:tasks.workunit.client.1.smithi044.stdout:4/912: write d5/db/d16/d4d/dad/dc5/dd9/fed [4462270,85457] 0 2024-04-22T23:46:22.022 INFO:tasks.workunit.client.1.smithi044.stdout:6/817: mknod d5/d3f/d4c/c100 0 2024-04-22T23:46:22.027 INFO:tasks.workunit.client.1.smithi044.stdout:1/900: write d1/d8/d1c/d7b/d80/d46/fc9 [4110641,40851] 0 2024-04-22T23:46:22.028 INFO:tasks.workunit.client.1.smithi044.stdout:6/818: read d5/d3f/d50/d79/d96/fa2 [56499,52329] 0 2024-04-22T23:46:22.028 INFO:tasks.workunit.client.1.smithi044.stdout:5/924: truncate d15/fea 446527 0 2024-04-22T23:46:22.029 INFO:tasks.workunit.client.1.smithi044.stdout:6/819: chown d5/dd/d28/fe7 2 1 2024-04-22T23:46:22.029 INFO:tasks.workunit.client.1.smithi044.stdout:5/925: truncate d15/d3c/f53 5072374 0 2024-04-22T23:46:22.029 INFO:tasks.workunit.client.1.smithi044.stdout:2/704: write d8/ff [3878340,22483] 0 2024-04-22T23:46:22.039 INFO:tasks.workunit.client.1.smithi044.stdout:8/883: dwrite db/d14/d9e/ffa [0,4194304] 0 2024-04-22T23:46:22.050 INFO:tasks.workunit.client.1.smithi044.stdout:1/901: dread d1/d8/d20/d45/d5d/f61 [0,4194304] 0 2024-04-22T23:46:22.051 INFO:tasks.workunit.client.1.smithi044.stdout:1/902: readlink d1/d8/d34/d112/lda 0 2024-04-22T23:46:22.098 INFO:tasks.workunit.client.1.smithi044.stdout:9/771: getdents d2/dc/d1d/d34/d38/d5c/d7b/df0 0 2024-04-22T23:46:22.104 INFO:tasks.workunit.client.1.smithi044.stdout:4/913: dread d5/d31/d24/d69/d90/fdd [0,4194304] 0 2024-04-22T23:46:22.107 INFO:tasks.workunit.client.1.smithi044.stdout:6/820: mknod d5/dd/d28/c101 0 2024-04-22T23:46:22.110 INFO:tasks.workunit.client.1.smithi044.stdout:5/926: creat d15/d16/d1d/d22/d2a/d3a/d43/d11d/f13c x:0 0 0 2024-04-22T23:46:22.115 INFO:tasks.workunit.client.1.smithi044.stdout:8/884: rename db/d68/d8b/dca/dd6/l104 to db/d52/d9c/dad/l135 0 2024-04-22T23:46:22.116 INFO:tasks.workunit.client.1.smithi044.stdout:8/885: dread - db/d68/d74/d112/f11f zero size 2024-04-22T23:46:22.118 INFO:tasks.workunit.client.1.smithi044.stdout:1/903: mkdir d1/d8/d34/d13b 0 2024-04-22T23:46:22.123 INFO:tasks.workunit.client.1.smithi044.stdout:3/959: mknod d9/db/d15/d25/d5b/c13e 0 2024-04-22T23:46:22.124 INFO:tasks.workunit.client.1.smithi044.stdout:7/902: creat dd/d1a/db7/dc8/d97/d13c/f146 x:0 0 0 2024-04-22T23:46:22.125 INFO:tasks.workunit.client.1.smithi044.stdout:7/903: fdatasync dd/df7/db8/f109 0 2024-04-22T23:46:22.126 INFO:tasks.workunit.client.1.smithi044.stdout:4/914: rmdir d5/d31/d44/d20/d22/d8d 39 2024-04-22T23:46:22.129 INFO:tasks.workunit.client.1.smithi044.stdout:5/927: mkdir d15/d16/d1d/d22/d2a/d3a/dd2/d13d 0 2024-04-22T23:46:22.130 INFO:tasks.workunit.client.1.smithi044.stdout:8/886: mknod db/d14/d24/d127/c136 0 2024-04-22T23:46:22.131 INFO:tasks.workunit.client.1.smithi044.stdout:1/904: chown d1/d8/d1c/d23/c82 217369348 1 2024-04-22T23:46:22.133 INFO:tasks.workunit.client.1.smithi044.stdout:8/887: write db/d14/d72/ffd [3084507,40115] 0 2024-04-22T23:46:22.133 INFO:tasks.workunit.client.1.smithi044.stdout:7/904: truncate fa 4984384 0 2024-04-22T23:46:22.135 INFO:tasks.workunit.client.1.smithi044.stdout:1/905: dread d1/d8/d34/da1/dab/fe8 [0,4194304] 0 2024-04-22T23:46:22.139 INFO:tasks.workunit.client.1.smithi044.stdout:6/821: symlink d5/d3f/da7/l102 0 2024-04-22T23:46:22.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/928: mknod d15/d16/d1d/d22/d4f/c13e 0 2024-04-22T23:46:22.140 INFO:tasks.workunit.client.1.smithi044.stdout:5/929: stat d15/d16/d1d/d22/d4f/dc9/c13a 0 2024-04-22T23:46:22.143 INFO:tasks.workunit.client.1.smithi044.stdout:3/960: getdents d9/d12/d130 0 2024-04-22T23:46:22.147 INFO:tasks.workunit.client.1.smithi044.stdout:8/888: mkdir db/d68/d137 0 2024-04-22T23:46:22.148 INFO:tasks.workunit.client.1.smithi044.stdout:8/889: write db/d14/d24/f59 [5025594,78385] 0 2024-04-22T23:46:22.150 INFO:tasks.workunit.client.1.smithi044.stdout:7/905: read dd/d27/f48 [159896,45413] 0 2024-04-22T23:46:22.153 INFO:tasks.workunit.client.1.smithi044.stdout:2/705: getdents d8/d19/d36/d93/dae 0 2024-04-22T23:46:22.154 INFO:tasks.workunit.client.1.smithi044.stdout:4/915: write d5/d31/d44/f29 [588083,100488] 0 2024-04-22T23:46:22.157 INFO:tasks.workunit.client.1.smithi044.stdout:2/706: dread d8/d19/f25 [0,4194304] 0 2024-04-22T23:46:22.161 INFO:tasks.workunit.client.1.smithi044.stdout:9/772: getdents d2/dc/d1d/d34/d38 0 2024-04-22T23:46:22.162 INFO:tasks.workunit.client.1.smithi044.stdout:3/961: read d9/d14/dd2/f11b [475358,99655] 0 2024-04-22T23:46:22.164 INFO:tasks.workunit.client.1.smithi044.stdout:3/962: write d9/db/d15/d2e/d48/f102 [905820,52162] 0 2024-04-22T23:46:22.165 INFO:tasks.workunit.client.1.smithi044.stdout:7/906: creat dd/d114/f147 x:0 0 0 2024-04-22T23:46:22.171 INFO:tasks.workunit.client.1.smithi044.stdout:4/916: fdatasync d5/d31/d24/d38/f91 0 2024-04-22T23:46:22.172 INFO:tasks.workunit.client.1.smithi044.stdout:6/822: mknod d5/d3f/d4c/d23/ddb/c103 0 2024-04-22T23:46:22.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:22 smithi031 ceph-mon[26721]: pgmap v14: 65 pgs: 65 active+clean; 2.8 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 62 MiB/s rd, 176 MiB/s wr, 410 op/s 2024-04-22T23:46:22.173 INFO:tasks.workunit.client.1.smithi044.stdout:6/823: write d5/d3f/d4c/d23/d3d/d64/f94 [1612866,22393] 0 2024-04-22T23:46:22.174 INFO:tasks.workunit.client.1.smithi044.stdout:5/930: mkdir d15/d5d/db5/d10f/d13f 0 2024-04-22T23:46:22.174 INFO:tasks.workunit.client.1.smithi044.stdout:1/906: write d1/d8/d20/d106/f109 [1641676,116666] 0 2024-04-22T23:46:22.180 INFO:tasks.workunit.client.1.smithi044.stdout:7/907: symlink dd/d55/d5e/d60/d68/l148 0 2024-04-22T23:46:22.186 INFO:tasks.workunit.client.1.smithi044.stdout:7/908: dwrite dd/d1a/f8e [0,4194304] 0 2024-04-22T23:46:22.191 INFO:tasks.workunit.client.1.smithi044.stdout:1/907: creat d1/d8/d20/d47/d92/f13c x:0 0 0 2024-04-22T23:46:22.194 INFO:tasks.workunit.client.1.smithi044.stdout:9/773: unlink d2/dc/l5f 0 2024-04-22T23:46:22.200 INFO:tasks.workunit.client.1.smithi044.stdout:6/824: mknod d5/dd/c104 0 2024-04-22T23:46:22.201 INFO:tasks.workunit.client.1.smithi044.stdout:5/931: mknod d15/d3c/df9/d133/c140 0 2024-04-22T23:46:22.204 INFO:tasks.workunit.client.1.smithi044.stdout:9/774: dwrite d2/f3d [4194304,4194304] 0 2024-04-22T23:46:22.204 INFO:tasks.workunit.client.1.smithi044.stdout:1/908: mknod d1/d8/d34/d112/c13d 0 2024-04-22T23:46:22.204 INFO:tasks.workunit.client.1.smithi044.stdout:4/917: dwrite d5/db/d3c/ffa [0,4194304] 0 2024-04-22T23:46:22.205 INFO:tasks.workunit.client.1.smithi044.stdout:7/909: dread dd/d99/dbb/ff6 [0,4194304] 0 2024-04-22T23:46:22.209 INFO:tasks.workunit.client.1.smithi044.stdout:3/963: rename d9/db/d15/d25/d4c/f4d to d9/d14/d7d/df8/dfd/d3c/f13f 0 2024-04-22T23:46:22.215 INFO:tasks.workunit.client.1.smithi044.stdout:1/909: write d1/d8/d34/f9d [239829,30922] 0 2024-04-22T23:46:22.223 INFO:tasks.workunit.client.1.smithi044.stdout:2/707: rename d8/l1b to d8/d60/dc1/lfc 0 2024-04-22T23:46:22.230 INFO:tasks.workunit.client.1.smithi044.stdout:5/932: symlink d15/d16/l141 0 2024-04-22T23:46:22.232 INFO:tasks.workunit.client.1.smithi044.stdout:4/918: creat d5/d31/d44/d20/d93/f134 x:0 0 0 2024-04-22T23:46:22.234 INFO:tasks.workunit.client.1.smithi044.stdout:9/775: mknod d2/dc/d1d/d34/d38/d5c/d69/d9f/cfc 0 2024-04-22T23:46:22.234 INFO:tasks.workunit.client.1.smithi044.stdout:4/919: write d5/d31/d24/f72 [4072404,22416] 0 2024-04-22T23:46:22.238 INFO:tasks.workunit.client.1.smithi044.stdout:2/708: symlink d8/d19/d36/d44/dd8/lfd 0 2024-04-22T23:46:22.239 INFO:tasks.workunit.client.1.smithi044.stdout:7/910: write dd/d99/fc2 [951117,88221] 0 2024-04-22T23:46:22.242 INFO:tasks.workunit.client.1.smithi044.stdout:5/933: dread - d15/d5d/db5/dc0/f8e zero size 2024-04-22T23:46:22.244 INFO:tasks.workunit.client.1.smithi044.stdout:4/920: creat d5/d31/d24/d69/f135 x:0 0 0 2024-04-22T23:46:22.246 INFO:tasks.workunit.client.1.smithi044.stdout:2/709: chown d8/d19/d36/d46/d63/de0/ffb 15 1 2024-04-22T23:46:22.247 INFO:tasks.workunit.client.1.smithi044.stdout:3/964: dwrite d9/db/d33/faf [0,4194304] 0 2024-04-22T23:46:22.252 INFO:tasks.workunit.client.1.smithi044.stdout:7/911: unlink dd/d1a/db7/dc8/d97/c83 0 2024-04-22T23:46:22.252 INFO:tasks.workunit.client.1.smithi044.stdout:7/912: dread - dd/df7/db8/f141 zero size 2024-04-22T23:46:22.253 INFO:tasks.workunit.client.1.smithi044.stdout:7/913: stat dd/d1f/l42 0 2024-04-22T23:46:22.253 INFO:tasks.workunit.client.1.smithi044.stdout:1/910: creat d1/d8/d1c/f13e x:0 0 0 2024-04-22T23:46:22.255 INFO:tasks.workunit.client.1.smithi044.stdout:5/934: chown d15/d5d/ld9 1427 1 2024-04-22T23:46:22.255 INFO:tasks.workunit.client.1.smithi044.stdout:4/921: dread - d5/d31/d24/d38/d6a/fcf zero size 2024-04-22T23:46:22.259 INFO:tasks.workunit.client.1.smithi044.stdout:6/825: link d5/dd/c37 d5/dd/c105 0 2024-04-22T23:46:22.261 INFO:tasks.workunit.client.1.smithi044.stdout:5/935: dread d15/d16/d1d/d22/d2a/d3a/d43/fa8 [0,4194304] 0 2024-04-22T23:46:22.262 INFO:tasks.workunit.client.1.smithi044.stdout:5/936: chown d15/d16/d1d/d22/c8b 1941 1 2024-04-22T23:46:22.263 INFO:tasks.workunit.client.1.smithi044.stdout:8/890: rename db/d68/d8b/ded/c128 to db/d14/d84/c138 0 2024-04-22T23:46:22.264 INFO:tasks.workunit.client.1.smithi044.stdout:8/891: write db/d6c/fa0 [199003,29317] 0 2024-04-22T23:46:22.265 INFO:tasks.workunit.client.1.smithi044.stdout:8/892: chown db/l23 1 1 2024-04-22T23:46:22.265 INFO:tasks.workunit.client.1.smithi044.stdout:4/922: creat d5/d31/d44/dd2/f136 x:0 0 0 2024-04-22T23:46:22.269 INFO:tasks.workunit.client.1.smithi044.stdout:6/826: truncate d5/d30/f39 163974 0 2024-04-22T23:46:22.271 INFO:tasks.workunit.client.1.smithi044.stdout:7/914: truncate dd/d1a/f7e 3564597 0 2024-04-22T23:46:22.272 INFO:tasks.workunit.client.1.smithi044.stdout:7/915: chown dd/d55/d5e/d60/d106 9 1 2024-04-22T23:46:22.273 INFO:tasks.workunit.client.1.smithi044.stdout:5/937: mknod d15/d5d/d82/c142 0 2024-04-22T23:46:22.274 INFO:tasks.workunit.client.1.smithi044.stdout:7/916: chown dd/d1a/db7/dc8/d97/db9/f105 6174771 1 2024-04-22T23:46:22.276 INFO:tasks.workunit.client.1.smithi044.stdout:9/776: rename d2/dc/d94/dd3 to d2/dc/d1d/d34/d38/d5c/d69/dcb/dfd 0 2024-04-22T23:46:22.277 INFO:tasks.workunit.client.1.smithi044.stdout:9/777: chown d2/d6/d42 0 1 2024-04-22T23:46:22.277 INFO:tasks.workunit.client.1.smithi044.stdout:2/710: dwrite d8/d19/d5d/f5e [0,4194304] 0 2024-04-22T23:46:22.282 INFO:tasks.workunit.client.1.smithi044.stdout:3/965: symlink d9/db/d15/d2e/d13b/d6b/l140 0 2024-04-22T23:46:22.290 INFO:tasks.workunit.client.1.smithi044.stdout:4/923: write d5/db/d34/f78 [5238013,130737] 0 2024-04-22T23:46:22.290 INFO:tasks.workunit.client.1.smithi044.stdout:4/924: stat d5/d31/fef 0 2024-04-22T23:46:22.291 INFO:tasks.workunit.client.1.smithi044.stdout:9/778: creat d2/dc/d1d/d34/d38/d5c/ffe x:0 0 0 2024-04-22T23:46:22.295 INFO:tasks.workunit.client.1.smithi044.stdout:5/938: write d15/d5d/db5/dc0/f54 [12508418,30852] 0 2024-04-22T23:46:22.296 INFO:tasks.workunit.client.1.smithi044.stdout:6/827: fsync d5/f21 0 2024-04-22T23:46:22.299 INFO:tasks.workunit.client.1.smithi044.stdout:4/925: rename d5/db/d34/c3b to d5/d31/d24/da6/db6/c137 0 2024-04-22T23:46:22.301 INFO:tasks.workunit.client.1.smithi044.stdout:8/893: write db/d14/dc7/dc9/fe9 [1449233,54752] 0 2024-04-22T23:46:22.304 INFO:tasks.workunit.client.1.smithi044.stdout:3/966: mkdir d9/db/d141 0 2024-04-22T23:46:22.305 INFO:tasks.workunit.client.1.smithi044.stdout:3/967: dread - d9/db/d15/d2e/d13b/d6b/f112 zero size 2024-04-22T23:46:22.308 INFO:tasks.workunit.client.1.smithi044.stdout:5/939: rename d15/d16/d1d/f49 to d15/d16/d1d/daf/f143 0 2024-04-22T23:46:22.309 INFO:tasks.workunit.client.1.smithi044.stdout:5/940: write d15/d5d/d82/f130 [861711,2946] 0 2024-04-22T23:46:22.312 INFO:tasks.workunit.client.1.smithi044.stdout:9/779: mknod d2/dc/d1d/d34/d38/d5c/d69/d75/cff 0 2024-04-22T23:46:22.312 INFO:tasks.workunit.client.1.smithi044.stdout:2/711: dread d8/d19/d36/d44/f52 [0,4194304] 0 2024-04-22T23:46:22.318 INFO:tasks.workunit.client.1.smithi044.stdout:2/712: dwrite d8/d19/f37 [0,4194304] 0 2024-04-22T23:46:22.318 INFO:tasks.workunit.client.1.smithi044.stdout:7/917: getdents dd/d114 0 2024-04-22T23:46:22.322 INFO:tasks.workunit.client.1.smithi044.stdout:6/828: rename d5/d3f/d4c/d40/d53/fc9 to d5/d3f/d50/d79/df8/f106 0 2024-04-22T23:46:22.324 INFO:tasks.workunit.client.1.smithi044.stdout:4/926: symlink d5/d31/d44/l138 0 2024-04-22T23:46:22.326 INFO:tasks.workunit.client.1.smithi044.stdout:5/941: creat d15/d5d/db5/dc0/f144 x:0 0 0 2024-04-22T23:46:22.332 INFO:tasks.workunit.client.1.smithi044.stdout:8/894: creat db/d68/d74/d112/f139 x:0 0 0 2024-04-22T23:46:22.333 INFO:tasks.workunit.client.1.smithi044.stdout:3/968: dread d9/db/d15/d25/f6d [0,4194304] 0 2024-04-22T23:46:22.334 INFO:tasks.workunit.client.1.smithi044.stdout:2/713: creat d8/d60/dc1/ffe x:0 0 0 2024-04-22T23:46:22.338 INFO:tasks.workunit.client.1.smithi044.stdout:6/829: fsync d5/d3f/d4c/d4f/f8f 0 2024-04-22T23:46:22.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/780: mkdir d2/d6/d42/df2/d100 0 2024-04-22T23:46:22.343 INFO:tasks.workunit.client.1.smithi044.stdout:8/895: creat db/d52/d9c/dad/db8/f13a x:0 0 0 2024-04-22T23:46:22.346 INFO:tasks.workunit.client.1.smithi044.stdout:5/942: dread d15/f94 [0,4194304] 0 2024-04-22T23:46:22.349 INFO:tasks.workunit.client.1.smithi044.stdout:9/781: dread d2/d6/d42/daa/fae [0,4194304] 0 2024-04-22T23:46:22.350 INFO:tasks.workunit.client.1.smithi044.stdout:2/714: mkdir d8/d19/d5d/db0/dff 0 2024-04-22T23:46:22.351 INFO:tasks.workunit.client.1.smithi044.stdout:2/715: chown d8/d19/d36/d46/l66 10909 1 2024-04-22T23:46:22.354 INFO:tasks.workunit.client.1.smithi044.stdout:4/927: symlink d5/d31/d44/l139 0 2024-04-22T23:46:22.357 INFO:tasks.workunit.client.1.smithi044.stdout:8/896: fsync db/d14/d24/f8d 0 2024-04-22T23:46:22.358 INFO:tasks.workunit.client.1.smithi044.stdout:5/943: rmdir d15/d5d/db5 39 2024-04-22T23:46:22.360 INFO:tasks.workunit.client.1.smithi044.stdout:7/918: rmdir dd/df7/d112/d11c/d11d 0 2024-04-22T23:46:22.363 INFO:tasks.workunit.client.1.smithi044.stdout:4/928: creat d5/d31/d44/dd2/f13a x:0 0 0 2024-04-22T23:46:22.365 INFO:tasks.workunit.client.1.smithi044.stdout:6/830: write d5/dd/d67/f24 [1939383,365] 0 2024-04-22T23:46:22.367 INFO:tasks.workunit.client.1.smithi044.stdout:6/831: truncate d5/d3f/d4c/d92/fea 499533 0 2024-04-22T23:46:22.372 INFO:tasks.workunit.client.1.smithi044.stdout:8/897: chown db/d68/d8b/da7/cc1 1 1 2024-04-22T23:46:22.374 INFO:tasks.workunit.client.1.smithi044.stdout:3/969: creat d9/db/f142 x:0 0 0 2024-04-22T23:46:22.376 INFO:tasks.workunit.client.1.smithi044.stdout:9/782: mkdir d2/dc/d1d/d3f/de3/d101 0 2024-04-22T23:46:22.378 INFO:tasks.workunit.client.1.smithi044.stdout:7/919: unlink dd/d55/d5e/d60/d68/f10a 0 2024-04-22T23:46:22.380 INFO:tasks.workunit.client.1.smithi044.stdout:5/944: dwrite d15/d5d/db5/dc0/d38/d9f/fc4 [0,4194304] 0 2024-04-22T23:46:22.386 INFO:tasks.workunit.client.1.smithi044.stdout:3/970: creat d9/d12/db1/f143 x:0 0 0 2024-04-22T23:46:22.391 INFO:tasks.workunit.client.1.smithi044.stdout:7/920: creat dd/df7/dc0/f149 x:0 0 0 2024-04-22T23:46:22.392 INFO:tasks.workunit.client.1.smithi044.stdout:7/921: readlink dd/d55/dcf/lef 0 2024-04-22T23:46:22.393 INFO:tasks.workunit.client.1.smithi044.stdout:7/922: truncate dd/df7/db8/f10c 4361058 0 2024-04-22T23:46:22.394 INFO:tasks.workunit.client.1.smithi044.stdout:7/923: readlink dd/d98/l9d 0 2024-04-22T23:46:22.395 INFO:tasks.workunit.client.1.smithi044.stdout:8/898: mkdir db/d52/d9c/d13b 0 2024-04-22T23:46:22.396 INFO:tasks.workunit.client.1.smithi044.stdout:7/924: write dd/d1a/db7/dc8/d97/d13c/f146 [189101,123571] 0 2024-04-22T23:46:22.402 INFO:tasks.workunit.client.1.smithi044.stdout:5/945: creat d15/d16/d1d/d22/d2a/d3a/dd2/d13d/f145 x:0 0 0 2024-04-22T23:46:22.403 INFO:tasks.workunit.client.1.smithi044.stdout:7/925: dwrite dd/d114/f147 [0,4194304] 0 2024-04-22T23:46:22.404 INFO:tasks.workunit.client.1.smithi044.stdout:8/899: dwrite db/d68/d74/d112/f11f [0,4194304] 0 2024-04-22T23:46:22.414 INFO:tasks.workunit.client.1.smithi044.stdout:4/929: dwrite d5/d42/fbe [0,4194304] 0 2024-04-22T23:46:22.415 INFO:tasks.workunit.client.1.smithi044.stdout:4/930: write d5/d52/f127 [1013179,14355] 0 2024-04-22T23:46:22.415 INFO:tasks.workunit.client.1.smithi044.stdout:7/926: rmdir dd/d55/d5e/d60/d68/d94/dab/dd5 39 2024-04-22T23:46:22.417 INFO:tasks.workunit.client.1.smithi044.stdout:6/832: truncate d5/d3f/d4c/d92/fa5 3654153 0 2024-04-22T23:46:22.420 INFO:tasks.workunit.client.1.smithi044.stdout:9/783: dwrite d2/dc/d1d/d34/d38/d5c/d69/f96 [0,4194304] 0 2024-04-22T23:46:22.420 INFO:tasks.workunit.client.1.smithi044.stdout:3/971: rename d9/d14/d7d/df8/dfd/d3c to d9/db/d144 0 2024-04-22T23:46:22.421 INFO:tasks.workunit.client.1.smithi044.stdout:9/784: write d2/dc/d1d/d34/f9c [2197254,3235] 0 2024-04-22T23:46:22.424 INFO:tasks.workunit.client.1.smithi044.stdout:7/927: symlink dd/d114/l14a 0 2024-04-22T23:46:22.434 INFO:tasks.workunit.client.1.smithi044.stdout:4/931: dwrite d5/d31/d44/d20/d22/d8d/f10a [0,4194304] 0 2024-04-22T23:46:22.441 INFO:tasks.workunit.client.1.smithi044.stdout:4/932: fdatasync d5/d31/d44/d20/f55 0 2024-04-22T23:46:22.442 INFO:tasks.workunit.client.1.smithi044.stdout:8/900: write db/d14/d72/f7b [431710,76828] 0 2024-04-22T23:46:22.443 INFO:tasks.workunit.client.1.smithi044.stdout:4/933: write d5/d52/f127 [980148,101058] 0 2024-04-22T23:46:22.444 INFO:tasks.workunit.client.1.smithi044.stdout:4/934: read - d5/d31/d44/dd2/f136 zero size 2024-04-22T23:46:22.448 INFO:tasks.workunit.client.1.smithi044.stdout:9/785: write d2/d9/fa5 [696792,108186] 0 2024-04-22T23:46:22.451 INFO:tasks.workunit.client.1.smithi044.stdout:9/786: readlink d2/d6/l21 0 2024-04-22T23:46:22.452 INFO:tasks.workunit.client.1.smithi044.stdout:7/928: write dd/d1a/db7/dc8/f74 [1265891,67015] 0 2024-04-22T23:46:22.454 INFO:tasks.workunit.client.1.smithi044.stdout:3/972: dwrite f5 [4194304,4194304] 0 2024-04-22T23:46:22.458 INFO:tasks.workunit.client.1.smithi044.stdout:9/787: symlink d2/dc/d51/l102 0 2024-04-22T23:46:22.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:22 smithi044 ceph-mon[34463]: pgmap v14: 65 pgs: 65 active+clean; 2.8 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 62 MiB/s rd, 176 MiB/s wr, 410 op/s 2024-04-22T23:46:22.464 INFO:tasks.workunit.client.1.smithi044.stdout:4/935: write d5/d31/d24/fde [2475736,72757] 0 2024-04-22T23:46:22.466 INFO:tasks.workunit.client.1.smithi044.stdout:9/788: dwrite d2/d6/fa [0,4194304] 0 2024-04-22T23:46:22.466 INFO:tasks.workunit.client.1.smithi044.stdout:4/936: stat d5/d31/d44/d20/d22/f67 0 2024-04-22T23:46:22.467 INFO:tasks.workunit.client.1.smithi044.stdout:3/973: rmdir d9/db/d9f/d38 39 2024-04-22T23:46:22.468 INFO:tasks.workunit.client.1.smithi044.stdout:9/789: write d2/dc/d1d/d3f/d9d/fe5 [1771827,126499] 0 2024-04-22T23:46:22.470 INFO:tasks.workunit.client.1.smithi044.stdout:8/901: rename db/f50 to db/d14/dc7/dc9/f13c 0 2024-04-22T23:46:22.476 INFO:tasks.workunit.client.1.smithi044.stdout:4/937: creat d5/d31/d44/dd2/f13b x:0 0 0 2024-04-22T23:46:22.479 INFO:tasks.workunit.client.1.smithi044.stdout:7/929: dread dd/d27/f46 [0,4194304] 0 2024-04-22T23:46:22.481 INFO:tasks.workunit.client.1.smithi044.stdout:7/930: write dd/df7/d9c/f12a [235518,118510] 0 2024-04-22T23:46:22.483 INFO:tasks.workunit.client.1.smithi044.stdout:9/790: rmdir d2/dc/d1d/d34/d38/d5c/d69/d7a/deb 39 2024-04-22T23:46:22.483 INFO:tasks.workunit.client.1.smithi044.stdout:7/931: truncate dd/df7/d9c/f12a 716495 0 2024-04-22T23:46:22.484 INFO:tasks.workunit.client.1.smithi044.stdout:9/791: chown d2/d9/c2b 14 1 2024-04-22T23:46:22.484 INFO:tasks.workunit.client.1.smithi044.stdout:7/932: write dd/df7/d9c/f12a [912311,10344] 0 2024-04-22T23:46:22.486 INFO:tasks.workunit.client.1.smithi044.stdout:9/792: write d2/dc/d1d/d3f/d9d/fe5 [1279749,85014] 0 2024-04-22T23:46:22.492 INFO:tasks.workunit.client.1.smithi044.stdout:4/938: write d5/d31/d24/d38/d6a/d75/fdf [661778,74442] 0 2024-04-22T23:46:22.493 INFO:tasks.workunit.client.1.smithi044.stdout:8/902: mkdir db/d68/d13d 0 2024-04-22T23:46:22.500 INFO:tasks.workunit.client.1.smithi044.stdout:9/793: dread d2/dc/d1d/d34/d38/d5c/d7b/fde [0,4194304] 0 2024-04-22T23:46:22.506 INFO:tasks.workunit.client.1.smithi044.stdout:3/974: getdents d9/db/d15/d25/d5b 0 2024-04-22T23:46:22.512 INFO:tasks.workunit.client.1.smithi044.stdout:8/903: dread db/d68/d8b/da7/fbc [0,4194304] 0 2024-04-22T23:46:22.514 INFO:tasks.workunit.client.1.smithi044.stdout:4/939: rename d5/db/d34/f78 to d5/db/d16/f13c 0 2024-04-22T23:46:22.515 INFO:tasks.workunit.client.1.smithi044.stdout:4/940: chown d5/d31/d44/f7d 383 1 2024-04-22T23:46:22.516 INFO:tasks.workunit.client.1.smithi044.stdout:4/941: fsync d5/d31/d24/d69/f114 0 2024-04-22T23:46:22.516 INFO:tasks.workunit.client.1.smithi044.stdout:9/794: creat d2/dc/d1d/f103 x:0 0 0 2024-04-22T23:46:22.518 INFO:tasks.workunit.client.1.smithi044.stdout:7/933: getdents dd/d55/d5e/d60/d68/d94/dab/dd5 0 2024-04-22T23:46:22.518 INFO:tasks.workunit.client.1.smithi044.stdout:7/934: chown dd/d1a 594463 1 2024-04-22T23:46:22.519 INFO:tasks.workunit.client.1.smithi044.stdout:4/942: chown d5/db/d16/d4d/dad/l10f 10 1 2024-04-22T23:46:22.521 INFO:tasks.workunit.client.1.smithi044.stdout:9/795: symlink d2/d9/l104 0 2024-04-22T23:46:22.521 INFO:tasks.workunit.client.1.smithi044.stdout:8/904: mkdir db/d14/d84/dbb/d134/d13e 0 2024-04-22T23:46:22.523 INFO:tasks.workunit.client.1.smithi044.stdout:4/943: fsync d5/d31/fce 0 2024-04-22T23:46:22.525 INFO:tasks.workunit.client.1.smithi044.stdout:7/935: creat dd/d99/d135/f14b x:0 0 0 2024-04-22T23:46:22.528 INFO:tasks.workunit.client.1.smithi044.stdout:8/905: dread db/d52/d9c/dad/fb3 [0,4194304] 0 2024-04-22T23:46:22.528 INFO:tasks.workunit.client.1.smithi044.stdout:9/796: symlink d2/dc/d1d/d34/d38/d5c/d69/l105 0 2024-04-22T23:46:22.530 INFO:tasks.workunit.client.1.smithi044.stdout:4/944: symlink d5/db/d16/d4d/dad/deb/dfc/l13d 0 2024-04-22T23:46:22.533 INFO:tasks.workunit.client.1.smithi044.stdout:9/797: creat d2/d6/d42/d8b/f106 x:0 0 0 2024-04-22T23:46:22.535 INFO:tasks.workunit.client.1.smithi044.stdout:7/936: creat dd/d55/dcb/d13f/f14c x:0 0 0 2024-04-22T23:46:22.536 INFO:tasks.workunit.client.1.smithi044.stdout:8/906: mkdir db/d14/d72/d13f 0 2024-04-22T23:46:22.538 INFO:tasks.workunit.client.1.smithi044.stdout:4/945: dwrite d5/d31/d24/da6/ff4 [0,4194304] 0 2024-04-22T23:46:22.538 INFO:tasks.workunit.client.1.smithi044.stdout:7/937: fdatasync dd/d1a/db7/dc8/f74 0 2024-04-22T23:46:22.540 INFO:tasks.workunit.client.1.smithi044.stdout:7/938: fsync dd/df7/db8/f109 0 2024-04-22T23:46:22.542 INFO:tasks.workunit.client.1.smithi044.stdout:7/939: chown dd/d98/de8/d12e 0 1 2024-04-22T23:46:22.544 INFO:tasks.workunit.client.1.smithi044.stdout:9/798: rmdir d2/dc/d1d/d34/d8e/dbd/dd1 39 2024-04-22T23:46:22.547 INFO:tasks.workunit.client.1.smithi044.stdout:8/907: dwrite db/d68/d8b/dca/f123 [0,4194304] 0 2024-04-22T23:46:22.548 INFO:tasks.workunit.client.1.smithi044.stdout:8/908: chown db/d14/d84/dc0 0 1 2024-04-22T23:46:22.552 INFO:tasks.workunit.client.1.smithi044.stdout:7/940: rename dd/d1f/f134 to dd/d55/d5e/d60/d106/f14d 0 2024-04-22T23:46:22.554 INFO:tasks.workunit.client.1.smithi044.stdout:7/941: write dd/d55/d5e/d60/ffa [3841178,70939] 0 2024-04-22T23:46:22.555 INFO:tasks.workunit.client.1.smithi044.stdout:9/799: mkdir d2/dc/d1d/d3f/d107 0 2024-04-22T23:46:22.558 INFO:tasks.workunit.client.1.smithi044.stdout:7/942: mknod dd/d55/d5e/d60/d68/d94/d127/c14e 0 2024-04-22T23:46:22.559 INFO:tasks.workunit.client.1.smithi044.stdout:4/946: getdents d5/db/d16 0 2024-04-22T23:46:22.560 INFO:tasks.workunit.client.1.smithi044.stdout:8/909: creat db/d14/d72/f140 x:0 0 0 2024-04-22T23:46:22.564 INFO:tasks.workunit.client.1.smithi044.stdout:7/943: mknod dd/df7/d9c/d113/c14f 0 2024-04-22T23:46:22.567 INFO:tasks.workunit.client.1.smithi044.stdout:4/947: mknod d5/d31/d44/d20/d22/c13e 0 2024-04-22T23:46:22.570 INFO:tasks.workunit.client.1.smithi044.stdout:7/944: mkdir dd/d55/d5e/d60/d68/d94/dbd/d150 0 2024-04-22T23:46:22.571 INFO:tasks.workunit.client.1.smithi044.stdout:4/948: dread d5/fd3 [0,4194304] 0 2024-04-22T23:46:22.576 INFO:tasks.workunit.client.1.smithi044.stdout:8/910: dread db/d14/d24/f5c [0,4194304] 0 2024-04-22T23:46:22.577 INFO:tasks.workunit.client.1.smithi044.stdout:7/945: dwrite dd/d99/dbb/fd0 [0,4194304] 0 2024-04-22T23:46:22.585 INFO:tasks.workunit.client.1.smithi044.stdout:7/946: unlink dd/d55/d5e/d60/d68/d94/dab/dd5/fff 0 2024-04-22T23:46:22.592 INFO:tasks.workunit.client.1.smithi044.stdout:4/949: write d5/d31/d44/d20/d22/fd7 [807961,106879] 0 2024-04-22T23:46:22.594 INFO:tasks.workunit.client.1.smithi044.stdout:8/911: write db/d68/d74/fb0 [466288,25562] 0 2024-04-22T23:46:22.597 INFO:tasks.workunit.client.1.smithi044.stdout:8/912: read - db/d6c/f100 zero size 2024-04-22T23:46:22.598 INFO:tasks.workunit.client.1.smithi044.stdout:7/947: truncate dd/d1a/f7d 6707325 0 2024-04-22T23:46:22.601 INFO:tasks.workunit.client.1.smithi044.stdout:4/950: write d5/d31/d24/d38/d6a/d75/fe1 [1664605,111981] 0 2024-04-22T23:46:22.602 INFO:tasks.workunit.client.1.smithi044.stdout:7/948: stat dd/d1a/c22 0 2024-04-22T23:46:22.605 INFO:tasks.workunit.client.1.smithi044.stdout:8/913: dread db/d14/d24/f51 [0,4194304] 0 2024-04-22T23:46:22.606 INFO:tasks.workunit.client.1.smithi044.stdout:7/949: mknod dd/d55/d5e/d60/d7b/c151 0 2024-04-22T23:46:22.610 INFO:tasks.workunit.client.1.smithi044.stdout:4/951: mknod d5/d52/c13f 0 2024-04-22T23:46:22.611 INFO:tasks.workunit.client.1.smithi044.stdout:8/914: fdatasync db/d14/dc7/dc9/fdc 0 2024-04-22T23:46:22.613 INFO:tasks.workunit.client.1.smithi044.stdout:4/952: truncate d5/d31/d24/d69/d90/d116/f12c 278991 0 2024-04-22T23:46:22.613 INFO:tasks.workunit.client.1.smithi044.stdout:4/953: dread - d5/d31/d24/da6/db6/f115 zero size 2024-04-22T23:46:22.614 INFO:tasks.workunit.client.1.smithi044.stdout:4/954: chown d5/d31/d44/d20/d22/fbf 15 1 2024-04-22T23:46:22.617 INFO:tasks.workunit.client.1.smithi044.stdout:7/950: creat dd/df7/d112/d11c/f152 x:0 0 0 2024-04-22T23:46:22.618 INFO:tasks.workunit.client.1.smithi044.stdout:7/951: fdatasync dd/d1a/db7/dc8/d97/db9/f105 0 2024-04-22T23:46:22.618 INFO:tasks.workunit.client.1.smithi044.stdout:7/952: chown dd/df7/d9c/d113/c14f 310235 1 2024-04-22T23:46:22.630 INFO:tasks.workunit.client.1.smithi044.stdout:4/955: symlink d5/db/d16/d4d/dad/deb/d122/l140 0 2024-04-22T23:46:22.631 INFO:tasks.workunit.client.1.smithi044.stdout:4/956: read d5/d31/d24/d69/d90/d116/f12c [179886,119366] 0 2024-04-22T23:46:22.633 INFO:tasks.workunit.client.1.smithi044.stdout:7/953: link dd/d55/d5e/d60/d75/lf5 dd/d55/d5e/d60/d68/d94/l153 0 2024-04-22T23:46:22.635 INFO:tasks.workunit.client.1.smithi044.stdout:8/915: getdents db/d14/dc7 0 2024-04-22T23:46:22.637 INFO:tasks.workunit.client.1.smithi044.stdout:8/916: readlink db/d14/l86 0 2024-04-22T23:46:22.640 INFO:tasks.workunit.client.1.smithi044.stdout:4/957: dwrite d5/db/d34/f112 [4194304,4194304] 0 2024-04-22T23:46:22.640 INFO:tasks.workunit.client.1.smithi044.stdout:7/954: rename dd/df7/db8/f109 to dd/d1f/f154 0 2024-04-22T23:46:22.644 INFO:tasks.workunit.client.1.smithi044.stdout:7/955: unlink dd/d27/f48 0 2024-04-22T23:46:22.646 INFO:tasks.workunit.client.1.smithi044.stdout:4/958: mkdir d5/d31/d24/d141 0 2024-04-22T23:46:22.651 INFO:tasks.workunit.client.1.smithi044.stdout:4/959: rename d5/db/d34/cb8 to d5/d31/d24/d141/c142 0 2024-04-22T23:46:22.657 INFO:tasks.workunit.client.1.smithi044.stdout:4/960: chown d5/d31/d44/d20/c58 115519876 1 2024-04-22T23:46:22.665 INFO:tasks.workunit.client.1.smithi044.stdout:8/917: dwrite db/d14/d84/fb4 [0,4194304] 0 2024-04-22T23:46:22.665 INFO:tasks.workunit.client.1.smithi044.stdout:7/956: write dd/d114/f11e [250478,10080] 0 2024-04-22T23:46:22.674 INFO:tasks.workunit.client.1.smithi044.stdout:7/957: dwrite dd/d55/d5e/fe4 [0,4194304] 0 2024-04-22T23:46:22.675 INFO:tasks.workunit.client.1.smithi044.stdout:7/958: chown dd/d1f/l5a 59601 1 2024-04-22T23:46:22.676 INFO:tasks.workunit.client.1.smithi044.stdout:4/961: creat d5/d31/d44/f143 x:0 0 0 2024-04-22T23:46:22.681 INFO:tasks.workunit.client.1.smithi044.stdout:8/918: mknod db/d52/d9c/d13b/c141 0 2024-04-22T23:46:22.681 INFO:tasks.workunit.client.1.smithi044.stdout:8/919: write db/ddd/df2/fff [6721232,34525] 0 2024-04-22T23:46:22.685 INFO:tasks.workunit.client.1.smithi044.stdout:4/962: dwrite d5/db/d16/d4d/dad/dc5/f12a [0,4194304] 0 2024-04-22T23:46:22.686 INFO:tasks.workunit.client.1.smithi044.stdout:7/959: link dd/l13e dd/d55/d5e/l155 0 2024-04-22T23:46:22.692 INFO:tasks.workunit.client.1.smithi044.stdout:4/963: mkdir d5/d31/d44/dd2/d144 0 2024-04-22T23:46:22.693 INFO:tasks.workunit.client.1.smithi044.stdout:7/960: read dd/d55/d5e/d60/d75/f92 [361787,12060] 0 2024-04-22T23:46:22.694 INFO:tasks.workunit.client.1.smithi044.stdout:4/964: write d5/d31/d44/dd2/f13b [82514,17599] 0 2024-04-22T23:46:22.697 INFO:tasks.workunit.client.1.smithi044.stdout:7/961: dread - dd/d27/de7/d111/f133 zero size 2024-04-22T23:46:22.699 INFO:tasks.workunit.client.1.smithi044.stdout:8/920: link db/d14/f63 db/d52/f142 0 2024-04-22T23:46:22.700 INFO:tasks.workunit.client.1.smithi044.stdout:8/921: dread - db/d14/d72/f8f zero size 2024-04-22T23:46:22.701 INFO:tasks.workunit.client.1.smithi044.stdout:8/922: dread - db/f130 zero size 2024-04-22T23:46:22.701 INFO:tasks.workunit.client.1.smithi044.stdout:4/965: link d5/db/d34/f124 d5/db/d16/d4d/dad/dc5/f145 0 2024-04-22T23:46:22.706 INFO:tasks.workunit.client.1.smithi044.stdout:4/966: symlink d5/db/d16/d4d/dad/dc5/dd9/l146 0 2024-04-22T23:46:22.707 INFO:tasks.workunit.client.1.smithi044.stdout:4/967: readlink d5/l3f 0 2024-04-22T23:46:22.708 INFO:tasks.workunit.client.1.smithi044.stdout:7/962: getdents dd/d27 0 2024-04-22T23:46:22.710 INFO:tasks.workunit.client.1.smithi044.stdout:4/968: mkdir d5/db/d34/d147 0 2024-04-22T23:46:22.710 INFO:tasks.workunit.client.1.smithi044.stdout:4/969: chown d5/d31/fe2 2 1 2024-04-22T23:46:22.712 INFO:tasks.workunit.client.1.smithi044.stdout:7/963: mkdir dd/d1f/d156 0 2024-04-22T23:46:22.713 INFO:tasks.workunit.client.1.smithi044.stdout:4/970: truncate d5/db/d16/d4d/f132 435208 0 2024-04-22T23:46:22.713 INFO:tasks.workunit.client.1.smithi044.stdout:4/971: chown d5/d31/l133 424838 1 2024-04-22T23:46:22.714 INFO:tasks.workunit.client.1.smithi044.stdout:7/964: chown dd/d1f/c4b 80719081 1 2024-04-22T23:46:22.715 INFO:tasks.workunit.client.1.smithi044.stdout:4/972: read d5/d31/d44/d20/d22/d8d/f97 [2058199,23711] 0 2024-04-22T23:46:22.716 INFO:tasks.workunit.client.1.smithi044.stdout:7/965: write dd/d55/d5e/d60/d7b/f117 [890746,39987] 0 2024-04-22T23:46:22.717 INFO:tasks.workunit.client.1.smithi044.stdout:7/966: chown dd/d1a/d61 54364 1 2024-04-22T23:46:22.718 INFO:tasks.workunit.client.1.smithi044.stdout:8/923: dread db/d14/d72/f9f [0,4194304] 0 2024-04-22T23:46:22.720 INFO:tasks.workunit.client.1.smithi044.stdout:4/973: creat d5/d31/d44/d20/d22/d8d/f148 x:0 0 0 2024-04-22T23:46:22.721 INFO:tasks.workunit.client.1.smithi044.stdout:7/967: rmdir dd/d55/d5e/d60/d68/d94 39 2024-04-22T23:46:22.722 INFO:tasks.workunit.client.1.smithi044.stdout:7/968: fsync dd/d1a/db7/dc8/f8f 0 2024-04-22T23:46:22.722 INFO:tasks.workunit.client.1.smithi044.stdout:8/924: creat db/d14/d84/dbb/f143 x:0 0 0 2024-04-22T23:46:22.729 INFO:tasks.workunit.client.1.smithi044.stdout:8/925: read db/d6c/fcc [3253261,53611] 0 2024-04-22T23:46:22.729 INFO:tasks.workunit.client.1.smithi044.stdout:8/926: chown db/d14/dc7/dc9 4 1 2024-04-22T23:46:22.731 INFO:tasks.workunit.client.1.smithi044.stdout:8/927: truncate db/d52/d9c/dad/db8/f13a 844152 0 2024-04-22T23:46:22.732 INFO:tasks.workunit.client.1.smithi044.stdout:7/969: fsync dd/f59 0 2024-04-22T23:46:22.736 INFO:tasks.workunit.client.1.smithi044.stdout:7/970: unlink dd/d1f/f77 0 2024-04-22T23:46:22.738 INFO:tasks.workunit.client.1.smithi044.stdout:8/928: read - db/d68/daf/fd0 zero size 2024-04-22T23:46:22.738 INFO:tasks.workunit.client.1.smithi044.stdout:7/971: write dd/d1a/db7/dc8/f8f [2742048,29998] 0 2024-04-22T23:46:22.740 INFO:tasks.workunit.client.1.smithi044.stdout:4/974: write d5/d31/d24/d38/fbb [86237,12455] 0 2024-04-22T23:46:22.741 INFO:tasks.workunit.client.1.smithi044.stdout:4/975: chown d5/db/d16/fa0 493549 1 2024-04-22T23:46:22.747 INFO:tasks.workunit.client.1.smithi044.stdout:7/972: mknod dd/d55/d5e/d60/d68/d94/dab/c157 0 2024-04-22T23:46:22.750 INFO:tasks.workunit.client.1.smithi044.stdout:8/929: rmdir db/d14/d72/de2 0 2024-04-22T23:46:22.750 INFO:tasks.workunit.client.1.smithi044.stdout:4/976: write d5/d31/fef [115707,61672] 0 2024-04-22T23:46:22.751 INFO:tasks.workunit.client.1.smithi044.stdout:4/977: chown d5/d31/d44/f143 0 1 2024-04-22T23:46:22.757 INFO:tasks.workunit.client.1.smithi044.stdout:8/930: dwrite db/d68/f77 [0,4194304] 0 2024-04-22T23:46:22.758 INFO:tasks.workunit.client.1.smithi044.stdout:7/973: write dd/d55/d5e/d60/d7b/fce [756381,88266] 0 2024-04-22T23:46:22.758 INFO:tasks.workunit.client.1.smithi044.stdout:8/931: fsync db/ddd/df2/fe1 0 2024-04-22T23:46:22.760 INFO:tasks.workunit.client.1.smithi044.stdout:4/978: rename d5/d31/d44/cd4 to d5/db/d34/c149 0 2024-04-22T23:46:22.764 INFO:tasks.workunit.client.1.smithi044.stdout:4/979: dread d5/db/d3c/ffa [0,4194304] 0 2024-04-22T23:46:22.765 INFO:tasks.workunit.client.1.smithi044.stdout:7/974: unlink dd/d55/d5e/d60/d106/l139 0 2024-04-22T23:46:22.767 INFO:tasks.workunit.client.1.smithi044.stdout:7/975: truncate dd/d1a/db7/fda 423793 0 2024-04-22T23:46:22.771 INFO:tasks.workunit.client.1.smithi044.stdout:4/980: creat d5/d31/d24/d69/d90/d116/f14a x:0 0 0 2024-04-22T23:46:22.775 INFO:tasks.workunit.client.1.smithi044.stdout:8/932: creat db/d14/dc7/dc9/f144 x:0 0 0 2024-04-22T23:46:22.777 INFO:tasks.workunit.client.1.smithi044.stdout:7/976: stat dd/d1a/f7d 0 2024-04-22T23:46:22.779 INFO:tasks.workunit.client.1.smithi044.stdout:8/933: rmdir db/d14/d9e 39 2024-04-22T23:46:22.782 INFO:tasks.workunit.client.1.smithi044.stdout:4/981: mkdir d5/db/d14b 0 2024-04-22T23:46:22.783 INFO:tasks.workunit.client.1.smithi044.stdout:4/982: stat d5/db/d16/d4d/dad/dc5/f12a 0 2024-04-22T23:46:22.785 INFO:tasks.workunit.client.1.smithi044.stdout:8/934: rename db/d52/cb1 to db/d14/d24/d127/c145 0 2024-04-22T23:46:22.787 INFO:tasks.workunit.client.1.smithi044.stdout:7/977: write dd/d1a/db7/dc8/fa8 [1976132,26051] 0 2024-04-22T23:46:22.793 INFO:tasks.workunit.client.1.smithi044.stdout:4/983: creat d5/f14c x:0 0 0 2024-04-22T23:46:22.801 INFO:tasks.workunit.client.1.smithi044.stdout:4/984: dwrite d5/d31/d44/d20/d22/fbf [4194304,4194304] 0 2024-04-22T23:46:22.802 INFO:tasks.workunit.client.1.smithi044.stdout:8/935: dread db/ddd/fe3 [0,4194304] 0 2024-04-22T23:46:22.804 INFO:tasks.workunit.client.1.smithi044.stdout:7/978: dread dd/d1f/d24/f70 [0,4194304] 0 2024-04-22T23:46:22.818 INFO:tasks.workunit.client.1.smithi044.stdout:8/936: dwrite db/d14/d72/f7b [0,4194304] 0 2024-04-22T23:46:22.821 INFO:tasks.workunit.client.1.smithi044.stdout:7/979: rmdir dd/d55/d5e/d60/d68/d94/dab 39 2024-04-22T23:46:22.823 INFO:tasks.workunit.client.1.smithi044.stdout:7/980: fsync dd/d1a/db7/dc8/f8f 0 2024-04-22T23:46:22.824 INFO:tasks.workunit.client.1.smithi044.stdout:7/981: chown dd/d1f/d24/f19 3251754 1 2024-04-22T23:46:22.824 INFO:tasks.workunit.client.1.smithi044.stdout:4/985: unlink l4 0 2024-04-22T23:46:22.827 INFO:tasks.workunit.client.1.smithi044.stdout:8/937: truncate db/d14/f11d 1729263 0 2024-04-22T23:46:22.954 INFO:tasks.workunit.client.1.smithi044.stdout:4/986: symlink d5/d31/l14d 0 2024-04-22T23:46:22.954 INFO:tasks.workunit.client.1.smithi044.stdout:8/938: write db/d14/d72/f95 [5355610,56262] 0 2024-04-22T23:46:22.958 INFO:tasks.workunit.client.1.smithi044.stdout:4/987: fdatasync d5/d31/d24/d38/f91 0 2024-04-22T23:46:22.958 INFO:tasks.workunit.client.1.smithi044.stdout:4/988: dread - d5/f14c zero size 2024-04-22T23:46:22.962 INFO:tasks.workunit.client.1.smithi044.stdout:4/989: symlink d5/db/d16/d4d/dad/dc5/l14e 0 2024-04-22T23:46:22.963 INFO:tasks.workunit.client.1.smithi044.stdout:4/990: chown d5/db/d16/d4d/lba 3676343 1 2024-04-22T23:46:22.963 INFO:tasks.workunit.client.1.smithi044.stdout:8/939: unlink db/d52/d9c/dad/db8/fe8 0 2024-04-22T23:46:22.965 INFO:tasks.workunit.client.1.smithi044.stdout:4/991: creat d5/d31/d24/d141/f14f x:0 0 0 2024-04-22T23:46:22.966 INFO:tasks.workunit.client.1.smithi044.stdout:4/992: write d5/db/d16/fc9 [1335312,63682] 0 2024-04-22T23:46:22.973 INFO:tasks.workunit.client.1.smithi044.stdout:4/993: dwrite d5/d31/d24/d69/f114 [0,4194304] 0 2024-04-22T23:46:22.982 INFO:tasks.workunit.client.1.smithi044.stdout:4/994: dwrite d5/d31/d44/d20/d22/d8d/f11d [0,4194304] 0 2024-04-22T23:46:22.990 INFO:tasks.workunit.client.1.smithi044.stdout:4/995: dwrite d5/d31/d24/da6/ff4 [4194304,4194304] 0 2024-04-22T23:46:23.020 INFO:tasks.workunit.client.1.smithi044.stdout:4/996: dwrite d5/d31/d24/d69/d90/d116/f14a [0,4194304] 0 2024-04-22T23:46:23.095 INFO:tasks.workunit.client.1.smithi044.stdout:4/997: truncate d5/db/d16/d4d/f8a 4064990 0 2024-04-22T23:46:23.097 INFO:tasks.workunit.client.1.smithi044.stdout:4/998: creat d5/d31/d24/d141/f150 x:0 0 0 2024-04-22T23:46:23.098 INFO:tasks.workunit.client.1.smithi044.stdout:4/999: rmdir d5/db/d34 39 2024-04-22T23:46:23.120 INFO:tasks.workunit.client.1.smithi044.stdout:1/911: sync 2024-04-22T23:46:23.122 INFO:tasks.workunit.client.1.smithi044.stdout:1/912: symlink d1/d8/d20/d45/d75/dfb/l13f 0 2024-04-22T23:46:23.125 INFO:tasks.workunit.client.1.smithi044.stdout:1/913: rename d1/d8/d20/d45/d75/df2/d99/f10a to d1/d8/d1c/d7b/d80/d4b/d12c/f140 0 2024-04-22T23:46:23.130 INFO:tasks.workunit.client.1.smithi044.stdout:1/914: dwrite d1/d8/d20/d45/d5d/f86 [0,4194304] 0 2024-04-22T23:46:23.133 INFO:tasks.workunit.client.1.smithi044.stdout:1/915: truncate d1/d8/d20/d47/d92/f13c 93693 0 2024-04-22T23:46:23.144 INFO:tasks.workunit.client.1.smithi044.stdout:1/916: dwrite d1/d8/d20/d47/d4a/f129 [0,4194304] 0 2024-04-22T23:46:23.156 INFO:tasks.workunit.client.1.smithi044.stdout:1/917: dread d1/d8/d20/fea [0,4194304] 0 2024-04-22T23:46:23.157 INFO:tasks.workunit.client.1.smithi044.stdout:1/918: rename d1/d8 to d1/d8/d1c/d74/d141 22 2024-04-22T23:46:23.160 INFO:tasks.workunit.client.1.smithi044.stdout:1/919: creat d1/d8/d1c/d7b/d80/f142 x:0 0 0 2024-04-22T23:46:23.163 INFO:tasks.workunit.client.1.smithi044.stdout:1/920: write d1/d8/d20/d45/d75/dfb/d95/f11d [2129931,61877] 0 2024-04-22T23:46:23.164 INFO:tasks.workunit.client.1.smithi044.stdout:1/921: chown d1/d8/d20/d45/d5d/fee 314 1 2024-04-22T23:46:23.168 INFO:tasks.workunit.client.1.smithi044.stdout:1/922: creat d1/d8/d20/dae/f143 x:0 0 0 2024-04-22T23:46:23.171 INFO:tasks.workunit.client.1.smithi044.stdout:1/923: mkdir d1/d8/d1c/d7b/d80/dfd/d144 0 2024-04-22T23:46:23.172 INFO:tasks.workunit.client.1.smithi044.stdout:1/924: chown d1/d8/d1c/d7b/d80/dfd/dbf/c13a 127046878 1 2024-04-22T23:46:23.176 INFO:tasks.workunit.client.1.smithi044.stdout:1/925: truncate d1/d8/d34/da1/fbd 370525 0 2024-04-22T23:46:23.179 INFO:tasks.workunit.client.1.smithi044.stdout:1/926: creat d1/d8/d20/d45/d75/df2/d72/f145 x:0 0 0 2024-04-22T23:46:23.181 INFO:tasks.workunit.client.1.smithi044.stdout:2/716: sync 2024-04-22T23:46:23.184 INFO:tasks.workunit.client.1.smithi044.stdout:2/717: mknod d8/d19/d36/d46/d63/de0/c100 0 2024-04-22T23:46:23.188 INFO:tasks.workunit.client.1.smithi044.stdout:2/718: write d8/d19/f39 [916299,68566] 0 2024-04-22T23:46:23.189 INFO:tasks.workunit.client.1.smithi044.stdout:1/927: dwrite d1/d8/f1e [0,4194304] 0 2024-04-22T23:46:23.196 INFO:tasks.workunit.client.1.smithi044.stdout:2/719: creat d8/d27/f101 x:0 0 0 2024-04-22T23:46:23.199 INFO:tasks.workunit.client.1.smithi044.stdout:2/720: creat d8/d19/d36/f102 x:0 0 0 2024-04-22T23:46:23.201 INFO:tasks.workunit.client.1.smithi044.stdout:1/928: write d1/d8/d1c/d23/d3e/fcf [406391,86118] 0 2024-04-22T23:46:23.204 INFO:tasks.workunit.client.1.smithi044.stdout:2/721: dread d8/d27/d5b/f9a [0,4194304] 0 2024-04-22T23:46:23.206 INFO:tasks.workunit.client.1.smithi044.stdout:1/929: mknod d1/d8/d20/d47/d4a/c146 0 2024-04-22T23:46:23.208 INFO:tasks.workunit.client.1.smithi044.stdout:2/722: rmdir d8/d19/d1c/daf 39 2024-04-22T23:46:23.213 INFO:tasks.workunit.client.1.smithi044.stdout:2/723: getdents d8/d19/d36/d93 0 2024-04-22T23:46:23.214 INFO:tasks.workunit.client.1.smithi044.stdout:1/930: dwrite d1/d8/d20/d45/d75/df2/fc0 [4194304,4194304] 0 2024-04-22T23:46:23.218 INFO:tasks.workunit.client.1.smithi044.stdout:2/724: rename d8/d27/d5b/d74/d81/fac to d8/d60/de1/f103 0 2024-04-22T23:46:23.219 INFO:tasks.workunit.client.1.smithi044.stdout:1/931: creat d1/d8/d1c/d7b/d80/dfd/dbf/ddb/f147 x:0 0 0 2024-04-22T23:46:23.223 INFO:tasks.workunit.client.1.smithi044.stdout:1/932: rmdir d1/d8/d34/d112 39 2024-04-22T23:46:23.226 INFO:tasks.workunit.client.1.smithi044.stdout:2/725: write d8/d19/d5d/f7e [949395,99187] 0 2024-04-22T23:46:23.227 INFO:tasks.workunit.client.1.smithi044.stdout:1/933: symlink d1/d8/d1c/d23/d3e/l148 0 2024-04-22T23:46:23.231 INFO:tasks.workunit.client.1.smithi044.stdout:2/726: rename d8/d27/d5b/d74/lbd to d8/d19/d36/d93/dae/de8/l104 0 2024-04-22T23:46:23.239 INFO:tasks.workunit.client.1.smithi044.stdout:2/727: write d8/d19/d1c/f1d [5178338,5058] 0 2024-04-22T23:46:23.240 INFO:tasks.workunit.client.1.smithi044.stdout:2/728: readlink d8/d60/dc1/lb3 0 2024-04-22T23:46:23.245 INFO:tasks.workunit.client.1.smithi044.stdout:2/729: rename d8/d27/d5b/l95 to d8/d19/d96/l105 0 2024-04-22T23:46:23.258 INFO:tasks.workunit.client.1.smithi044.stdout:2/730: link d8/d27/d5b/d74/d80/fb2 d8/d19/d1c/daf/dd0/f106 0 2024-04-22T23:46:23.262 INFO:tasks.workunit.client.1.smithi044.stdout:2/731: unlink d8/d19/d36/d44/cc9 0 2024-04-22T23:46:23.263 INFO:tasks.workunit.client.1.smithi044.stdout:2/732: mknod d8/d19/d1c/daf/dd0/c107 0 2024-04-22T23:46:23.264 INFO:tasks.workunit.client.1.smithi044.stdout:2/733: dread d8/d19/fed [0,4194304] 0 2024-04-22T23:46:23.279 INFO:tasks.workunit.client.1.smithi044.stdout:2/734: write d8/d60/f69 [760307,69766] 0 2024-04-22T23:46:23.280 INFO:tasks.workunit.client.1.smithi044.stdout:2/735: chown d8/d19/d96/f9b 3489388 1 2024-04-22T23:46:23.292 INFO:tasks.workunit.client.1.smithi044.stdout:2/736: dread d8/f43 [0,4194304] 0 2024-04-22T23:46:23.295 INFO:tasks.workunit.client.1.smithi044.stdout:2/737: mkdir d8/d19/d36/d93/dae/d108 0 2024-04-22T23:46:23.310 INFO:tasks.workunit.client.1.smithi044.stdout:2/738: dwrite d8/d19/d5d/f7c [0,4194304] 0 2024-04-22T23:46:23.319 INFO:tasks.workunit.client.1.smithi044.stdout:2/739: creat d8/d19/d36/d44/f109 x:0 0 0 2024-04-22T23:46:23.325 INFO:tasks.workunit.client.1.smithi044.stdout:2/740: creat d8/d19/f10a x:0 0 0 2024-04-22T23:46:23.327 INFO:tasks.workunit.client.1.smithi044.stdout:2/741: stat d8/caa 0 2024-04-22T23:46:23.329 INFO:tasks.workunit.client.1.smithi044.stdout:2/742: fdatasync d8/d19/d36/fa5 0 2024-04-22T23:46:23.331 INFO:tasks.workunit.client.1.smithi044.stdout:2/743: creat d8/d19/d36/d93/dae/d108/f10b x:0 0 0 2024-04-22T23:46:23.333 INFO:tasks.workunit.client.1.smithi044.stdout:2/744: fsync d8/d19/f1f 0 2024-04-22T23:46:23.333 INFO:tasks.workunit.client.1.smithi044.stdout:2/745: stat d8/d19/d1c/l21 0 2024-04-22T23:46:23.336 INFO:tasks.workunit.client.1.smithi044.stdout:2/746: truncate d8/d19/d36/d93/dae/de8/feb 906523 0 2024-04-22T23:46:23.337 INFO:tasks.workunit.client.1.smithi044.stdout:2/747: dread - d8/d60/de1/f103 zero size 2024-04-22T23:46:23.340 INFO:tasks.workunit.client.1.smithi044.stdout:2/748: mkdir d8/d19/d36/d46/df0/d10c 0 2024-04-22T23:46:23.342 INFO:tasks.workunit.client.1.smithi044.stdout:2/749: creat d8/d19/d96/f10d x:0 0 0 2024-04-22T23:46:23.342 INFO:tasks.workunit.client.1.smithi044.stdout:2/750: readlink d8/lc2 0 2024-04-22T23:46:23.343 INFO:tasks.workunit.client.1.smithi044.stdout:2/751: chown d8/d27/d5b/f7f 487 1 2024-04-22T23:46:23.350 INFO:tasks.workunit.client.1.smithi044.stdout:2/752: rmdir d8/d19/d36/d46/df0/d10c 0 2024-04-22T23:46:23.373 INFO:tasks.workunit.client.1.smithi044.stdout:2/753: write d8/d27/f4f [1030029,58675] 0 2024-04-22T23:46:23.377 INFO:tasks.workunit.client.1.smithi044.stdout:6/833: sync 2024-04-22T23:46:23.377 INFO:tasks.workunit.client.1.smithi044.stdout:5/946: sync 2024-04-22T23:46:23.377 INFO:tasks.workunit.client.1.smithi044.stdout:2/754: link d8/cda d8/d19/d5d/db0/c10e 0 2024-04-22T23:46:23.378 INFO:tasks.workunit.client.1.smithi044.stdout:6/834: fsync d5/d3f/d50/d79/d96/ff5 0 2024-04-22T23:46:23.382 INFO:tasks.workunit.client.1.smithi044.stdout:2/755: mknod d8/d19/d36/d46/ddb/ddd/df5/c10f 0 2024-04-22T23:46:23.387 INFO:tasks.workunit.client.1.smithi044.stdout:2/756: rename d8/d19/d1c to d8/d27/d5b/d110 0 2024-04-22T23:46:23.388 INFO:tasks.workunit.client.1.smithi044.stdout:5/947: dwrite d15/d16/d1d/d22/fe2 [0,4194304] 0 2024-04-22T23:46:23.392 INFO:tasks.workunit.client.1.smithi044.stdout:2/757: mkdir d8/d27/d5b/d111 0 2024-04-22T23:46:23.392 INFO:tasks.workunit.client.1.smithi044.stdout:6/835: link d5/f6b d5/d3f/d4c/d23/d3d/f107 0 2024-04-22T23:46:23.393 INFO:tasks.workunit.client.1.smithi044.stdout:6/836: chown d5/dd/d28/fe7 366 1 2024-04-22T23:46:23.395 INFO:tasks.workunit.client.1.smithi044.stdout:5/948: stat d15/d5d/db5/dc0/f3b 0 2024-04-22T23:46:23.396 INFO:tasks.workunit.client.1.smithi044.stdout:2/758: creat d8/d19/d36/d93/dae/de8/f112 x:0 0 0 2024-04-22T23:46:23.397 INFO:tasks.workunit.client.1.smithi044.stdout:2/759: rename d8 to d8/d27/d5b/d110/daf/dd0/d113 22 2024-04-22T23:46:23.399 INFO:tasks.workunit.client.1.smithi044.stdout:6/837: mknod d5/dd/d28/d75/c108 0 2024-04-22T23:46:23.400 INFO:tasks.workunit.client.1.smithi044.stdout:5/949: fsync d15/d16/d1d/d22/d4f/fe1 0 2024-04-22T23:46:23.402 INFO:tasks.workunit.client.1.smithi044.stdout:2/760: symlink d8/d60/dc1/l114 0 2024-04-22T23:46:23.403 INFO:tasks.workunit.client.1.smithi044.stdout:6/838: mkdir d5/d3f/d4c/d92/d109 0 2024-04-22T23:46:23.404 INFO:tasks.workunit.client.1.smithi044.stdout:6/839: fdatasync d5/dd/f1e 0 2024-04-22T23:46:23.405 INFO:tasks.workunit.client.1.smithi044.stdout:5/950: creat d15/d16/dbc/d116/d115/d117/f146 x:0 0 0 2024-04-22T23:46:23.406 INFO:tasks.workunit.client.1.smithi044.stdout:2/761: creat d8/d19/d36/d46/df0/f115 x:0 0 0 2024-04-22T23:46:23.407 INFO:tasks.workunit.client.1.smithi044.stdout:2/762: stat d8/d19/c6e 0 2024-04-22T23:46:23.409 INFO:tasks.workunit.client.1.smithi044.stdout:5/951: mkdir d15/d5d/d64/d6f/d7f/d147 0 2024-04-22T23:46:23.410 INFO:tasks.workunit.client.1.smithi044.stdout:5/952: chown d15/d16/f100 2124 1 2024-04-22T23:46:23.411 INFO:tasks.workunit.client.1.smithi044.stdout:2/763: unlink d8/d60/fd1 0 2024-04-22T23:46:23.412 INFO:tasks.workunit.client.1.smithi044.stdout:2/764: write d8/d27/d5b/d110/daf/dd0/fee [647584,11390] 0 2024-04-22T23:46:23.414 INFO:tasks.workunit.client.1.smithi044.stdout:5/953: fsync d15/d16/d1d/d22/d2a/d3a/d43/f58 0 2024-04-22T23:46:23.420 INFO:tasks.workunit.client.1.smithi044.stdout:5/954: dread d15/f30 [4194304,4194304] 0 2024-04-22T23:46:23.423 INFO:tasks.workunit.client.1.smithi044.stdout:5/955: unlink d15/d5d/d82/f83 0 2024-04-22T23:46:23.428 INFO:tasks.workunit.client.1.smithi044.stdout:9/800: sync 2024-04-22T23:46:23.428 INFO:tasks.workunit.client.1.smithi044.stdout:3/975: sync 2024-04-22T23:46:23.430 INFO:tasks.workunit.client.1.smithi044.stdout:3/976: read d9/d14/dd2/f11b [61960,50588] 0 2024-04-22T23:46:23.431 INFO:tasks.workunit.client.1.smithi044.stdout:2/765: rename d8/d60/dc1 to d8/d19/d36/d46/d63/d116 0 2024-04-22T23:46:23.431 INFO:tasks.workunit.client.1.smithi044.stdout:3/977: chown d9/db/d33/l53 785921 1 2024-04-22T23:46:23.436 INFO:tasks.workunit.client.1.smithi044.stdout:9/801: link d2/d6/d4d/f5e d2/d6/f108 0 2024-04-22T23:46:23.437 INFO:tasks.workunit.client.1.smithi044.stdout:5/956: dread d15/d5d/db5/dc0/d38/f136 [0,4194304] 0 2024-04-22T23:46:23.438 INFO:tasks.workunit.client.1.smithi044.stdout:5/957: dread - d15/d3c/df9/f11e zero size 2024-04-22T23:46:23.438 INFO:tasks.workunit.client.1.smithi044.stdout:2/766: mkdir d8/d19/d117 0 2024-04-22T23:46:23.440 INFO:tasks.workunit.client.1.smithi044.stdout:9/802: creat d2/d6/d42/d8b/ddc/f109 x:0 0 0 2024-04-22T23:46:23.441 INFO:tasks.workunit.client.1.smithi044.stdout:5/958: dread d15/d16/d1d/d22/d2a/d3a/d43/fa8 [0,4194304] 0 2024-04-22T23:46:23.443 INFO:tasks.workunit.client.1.smithi044.stdout:6/840: truncate d5/d3f/d50/d79/f8c 1942470 0 2024-04-22T23:46:23.444 INFO:tasks.workunit.client.1.smithi044.stdout:2/767: truncate d8/d27/d5b/f7f 2517708 0 2024-04-22T23:46:23.447 INFO:tasks.workunit.client.1.smithi044.stdout:9/803: creat d2/dc/d1d/d34/d38/d5c/f10a x:0 0 0 2024-04-22T23:46:23.448 INFO:tasks.workunit.client.1.smithi044.stdout:9/804: write d2/d6/fd7 [222941,31932] 0 2024-04-22T23:46:23.459 INFO:tasks.workunit.client.1.smithi044.stdout:3/978: getdents d9/db/d144/d95 0 2024-04-22T23:46:23.463 INFO:tasks.workunit.client.1.smithi044.stdout:6/841: dread d5/dd/d67/f20 [0,4194304] 0 2024-04-22T23:46:23.466 INFO:tasks.workunit.client.1.smithi044.stdout:3/979: truncate d9/db/d15/d2e/d48/f73 708396 0 2024-04-22T23:46:23.467 INFO:tasks.workunit.client.1.smithi044.stdout:3/980: read d9/d14/f2b [88656,73277] 0 2024-04-22T23:46:23.469 INFO:tasks.workunit.client.1.smithi044.stdout:9/805: creat d2/dc/d1d/d34/d38/d5c/d69/d7a/f10b x:0 0 0 2024-04-22T23:46:23.469 INFO:tasks.workunit.client.1.smithi044.stdout:3/981: stat d9/dc8/ff4 0 2024-04-22T23:46:23.469 INFO:tasks.workunit.client.1.smithi044.stdout:6/842: truncate d5/d3f/d50/d79/d96/fa1 400246 0 2024-04-22T23:46:23.472 INFO:tasks.workunit.client.1.smithi044.stdout:5/959: dwrite d15/d16/d1d/d22/d2a/f79 [4194304,4194304] 0 2024-04-22T23:46:23.475 INFO:tasks.workunit.client.1.smithi044.stdout:5/960: write d15/d16/f100 [479897,85781] 0 2024-04-22T23:46:23.502 INFO:tasks.workunit.client.1.smithi044.stdout:6/843: creat d5/dd/d28/f10a x:0 0 0 2024-04-22T23:46:23.503 INFO:tasks.workunit.client.1.smithi044.stdout:2/768: dwrite d8/d19/d36/d46/d63/d65/d6c/fbe [0,4194304] 0 2024-04-22T23:46:23.507 INFO:tasks.workunit.client.1.smithi044.stdout:2/769: chown d8/d19/d36/d46/ddb/l58 94 1 2024-04-22T23:46:23.509 INFO:tasks.workunit.client.1.smithi044.stdout:5/961: unlink d15/d5d/db5/dc0/f3b 0 2024-04-22T23:46:23.509 INFO:tasks.workunit.client.1.smithi044.stdout:2/770: stat d8/d19/d36/d46/d63/d65/d6c 0 2024-04-22T23:46:23.511 INFO:tasks.workunit.client.1.smithi044.stdout:6/844: mknod d5/d3f/d4c/d92/c10b 0 2024-04-22T23:46:23.512 INFO:tasks.workunit.client.1.smithi044.stdout:9/806: dwrite d2/dc/d1d/d34/d38/d5c/d69/d7a/f6f [0,4194304] 0 2024-04-22T23:46:23.514 INFO:tasks.workunit.client.1.smithi044.stdout:2/771: unlink d8/d27/d5b/d110/l6d 0 2024-04-22T23:46:23.514 INFO:tasks.workunit.client.1.smithi044.stdout:5/962: readlink d15/d16/d1d/d22/d4f/d68/d76/lf3 0 2024-04-22T23:46:23.518 INFO:tasks.workunit.client.1.smithi044.stdout:6/845: symlink d5/d3f/d4c/d4f/l10c 0 2024-04-22T23:46:23.521 INFO:tasks.workunit.client.1.smithi044.stdout:5/963: creat d15/d3c/df9/d133/f148 x:0 0 0 2024-04-22T23:46:23.526 INFO:tasks.workunit.client.1.smithi044.stdout:6/846: symlink d5/dd/d28/d75/d88/dcd/de3/l10d 0 2024-04-22T23:46:23.528 INFO:tasks.workunit.client.1.smithi044.stdout:2/772: link d8/c13 d8/d19/d36/d46/d63/d116/c118 0 2024-04-22T23:46:23.532 INFO:tasks.workunit.client.1.smithi044.stdout:5/964: rename d15/d5d/fb1 to d15/d16/d1d/d22/d2a/f149 0 2024-04-22T23:46:23.534 INFO:tasks.workunit.client.1.smithi044.stdout:6/847: creat d5/d3f/d4c/df1/dfb/f10e x:0 0 0 2024-04-22T23:46:23.538 INFO:tasks.workunit.client.1.smithi044.stdout:5/965: mknod d15/d5d/d64/de0/c14a 0 2024-04-22T23:46:23.540 INFO:tasks.workunit.client.1.smithi044.stdout:2/773: link d8/d19/d36/d46/d63/d65/d6c/fbe d8/d19/d36/d46/ddb/f119 0 2024-04-22T23:46:23.541 INFO:tasks.workunit.client.1.smithi044.stdout:6/848: chown d5/d3f/f63 1 1 2024-04-22T23:46:23.547 INFO:tasks.workunit.client.1.smithi044.stdout:9/807: dwrite d2/dc/d1d/d3f/f48 [4194304,4194304] 0 2024-04-22T23:46:23.549 INFO:tasks.workunit.client.1.smithi044.stdout:6/849: dwrite f0 [4194304,4194304] 0 2024-04-22T23:46:23.551 INFO:tasks.workunit.client.1.smithi044.stdout:6/850: stat d5/dd/f7d 0 2024-04-22T23:46:23.553 INFO:tasks.workunit.client.1.smithi044.stdout:9/808: chown d2/dc/d1d/d34/d38/d5c/d69/d75/f7c 11 1 2024-04-22T23:46:23.553 INFO:tasks.workunit.client.1.smithi044.stdout:6/851: stat d5/dd/d28/d75/d88 0 2024-04-22T23:46:23.555 INFO:tasks.workunit.client.1.smithi044.stdout:6/852: chown d5/d3f/d4c/db4 3 1 2024-04-22T23:46:23.555 INFO:tasks.workunit.client.1.smithi044.stdout:9/809: fsync d2/dc/d1d/d34/d38/d5c/d69/d75/fef 0 2024-04-22T23:46:23.556 INFO:tasks.workunit.client.1.smithi044.stdout:6/853: stat d5/d30/d73/fc1 0 2024-04-22T23:46:23.558 INFO:tasks.workunit.client.1.smithi044.stdout:2/774: fsync d8/d19/d96/fc6 0 2024-04-22T23:46:23.560 INFO:tasks.workunit.client.1.smithi044.stdout:9/810: rmdir d2/dc/d1d/d34/d8e/dbd 39 2024-04-22T23:46:23.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/775: symlink d8/d60/de1/l11a 0 2024-04-22T23:46:23.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/776: stat d8/d19/f25 0 2024-04-22T23:46:23.569 INFO:tasks.workunit.client.1.smithi044.stdout:2/777: write d8/f11 [1485044,71021] 0 2024-04-22T23:46:23.571 INFO:tasks.workunit.client.1.smithi044.stdout:5/966: chown d15/d16/d1d/d22/d2a/d3a/d43/d11d/l137 5778691 1 2024-04-22T23:46:23.573 INFO:tasks.workunit.client.1.smithi044.stdout:9/811: fdatasync d2/dc/d1d/d34/f92 0 2024-04-22T23:46:23.577 INFO:tasks.workunit.client.1.smithi044.stdout:9/812: rmdir d2/dc/d1d/d34/d8e 39 2024-04-22T23:46:23.582 INFO:tasks.workunit.client.1.smithi044.stdout:5/967: mknod d15/d16/d1d/d22/d4f/d68/c14b 0 2024-04-22T23:46:23.583 INFO:tasks.workunit.client.1.smithi044.stdout:2/778: write d8/d27/f6f [714001,21097] 0 2024-04-22T23:46:23.584 INFO:tasks.workunit.client.1.smithi044.stdout:6/854: dread d5/dd/d28/f2c [0,4194304] 0 2024-04-22T23:46:23.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/779: fdatasync d8/d19/d36/d93/dbf/fd5 0 2024-04-22T23:46:23.586 INFO:tasks.workunit.client.1.smithi044.stdout:6/855: read d5/dd/d28/f2e [7545,119387] 0 2024-04-22T23:46:23.588 INFO:tasks.workunit.client.1.smithi044.stdout:9/813: symlink d2/dc/d94/l10c 0 2024-04-22T23:46:23.592 INFO:tasks.workunit.client.1.smithi044.stdout:2/780: mkdir d8/d19/d36/d46/d11b 0 2024-04-22T23:46:23.593 INFO:tasks.workunit.client.1.smithi044.stdout:6/856: dwrite d5/f90 [0,4194304] 0 2024-04-22T23:46:23.602 INFO:tasks.workunit.client.1.smithi044.stdout:9/814: creat d2/dc/d1d/d34/d38/d5c/d69/d9f/f10d x:0 0 0 2024-04-22T23:46:23.611 INFO:tasks.workunit.client.1.smithi044.stdout:5/968: creat d15/d5d/d64/f14c x:0 0 0 2024-04-22T23:46:23.615 INFO:tasks.workunit.client.1.smithi044.stdout:5/969: creat d15/d16/dbc/d116/df8/f14d x:0 0 0 2024-04-22T23:46:23.617 INFO:tasks.workunit.client.1.smithi044.stdout:5/970: dread - d15/d16/d1d/d22/d4f/fe1 zero size 2024-04-22T23:46:23.617 INFO:tasks.workunit.client.1.smithi044.stdout:2/781: dwrite d8/f43 [0,4194304] 0 2024-04-22T23:46:23.619 INFO:tasks.workunit.client.1.smithi044.stdout:1/934: sync 2024-04-22T23:46:23.619 INFO:tasks.workunit.client.1.smithi044.stdout:7/982: sync 2024-04-22T23:46:23.619 INFO:tasks.workunit.client.1.smithi044.stdout:8/940: sync 2024-04-22T23:46:23.619 INFO:tasks.workunit.client.1.smithi044.stdout:9/815: rename d2/dc/f87 to d2/dc/d1d/d34/d8e/dbd/f10e 0 2024-04-22T23:46:23.624 INFO:tasks.workunit.client.1.smithi044.stdout:2/782: read d8/d27/d5b/d110/daf/dd0/f106 [325767,25671] 0 2024-04-22T23:46:23.625 INFO:tasks.workunit.client.1.smithi044.stdout:2/783: chown d8/d19/d36/fa5 74 1 2024-04-22T23:46:23.625 INFO:tasks.workunit.client.1.smithi044.stdout:8/941: mknod db/d68/daf/c146 0 2024-04-22T23:46:23.628 INFO:tasks.workunit.client.1.smithi044.stdout:7/983: unlink dd/df7/d9c/d113/c14f 0 2024-04-22T23:46:23.631 INFO:tasks.workunit.client.1.smithi044.stdout:1/935: symlink d1/d8/d20/d45/d5d/l149 0 2024-04-22T23:46:23.636 INFO:tasks.workunit.client.1.smithi044.stdout:7/984: truncate dd/df7/d9c/faf 353595 0 2024-04-22T23:46:23.638 INFO:tasks.workunit.client.1.smithi044.stdout:1/936: read d1/d8/d1c/f5b [1376389,23860] 0 2024-04-22T23:46:23.639 INFO:tasks.workunit.client.1.smithi044.stdout:1/937: chown d1/d8/d20/d47 47 1 2024-04-22T23:46:23.639 INFO:tasks.workunit.client.1.smithi044.stdout:9/816: rename d2/d9/c2b to d2/dc/d1d/d34/c10f 0 2024-04-22T23:46:23.640 INFO:tasks.workunit.client.1.smithi044.stdout:5/971: dwrite f14 [0,4194304] 0 2024-04-22T23:46:23.643 INFO:tasks.workunit.client.1.smithi044.stdout:7/985: readlink dd/d55/d5e/d60/d68/d94/l153 0 2024-04-22T23:46:23.651 INFO:tasks.workunit.client.1.smithi044.stdout:7/986: dwrite dd/d55/d5e/d60/d7b/f117 [0,4194304] 0 2024-04-22T23:46:23.652 INFO:tasks.workunit.client.1.smithi044.stdout:1/938: write d1/d8/d1c/f8c [365060,68394] 0 2024-04-22T23:46:23.655 INFO:tasks.workunit.client.1.smithi044.stdout:2/784: getdents d8/d19/d36/d93/dae 0 2024-04-22T23:46:23.656 INFO:tasks.workunit.client.1.smithi044.stdout:9/817: truncate d2/dc/d1d/d3f/fd9 603385 0 2024-04-22T23:46:23.660 INFO:tasks.workunit.client.1.smithi044.stdout:8/942: dwrite db/d52/f142 [0,4194304] 0 2024-04-22T23:46:23.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/818: dread - d2/dc/d1d/d34/d38/d5c/f10a zero size 2024-04-22T23:46:23.663 INFO:tasks.workunit.client.1.smithi044.stdout:5/972: creat d15/d16/d1d/d22/d2a/f14e x:0 0 0 2024-04-22T23:46:23.664 INFO:tasks.workunit.client.1.smithi044.stdout:1/939: mknod d1/d8/d20/d45/c14a 0 2024-04-22T23:46:23.665 INFO:tasks.workunit.client.1.smithi044.stdout:2/785: creat d8/d19/d117/f11c x:0 0 0 2024-04-22T23:46:23.667 INFO:tasks.workunit.client.1.smithi044.stdout:9/819: getdents d2/d6/d42/df2/d100 0 2024-04-22T23:46:23.669 INFO:tasks.workunit.client.1.smithi044.stdout:5/973: creat d15/d5d/db5/dc0/d38/d9f/f14f x:0 0 0 2024-04-22T23:46:23.670 INFO:tasks.workunit.client.1.smithi044.stdout:9/820: write d2/dc/d1d/d34/d38/d5c/d69/fac [3905849,126433] 0 2024-04-22T23:46:23.671 INFO:tasks.workunit.client.1.smithi044.stdout:1/940: rmdir d1/d8/d1c/d10d 39 2024-04-22T23:46:23.673 INFO:tasks.workunit.client.1.smithi044.stdout:1/941: chown d1/d8/d20/d45/c14a 64462133 1 2024-04-22T23:46:23.674 INFO:tasks.workunit.client.1.smithi044.stdout:1/942: dread - d1/d8/d1c/d74/f8e zero size 2024-04-22T23:46:23.675 INFO:tasks.workunit.client.1.smithi044.stdout:2/786: creat d8/d19/d36/d46/d63/d65/d94/f11d x:0 0 0 2024-04-22T23:46:23.679 INFO:tasks.workunit.client.1.smithi044.stdout:9/821: unlink d2/dc/d1d/d3f/d9d/fe5 0 2024-04-22T23:46:23.682 INFO:tasks.workunit.client.1.smithi044.stdout:5/974: truncate d15/d16/d1d/d22/d2a/d3a/d43/f9b 519468 0 2024-04-22T23:46:23.686 INFO:tasks.workunit.client.1.smithi044.stdout:2/787: fdatasync d8/d19/d36/d46/ddb/fca 0 2024-04-22T23:46:23.691 INFO:tasks.workunit.client.1.smithi044.stdout:5/975: dwrite d15/d16/d1d/d22/d2a/f79 [4194304,4194304] 0 2024-04-22T23:46:23.693 INFO:tasks.workunit.client.1.smithi044.stdout:8/943: dwrite db/f34 [0,4194304] 0 2024-04-22T23:46:23.694 INFO:tasks.workunit.client.1.smithi044.stdout:8/944: chown db/d14/d84/dbb/d134/d13e 56 1 2024-04-22T23:46:23.699 INFO:tasks.workunit.client.1.smithi044.stdout:1/943: fdatasync d1/d8/d20/d47/d92/fba 0 2024-04-22T23:46:23.704 INFO:tasks.workunit.client.1.smithi044.stdout:9/822: fsync d2/dc/d1d/d3f/f63 0 2024-04-22T23:46:23.723 INFO:tasks.workunit.client.1.smithi044.stdout:1/944: truncate d1/d8/d1c/d23/f5f 4762386 0 2024-04-22T23:46:23.726 INFO:tasks.workunit.client.1.smithi044.stdout:2/788: write d8/d27/f4b [4251216,21309] 0 2024-04-22T23:46:23.727 INFO:tasks.workunit.client.1.smithi044.stdout:8/945: dwrite db/d14/d72/f9f [0,4194304] 0 2024-04-22T23:46:23.729 INFO:tasks.workunit.client.1.smithi044.stdout:2/789: write d8/d19/d96/f10d [976596,48503] 0 2024-04-22T23:46:23.732 INFO:tasks.workunit.client.1.smithi044.stdout:2/790: write d8/d19/d36/d93/dae/d108/f10b [715095,114640] 0 2024-04-22T23:46:23.733 INFO:tasks.workunit.client.1.smithi044.stdout:1/945: truncate d1/d8/d20/dae/fe2 3613328 0 2024-04-22T23:46:23.737 INFO:tasks.workunit.client.1.smithi044.stdout:5/976: truncate d15/d3c/f53 4936988 0 2024-04-22T23:46:23.739 INFO:tasks.workunit.client.1.smithi044.stdout:8/946: mkdir db/d68/d8b/ded/d147 0 2024-04-22T23:46:23.740 INFO:tasks.workunit.client.1.smithi044.stdout:9/823: dwrite d2/dc/d1d/d34/d38/d4a/d72/f8f [0,4194304] 0 2024-04-22T23:46:23.741 INFO:tasks.workunit.client.1.smithi044.stdout:9/824: chown d2/d6/d42/l54 3717 1 2024-04-22T23:46:23.744 INFO:tasks.workunit.client.1.smithi044.stdout:2/791: rename d8/d60/de1 to d8/d19/d36/d93/dae/d11e 0 2024-04-22T23:46:23.747 INFO:tasks.workunit.client.1.smithi044.stdout:8/947: creat db/ddd/df2/f148 x:0 0 0 2024-04-22T23:46:23.755 INFO:tasks.workunit.client.1.smithi044.stdout:9/825: rename d2/dc/d1d/d34/d38/l56 to d2/d6/d42/daa/l110 0 2024-04-22T23:46:23.756 INFO:tasks.workunit.client.1.smithi044.stdout:8/948: dwrite db/d14/d24/f59 [4194304,4194304] 0 2024-04-22T23:46:23.758 INFO:tasks.workunit.client.1.smithi044.stdout:8/949: chown db/d14/d72/f95 33428107 1 2024-04-22T23:46:23.759 INFO:tasks.workunit.client.1.smithi044.stdout:8/950: stat db/d6c/fa0 0 2024-04-22T23:46:23.761 INFO:tasks.workunit.client.1.smithi044.stdout:8/951: write db/d68/d74/ffb [4807431,86436] 0 2024-04-22T23:46:23.772 INFO:tasks.workunit.client.1.smithi044.stdout:9/826: mkdir d2/dc/d1d/d34/d38/d4a/d72/d111 0 2024-04-22T23:46:23.773 INFO:tasks.workunit.client.1.smithi044.stdout:8/952: dread db/d14/f2b [0,4194304] 0 2024-04-22T23:46:23.773 INFO:tasks.workunit.client.1.smithi044.stdout:1/946: write d1/d8/d20/d45/d75/df2/d99/fc4 [2609307,9036] 0 2024-04-22T23:46:23.775 INFO:tasks.workunit.client.1.smithi044.stdout:9/827: readlink d2/dc/d51/l57 0 2024-04-22T23:46:23.776 INFO:tasks.workunit.client.1.smithi044.stdout:2/792: dwrite d8/f92 [0,4194304] 0 2024-04-22T23:46:23.784 INFO:tasks.workunit.client.1.smithi044.stdout:5/977: getdents d15/d5d/d64/d6f/d7f 0 2024-04-22T23:46:23.785 INFO:tasks.workunit.client.1.smithi044.stdout:5/978: write d15/d3c/df9/f11e [42841,121696] 0 2024-04-22T23:46:23.787 INFO:tasks.workunit.client.1.smithi044.stdout:1/947: symlink d1/d8/d20/dae/l14b 0 2024-04-22T23:46:23.789 INFO:tasks.workunit.client.1.smithi044.stdout:9/828: creat d2/dc/d1d/d3f/d9d/f112 x:0 0 0 2024-04-22T23:46:23.792 INFO:tasks.workunit.client.1.smithi044.stdout:5/979: mknod d15/d16/d1d/d22/d2a/d3a/d43/d80/c150 0 2024-04-22T23:46:23.798 INFO:tasks.workunit.client.1.smithi044.stdout:1/948: rmdir d1/d8/d1c/d7b/d80/dfd/dbf 39 2024-04-22T23:46:23.800 INFO:tasks.workunit.client.1.smithi044.stdout:2/793: symlink d8/d19/d5d/db0/dff/l11f 0 2024-04-22T23:46:23.802 INFO:tasks.workunit.client.1.smithi044.stdout:9/829: creat d2/dc/d1d/d3f/d9d/f113 x:0 0 0 2024-04-22T23:46:23.805 INFO:tasks.workunit.client.1.smithi044.stdout:5/980: symlink d15/d5d/d82/l151 0 2024-04-22T23:46:23.805 INFO:tasks.workunit.client.1.smithi044.stdout:3/982: sync 2024-04-22T23:46:23.808 INFO:tasks.workunit.client.1.smithi044.stdout:8/953: dwrite db/d68/f12f [4194304,4194304] 0 2024-04-22T23:46:23.810 INFO:tasks.workunit.client.1.smithi044.stdout:5/981: rmdir d15/d16 39 2024-04-22T23:46:23.811 INFO:tasks.workunit.client.1.smithi044.stdout:5/982: readlink d15/d5d/d64/dc5/l101 0 2024-04-22T23:46:23.815 INFO:tasks.workunit.client.1.smithi044.stdout:1/949: truncate d1/d8/d34/da1/dab/de9/db9/fca 806832 0 2024-04-22T23:46:23.818 INFO:tasks.workunit.client.1.smithi044.stdout:2/794: truncate d8/d19/f30 25364 0 2024-04-22T23:46:23.819 INFO:tasks.workunit.client.1.smithi044.stdout:8/954: mknod db/d68/d8b/dca/d10f/c149 0 2024-04-22T23:46:23.820 INFO:tasks.workunit.client.1.smithi044.stdout:3/983: creat d9/db/d141/f145 x:0 0 0 2024-04-22T23:46:23.822 INFO:tasks.workunit.client.1.smithi044.stdout:9/830: rename d2/d9/l60 to d2/dc/l114 0 2024-04-22T23:46:23.824 INFO:tasks.workunit.client.1.smithi044.stdout:3/984: write d9/db/d144/f13f [2996971,13532] 0 2024-04-22T23:46:23.824 INFO:tasks.workunit.client.1.smithi044.stdout:5/983: fdatasync d15/d16/dbc/d116/feb 0 2024-04-22T23:46:23.826 INFO:tasks.workunit.client.1.smithi044.stdout:8/955: mknod db/d14/c14a 0 2024-04-22T23:46:23.829 INFO:tasks.workunit.client.1.smithi044.stdout:3/985: symlink d9/db/d15/d2e/d48/d121/l146 0 2024-04-22T23:46:23.832 INFO:tasks.workunit.client.1.smithi044.stdout:9/831: dwrite d2/dc/d94/fc2 [0,4194304] 0 2024-04-22T23:46:23.835 INFO:tasks.workunit.client.1.smithi044.stdout:3/986: creat d9/db/d15/d25/d5b/f147 x:0 0 0 2024-04-22T23:46:23.836 INFO:tasks.workunit.client.1.smithi044.stdout:5/984: link d15/d5d/db5/d10b/f138 d15/d16/d1d/d22/d4f/d68/f152 0 2024-04-22T23:46:23.838 INFO:tasks.workunit.client.1.smithi044.stdout:8/956: dwrite db/ddd/df2/f148 [0,4194304] 0 2024-04-22T23:46:23.839 INFO:tasks.workunit.client.1.smithi044.stdout:9/832: creat d2/dc/d1d/d34/d38/d4a/dca/f115 x:0 0 0 2024-04-22T23:46:23.858 INFO:tasks.workunit.client.1.smithi044.stdout:3/987: rename d9/db/d15/d25/cb5 to d9/d14/d7d/df8/dfd/c148 0 2024-04-22T23:46:23.861 INFO:tasks.workunit.client.1.smithi044.stdout:8/957: mknod db/d14/d9e/c14b 0 2024-04-22T23:46:23.863 INFO:tasks.workunit.client.1.smithi044.stdout:8/958: write db/d52/d9c/dad/db8/f13a [1055645,50884] 0 2024-04-22T23:46:23.869 INFO:tasks.workunit.client.1.smithi044.stdout:5/985: link d15/d16/d1d/d22/d4f/d68/d76/ff6 d15/d16/d1d/da2/d121/f153 0 2024-04-22T23:46:23.877 INFO:tasks.workunit.client.1.smithi044.stdout:5/986: dread d15/d16/d1d/d22/d2a/d3a/d43/d80/fda [0,4194304] 0 2024-04-22T23:46:23.880 INFO:tasks.workunit.client.1.smithi044.stdout:3/988: rename f7 to d9/db/d33/d5d/f149 0 2024-04-22T23:46:23.885 INFO:tasks.workunit.client.1.smithi044.stdout:3/989: dread d9/d12/db1/fbd [0,4194304] 0 2024-04-22T23:46:23.887 INFO:tasks.workunit.client.1.smithi044.stdout:5/987: mknod d15/d16/d1d/d22/d4f/d68/d76/c154 0 2024-04-22T23:46:23.888 INFO:tasks.workunit.client.1.smithi044.stdout:9/833: rmdir d2/d9 39 2024-04-22T23:46:23.892 INFO:tasks.workunit.client.1.smithi044.stdout:3/990: mkdir d9/d12/db1/d101/d14a 0 2024-04-22T23:46:23.894 INFO:tasks.workunit.client.1.smithi044.stdout:5/988: mkdir d15/d5d/db5/dc0/d38/d155 0 2024-04-22T23:46:23.895 INFO:tasks.workunit.client.1.smithi044.stdout:9/834: creat d2/dc/d1d/d34/d38/d4a/db7/f116 x:0 0 0 2024-04-22T23:46:23.899 INFO:tasks.workunit.client.1.smithi044.stdout:5/989: unlink d15/d16/dbc/d116/df8/f14d 0 2024-04-22T23:46:23.900 INFO:tasks.workunit.client.1.smithi044.stdout:9/835: creat d2/dc/d1d/d3f/d9d/f117 x:0 0 0 2024-04-22T23:46:23.900 INFO:tasks.workunit.client.1.smithi044.stdout:8/959: link db/d52/d9c/dad/l135 db/d14/d84/l14c 0 2024-04-22T23:46:23.902 INFO:tasks.workunit.client.1.smithi044.stdout:8/960: chown db/d14/c14a 24 1 2024-04-22T23:46:23.905 INFO:tasks.workunit.client.1.smithi044.stdout:8/961: symlink db/d68/d13d/l14d 0 2024-04-22T23:46:23.907 INFO:tasks.workunit.client.1.smithi044.stdout:8/962: chown db/d14/d84/fb4 1 1 2024-04-22T23:46:23.907 INFO:tasks.workunit.client.1.smithi044.stdout:9/836: dwrite d2/d6/fd7 [0,4194304] 0 2024-04-22T23:46:23.908 INFO:tasks.workunit.client.1.smithi044.stdout:5/990: link d15/d16/d1d/d22/d2a/d3a/d43/f9b d15/d16/dbc/d116/d115/f156 0 2024-04-22T23:46:23.908 INFO:tasks.workunit.client.1.smithi044.stdout:8/963: readlink db/d68/d8b/dca/lde 0 2024-04-22T23:46:23.909 INFO:tasks.workunit.client.1.smithi044.stdout:8/964: chown db/f106 199036 1 2024-04-22T23:46:23.910 INFO:tasks.workunit.client.1.smithi044.stdout:5/991: stat d15/d16/d1d/daf/f143 0 2024-04-22T23:46:23.911 INFO:tasks.workunit.client.1.smithi044.stdout:5/992: readlink d15/d5d/d82/la9 0 2024-04-22T23:46:23.911 INFO:tasks.workunit.client.1.smithi044.stdout:9/837: fdatasync d2/dc/d1d/d34/d38/d5c/d69/d9f/fb9 0 2024-04-22T23:46:23.913 INFO:tasks.workunit.client.1.smithi044.stdout:9/838: write d2/d6/d42/d8b/f106 [505281,78107] 0 2024-04-22T23:46:23.915 INFO:tasks.workunit.client.1.smithi044.stdout:9/839: fdatasync d2/dc/d1d/d34/d38/d5c/d69/d75/d78/fed 0 2024-04-22T23:46:23.915 INFO:tasks.workunit.client.1.smithi044.stdout:5/993: fdatasync d15/d16/d1d/f69 0 2024-04-22T23:46:23.918 INFO:tasks.workunit.client.1.smithi044.stdout:9/840: chown d2/d6/d42/la2 172478 1 2024-04-22T23:46:23.921 INFO:tasks.workunit.client.1.smithi044.stdout:9/841: creat d2/dc/d51/f118 x:0 0 0 2024-04-22T23:46:23.922 INFO:tasks.workunit.client.1.smithi044.stdout:8/965: link db/d14/d72/f81 db/d6c/f14e 0 2024-04-22T23:46:23.923 INFO:tasks.workunit.client.1.smithi044.stdout:9/842: write d2/d6/d42/d8b/fc5 [644373,70043] 0 2024-04-22T23:46:23.923 INFO:tasks.workunit.client.1.smithi044.stdout:5/994: mkdir d15/d16/d1d/d22/d4f/d68/d157 0 2024-04-22T23:46:23.924 INFO:tasks.workunit.client.1.smithi044.stdout:8/966: read db/ddd/df2/fe1 [8202404,44773] 0 2024-04-22T23:46:23.927 INFO:tasks.workunit.client.1.smithi044.stdout:8/967: rmdir db/d68/d8b/ded 39 2024-04-22T23:46:23.930 INFO:tasks.workunit.client.1.smithi044.stdout:8/968: truncate db/d14/f88 1066712 0 2024-04-22T23:46:23.931 INFO:tasks.workunit.client.1.smithi044.stdout:8/969: write db/d14/d72/f8e [30992,38683] 0 2024-04-22T23:46:23.932 INFO:tasks.workunit.client.1.smithi044.stdout:5/995: rename d15/d3c/l4e to d15/d16/d1d/d22/d4f/d99/l158 0 2024-04-22T23:46:23.936 INFO:tasks.workunit.client.1.smithi044.stdout:5/996: creat d15/d16/dbc/d116/d115/f159 x:0 0 0 2024-04-22T23:46:23.938 INFO:tasks.workunit.client.1.smithi044.stdout:5/997: chown d15/c125 2283409 1 2024-04-22T23:46:23.942 INFO:tasks.workunit.client.1.smithi044.stdout:5/998: mknod d15/d16/d1d/d22/d4f/d68/d134/c15a 0 2024-04-22T23:46:23.946 INFO:tasks.workunit.client.1.smithi044.stdout:5/999: truncate d15/d16/d1d/d22/d2a/d3a/d43/f58 134012 0 2024-04-22T23:46:23.955 INFO:tasks.workunit.client.1.smithi044.stdout:3/991: write d9/db/d144/f94 [1788220,9638] 0 2024-04-22T23:46:23.966 INFO:tasks.workunit.client.1.smithi044.stdout:9/843: write d2/f4 [871869,5208] 0 2024-04-22T23:46:23.968 INFO:tasks.workunit.client.1.smithi044.stdout:9/844: chown d2/dc/d1d/d34/d38/d5c/d69/fac 1 1 2024-04-22T23:46:23.969 INFO:tasks.workunit.client.1.smithi044.stdout:8/970: truncate db/d14/d24/f26 6735774 0 2024-04-22T23:46:23.972 INFO:tasks.workunit.client.1.smithi044.stdout:8/971: creat db/d68/d74/f14f x:0 0 0 2024-04-22T23:46:23.978 INFO:tasks.workunit.client.1.smithi044.stdout:8/972: mknod db/d14/d84/c150 0 2024-04-22T23:46:23.980 INFO:tasks.workunit.client.1.smithi044.stdout:9/845: rename d2/dc/d1d/d34/d38/d4a/db7/ff8 to d2/dc/d1d/d34/d8e/dbd/f119 0 2024-04-22T23:46:23.981 INFO:tasks.workunit.client.1.smithi044.stdout:8/973: creat db/d68/d13d/f151 x:0 0 0 2024-04-22T23:46:23.987 INFO:tasks.workunit.client.1.smithi044.stdout:8/974: read db/d14/f66 [889504,27036] 0 2024-04-22T23:46:23.988 INFO:tasks.workunit.client.1.smithi044.stdout:8/975: dread - db/d14/dc7/dc9/f144 zero size 2024-04-22T23:46:23.993 INFO:tasks.workunit.client.1.smithi044.stdout:9/846: write d2/f3 [7321604,33903] 0 2024-04-22T23:46:23.995 INFO:tasks.workunit.client.1.smithi044.stdout:6/857: sync 2024-04-22T23:46:23.996 INFO:tasks.workunit.client.1.smithi044.stdout:8/976: getdents db/d52/d9c/dad/db8 0 2024-04-22T23:46:23.996 INFO:tasks.workunit.client.1.smithi044.stdout:6/858: write d5/dd/d67/f47 [4274786,23024] 0 2024-04-22T23:46:24.004 INFO:tasks.workunit.client.1.smithi044.stdout:6/859: truncate d5/d3f/d50/d79/d96/fa2 1297389 0 2024-04-22T23:46:24.009 INFO:tasks.workunit.client.1.smithi044.stdout:9/847: truncate d2/dc/d1d/d3f/f4c 792693 0 2024-04-22T23:46:24.010 INFO:tasks.workunit.client.1.smithi044.stdout:6/860: dwrite d5/d30/d73/fd9 [0,4194304] 0 2024-04-22T23:46:24.010 INFO:tasks.workunit.client.1.smithi044.stdout:9/848: dread - d2/d6/d42/daa/ff9 zero size 2024-04-22T23:46:24.011 INFO:tasks.workunit.client.1.smithi044.stdout:8/977: truncate db/d14/d72/f9f 735429 0 2024-04-22T23:46:24.012 INFO:tasks.workunit.client.1.smithi044.stdout:8/978: chown db/f38 491351525 1 2024-04-22T23:46:24.018 INFO:tasks.workunit.client.1.smithi044.stdout:8/979: read db/f3c [2113889,125427] 0 2024-04-22T23:46:24.019 INFO:tasks.workunit.client.1.smithi044.stdout:9/849: dwrite d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/fcf [0,4194304] 0 2024-04-22T23:46:24.028 INFO:tasks.workunit.client.1.smithi044.stdout:9/850: dwrite d2/dc/d94/fc2 [0,4194304] 0 2024-04-22T23:46:24.036 INFO:tasks.workunit.client.1.smithi044.stdout:9/851: readlink d2/dc/d51/lc0 0 2024-04-22T23:46:24.037 INFO:tasks.workunit.client.1.smithi044.stdout:6/861: write d5/dd/f6a [1865602,91032] 0 2024-04-22T23:46:24.039 INFO:tasks.workunit.client.1.smithi044.stdout:8/980: dread db/d14/f25 [0,4194304] 0 2024-04-22T23:46:24.040 INFO:tasks.workunit.client.1.smithi044.stdout:6/862: write d5/d3f/d4c/d23/d3d/d64/f94 [618176,76239] 0 2024-04-22T23:46:24.094 INFO:tasks.workunit.client.1.smithi044.stdout:8/981: dread db/d52/f110 [0,4194304] 0 2024-04-22T23:46:24.094 INFO:tasks.workunit.client.1.smithi044.stdout:6/863: symlink d5/d3f/d50/d79/l10f 0 2024-04-22T23:46:24.094 INFO:tasks.workunit.client.1.smithi044.stdout:6/864: stat d5/dd/d67 0 2024-04-22T23:46:24.094 INFO:tasks.workunit.client.1.smithi044.stdout:8/982: readlink db/d14/d84/dc0/lcb 0 2024-04-22T23:46:24.095 INFO:tasks.workunit.client.1.smithi044.stdout:6/865: chown d5/d3f/cd7 28011 1 2024-04-22T23:46:24.096 INFO:tasks.workunit.client.1.smithi044.stdout:9/852: mkdir d2/dc/d1d/d34/d8e/dbd/dd1/d11a 0 2024-04-22T23:46:24.097 INFO:tasks.workunit.client.1.smithi044.stdout:8/983: readlink db/d14/d84/dc0/l118 0 2024-04-22T23:46:24.098 INFO:tasks.workunit.client.1.smithi044.stdout:6/866: write d5/d3f/d50/d79/fae [3714630,79880] 0 2024-04-22T23:46:24.099 INFO:tasks.workunit.client.1.smithi044.stdout:9/853: symlink d2/dc/d1d/d34/d38/d4a/db7/l11b 0 2024-04-22T23:46:24.099 INFO:tasks.workunit.client.1.smithi044.stdout:6/867: write d5/d3f/d4c/d23/f29 [1843633,129387] 0 2024-04-22T23:46:24.106 INFO:tasks.workunit.client.1.smithi044.stdout:9/854: creat d2/dc/d1d/d34/d38/d5c/d69/d7a/deb/f11c x:0 0 0 2024-04-22T23:46:24.106 INFO:tasks.workunit.client.1.smithi044.stdout:6/868: getdents d5/d3f/d4c/d40/db9/dc0 0 2024-04-22T23:46:24.108 INFO:tasks.workunit.client.1.smithi044.stdout:6/869: mkdir d5/d3f/d4c/d92/d97/d110 0 2024-04-22T23:46:24.111 INFO:tasks.workunit.client.1.smithi044.stdout:9/855: truncate d2/dc/d1d/d34/d38/d5c/d69/fb8 1637062 0 2024-04-22T23:46:24.111 INFO:tasks.workunit.client.1.smithi044.stdout:6/870: mkdir d5/dd/d28/d111 0 2024-04-22T23:46:24.112 INFO:tasks.workunit.client.1.smithi044.stdout:7/987: sync 2024-04-22T23:46:24.115 INFO:tasks.workunit.client.1.smithi044.stdout:7/988: rmdir dd/d55/d5e/d60/d106 39 2024-04-22T23:46:24.116 INFO:tasks.workunit.client.1.smithi044.stdout:6/871: mkdir d5/dd/d112 0 2024-04-22T23:46:24.116 INFO:tasks.workunit.client.1.smithi044.stdout:7/989: read dd/d1a/db7/dc8/f8f [1103853,88824] 0 2024-04-22T23:46:24.117 INFO:tasks.workunit.client.1.smithi044.stdout:6/872: chown d5/d3f/d4c/d23/d3d/d64/fa3 903 1 2024-04-22T23:46:24.124 INFO:tasks.workunit.client.1.smithi044.stdout:7/990: fdatasync dd/d55/d5e/d60/d106/f14d 0 2024-04-22T23:46:24.124 INFO:tasks.workunit.client.1.smithi044.stdout:7/991: readlink dd/d1f/la7 0 2024-04-22T23:46:24.127 INFO:tasks.workunit.client.1.smithi044.stdout:6/873: getdents d5/d3f/d4c/d40/db9 0 2024-04-22T23:46:24.127 INFO:tasks.workunit.client.1.smithi044.stdout:6/874: readlink d5/dd/d28/d75/d88/dcd/de3/l10d 0 2024-04-22T23:46:24.129 INFO:tasks.workunit.client.1.smithi044.stdout:7/992: mkdir dd/d99/dbb/d158 0 2024-04-22T23:46:24.147 INFO:tasks.workunit.client.1.smithi044.stdout:7/993: write dd/d27/f6d [568094,63906] 0 2024-04-22T23:46:24.152 INFO:tasks.workunit.client.1.smithi044.stdout:6/875: dwrite d5/d3f/d4c/d4f/f8f [0,4194304] 0 2024-04-22T23:46:24.158 INFO:tasks.workunit.client.1.smithi044.stdout:8/984: dwrite db/d14/d72/f9f [0,4194304] 0 2024-04-22T23:46:24.161 INFO:tasks.workunit.client.1.smithi044.stdout:7/994: dread dd/d1f/f6a [0,4194304] 0 2024-04-22T23:46:24.162 INFO:tasks.workunit.client.1.smithi044.stdout:6/876: dread - d5/d3f/d4c/d23/d3d/f107 zero size 2024-04-22T23:46:24.163 INFO:tasks.workunit.client.1.smithi044.stdout:8/985: unlink db/d14/dc7/f122 0 2024-04-22T23:46:24.170 INFO:tasks.workunit.client.1.smithi044.stdout:8/986: stat db/d14/d84/dbb/c10b 0 2024-04-22T23:46:24.171 INFO:tasks.workunit.client.1.smithi044.stdout:8/987: read db/fd [794940,99373] 0 2024-04-22T23:46:24.173 INFO:tasks.workunit.client.1.smithi044.stdout:6/877: fdatasync d5/dd/fa8 0 2024-04-22T23:46:24.177 INFO:tasks.workunit.client.1.smithi044.stdout:8/988: fsync db/d14/d72/f9f 0 2024-04-22T23:46:24.183 INFO:tasks.workunit.client.1.smithi044.stdout:7/995: dwrite dd/d1a/f7d [4194304,4194304] 0 2024-04-22T23:46:24.185 INFO:tasks.workunit.client.1.smithi044.stdout:7/996: truncate dd/df7/db8/f141 938519 0 2024-04-22T23:46:24.197 INFO:tasks.workunit.client.1.smithi044.stdout:7/997: mknod dd/d98/de8/c159 0 2024-04-22T23:46:24.199 INFO:tasks.workunit.client.1.smithi044.stdout:6/878: symlink d5/dd/d112/l113 0 2024-04-22T23:46:24.204 INFO:tasks.workunit.client.1.smithi044.stdout:6/879: creat d5/d3f/d50/d79/f114 x:0 0 0 2024-04-22T23:46:24.206 INFO:tasks.workunit.client.1.smithi044.stdout:7/998: dwrite dd/fe3 [0,4194304] 0 2024-04-22T23:46:24.207 INFO:tasks.workunit.client.1.smithi044.stdout:8/989: write db/f106 [8110,19916] 0 2024-04-22T23:46:24.208 INFO:tasks.workunit.client.1.smithi044.stdout:8/990: readlink db/d68/daf/lba 0 2024-04-22T23:46:24.209 INFO:tasks.workunit.client.1.smithi044.stdout:6/880: dread d5/d3f/d50/f6d [0,4194304] 0 2024-04-22T23:46:24.216 INFO:tasks.workunit.client.1.smithi044.stdout:8/991: truncate db/f44 2409178 0 2024-04-22T23:46:24.221 INFO:tasks.workunit.client.1.smithi044.stdout:8/992: getdents db/d68/d137 0 2024-04-22T23:46:24.223 INFO:tasks.workunit.client.1.smithi044.stdout:7/999: dwrite dd/df7/fa9 [0,4194304] 0 2024-04-22T23:46:24.229 INFO:tasks.workunit.client.1.smithi044.stdout:6/881: dread d5/dd/d67/f24 [0,4194304] 0 2024-04-22T23:46:24.237 INFO:tasks.workunit.client.1.smithi044.stdout:6/882: dwrite d5/d3f/d4c/d23/d3d/d64/f94 [0,4194304] 0 2024-04-22T23:46:24.242 INFO:tasks.workunit.client.1.smithi044.stdout:6/883: unlink d5/dd/d28/c101 0 2024-04-22T23:46:24.243 INFO:tasks.workunit.client.1.smithi044.stdout:8/993: dread db/d14/d9e/ff4 [0,4194304] 0 2024-04-22T23:46:24.244 INFO:tasks.workunit.client.1.smithi044.stdout:8/994: readlink db/d14/dc7/dc9/lf9 0 2024-04-22T23:46:24.249 INFO:tasks.workunit.client.1.smithi044.stdout:8/995: read db/d14/d84/fb4 [3396854,25810] 0 2024-04-22T23:46:24.251 INFO:tasks.workunit.client.1.smithi044.stdout:8/996: write db/d14/f25 [1205209,94868] 0 2024-04-22T23:46:24.259 INFO:tasks.workunit.client.1.smithi044.stdout:8/997: dwrite db/d14/d24/f5c [0,4194304] 0 2024-04-22T23:46:24.265 INFO:tasks.workunit.client.1.smithi044.stdout:6/884: dread d5/dd/d28/f2e [0,4194304] 0 2024-04-22T23:46:24.265 INFO:tasks.workunit.client.1.smithi044.stdout:3/992: sync 2024-04-22T23:46:24.265 INFO:tasks.workunit.client.1.smithi044.stdout:1/950: sync 2024-04-22T23:46:24.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/795: sync 2024-04-22T23:46:24.269 INFO:tasks.workunit.client.1.smithi044.stdout:8/998: dwrite db/d14/f25 [0,4194304] 0 2024-04-22T23:46:24.269 INFO:tasks.workunit.client.1.smithi044.stdout:3/993: chown d9/db/d33/d5d/d64/ff1 41 1 2024-04-22T23:46:24.270 INFO:tasks.workunit.client.1.smithi044.stdout:6/885: mkdir d5/dd/d115 0 2024-04-22T23:46:24.274 INFO:tasks.workunit.client.1.smithi044.stdout:6/886: chown d5/d3f/d4c/lb0 74536407 1 2024-04-22T23:46:24.276 INFO:tasks.workunit.client.1.smithi044.stdout:2/796: dwrite d8/d27/d5b/d74/fab [0,4194304] 0 2024-04-22T23:46:24.279 INFO:tasks.workunit.client.1.smithi044.stdout:1/951: creat d1/d8/d1c/d7b/d80/dfd/dbf/ddb/f14c x:0 0 0 2024-04-22T23:46:24.280 INFO:tasks.workunit.client.1.smithi044.stdout:2/797: write d8/d19/d36/d46/d63/d116/ffe [490764,46019] 0 2024-04-22T23:46:24.282 INFO:tasks.workunit.client.1.smithi044.stdout:2/798: chown d8/d19/d36/d44/f70 281 1 2024-04-22T23:46:24.283 INFO:tasks.workunit.client.1.smithi044.stdout:3/994: mknod d9/db/d144/d95/dcb/c14b 0 2024-04-22T23:46:24.284 INFO:tasks.workunit.client.1.smithi044.stdout:2/799: truncate d8/d19/f10a 382966 0 2024-04-22T23:46:24.285 INFO:tasks.workunit.client.1.smithi044.stdout:8/999: getdents db/d14/d72/d13f 0 2024-04-22T23:46:24.289 INFO:tasks.workunit.client.1.smithi044.stdout:1/952: truncate d1/d8/d20/d47/f8a 53055 0 2024-04-22T23:46:24.291 INFO:tasks.workunit.client.1.smithi044.stdout:3/995: symlink d9/db/d144/d95/l14c 0 2024-04-22T23:46:24.293 INFO:tasks.workunit.client.1.smithi044.stdout:1/953: creat d1/d8/d34/da1/dab/de9/db9/f14d x:0 0 0 2024-04-22T23:46:24.293 INFO:tasks.workunit.client.1.smithi044.stdout:2/800: symlink d8/d27/d5b/d111/l120 0 2024-04-22T23:46:24.300 INFO:tasks.workunit.client.1.smithi044.stdout:1/954: creat d1/d8/d20/d45/d75/dfb/d95/f14e x:0 0 0 2024-04-22T23:46:24.306 INFO:tasks.workunit.client.1.smithi044.stdout:6/887: dwrite d5/d3f/d4c/d23/d3d/f4a [0,4194304] 0 2024-04-22T23:46:24.310 INFO:tasks.workunit.client.1.smithi044.stdout:1/955: mknod d1/d8/d34/da1/dab/de9/d116/c14f 0 2024-04-22T23:46:24.313 INFO:tasks.workunit.client.1.smithi044.stdout:2/801: dwrite d8/d19/d36/d93/fbc [0,4194304] 0 2024-04-22T23:46:24.315 INFO:tasks.workunit.client.1.smithi044.stdout:3/996: truncate d9/db/d15/d2e/d13b/d6b/ded/d113/dbb/d10c/f118 475669 0 2024-04-22T23:46:24.319 INFO:tasks.workunit.client.1.smithi044.stdout:6/888: creat d5/d3f/d50/d79/f116 x:0 0 0 2024-04-22T23:46:24.319 INFO:tasks.workunit.client.1.smithi044.stdout:6/889: stat d5/dd/d28/f2c 0 2024-04-22T23:46:24.320 INFO:tasks.workunit.client.1.smithi044.stdout:1/956: dread - d1/d8/d1c/d7b/d80/dfd/dad/fbb zero size 2024-04-22T23:46:24.321 INFO:tasks.workunit.client.1.smithi044.stdout:2/802: symlink d8/d19/d36/d46/ddb/ddd/df5/l121 0 2024-04-22T23:46:24.328 INFO:tasks.workunit.client.1.smithi044.stdout:6/890: rename d5/dd to d5/d3f/d4c/d4f/d117 0 2024-04-22T23:46:24.331 INFO:tasks.workunit.client.1.smithi044.stdout:1/957: unlink d1/d8/l13 0 2024-04-22T23:46:24.332 INFO:tasks.workunit.client.1.smithi044.stdout:2/803: mknod d8/d19/d36/d93/dae/d11e/c122 0 2024-04-22T23:46:24.334 INFO:tasks.workunit.client.1.smithi044.stdout:6/891: symlink d5/d3f/d4c/d92/d97/l118 0 2024-04-22T23:46:24.339 INFO:tasks.workunit.client.1.smithi044.stdout:1/958: dwrite d1/d8/d1c/d23/d3e/fcf [0,4194304] 0 2024-04-22T23:46:24.350 INFO:tasks.workunit.client.1.smithi044.stdout:3/997: write d9/db/d15/f85 [1720629,88560] 0 2024-04-22T23:46:24.354 INFO:tasks.workunit.client.1.smithi044.stdout:1/959: creat d1/d8/d20/d45/f150 x:0 0 0 2024-04-22T23:46:24.357 INFO:tasks.workunit.client.1.smithi044.stdout:9/856: sync 2024-04-22T23:46:24.358 INFO:tasks.workunit.client.1.smithi044.stdout:3/998: dwrite d9/db/d141/f145 [0,4194304] 0 2024-04-22T23:46:24.359 INFO:tasks.workunit.client.1.smithi044.stdout:9/857: chown d2/dc/d1d/d3f/de3 4519 1 2024-04-22T23:46:24.361 INFO:tasks.workunit.client.1.smithi044.stdout:2/804: write f4 [1843412,39280] 0 2024-04-22T23:46:24.367 INFO:tasks.workunit.client.1.smithi044.stdout:9/858: write d2/dc/d1d/d34/d38/d5c/d69/d7a/deb/fec [334027,45383] 0 2024-04-22T23:46:24.367 INFO:tasks.workunit.client.1.smithi044.stdout:3/999: mknod d9/d14/d7d/c14d 0 2024-04-22T23:46:24.369 INFO:tasks.workunit.client.1.smithi044.stdout:6/892: dwrite d5/f6b [0,4194304] 0 2024-04-22T23:46:24.374 INFO:tasks.workunit.client.1.smithi044.stdout:1/960: write d1/d8/d20/dae/deb/f101 [1040955,22638] 0 2024-04-22T23:46:24.379 INFO:tasks.workunit.client.1.smithi044.stdout:9/859: unlink d2/d6/d42/d8b/fc5 0 2024-04-22T23:46:24.388 INFO:tasks.workunit.client.1.smithi044.stdout:9/860: unlink d2/d6/l39 0 2024-04-22T23:46:24.389 INFO:tasks.workunit.client.1.smithi044.stdout:6/893: dread d5/d3f/d4c/d23/d3d/d64/f77 [0,4194304] 0 2024-04-22T23:46:24.389 INFO:tasks.workunit.client.1.smithi044.stdout:1/961: mknod d1/d8/d34/c151 0 2024-04-22T23:46:24.392 INFO:tasks.workunit.client.1.smithi044.stdout:1/962: stat d1/d8/d20/d45/d75/df2 0 2024-04-22T23:46:24.392 INFO:tasks.workunit.client.1.smithi044.stdout:9/861: fdatasync d2/dc/fb0 0 2024-04-22T23:46:24.393 INFO:tasks.workunit.client.1.smithi044.stdout:2/805: dwrite d8/d19/d96/f7a [0,4194304] 0 2024-04-22T23:46:24.397 INFO:tasks.workunit.client.1.smithi044.stdout:6/894: dread f1 [8388608,4194304] 0 2024-04-22T23:46:24.402 INFO:tasks.workunit.client.1.smithi044.stdout:1/963: mkdir d1/d8/d20/dae/deb/d152 0 2024-04-22T23:46:24.407 INFO:tasks.workunit.client.1.smithi044.stdout:6/895: creat d5/d3f/d50/d79/f119 x:0 0 0 2024-04-22T23:46:24.413 INFO:tasks.workunit.client.1.smithi044.stdout:6/896: mknod d5/d3f/d4c/d23/d3d/c11a 0 2024-04-22T23:46:24.414 INFO:tasks.workunit.client.1.smithi044.stdout:6/897: stat d5/d3f/ff7 0 2024-04-22T23:46:24.415 INFO:tasks.workunit.client.1.smithi044.stdout:2/806: creat d8/d19/d36/d46/d63/f123 x:0 0 0 2024-04-22T23:46:24.416 INFO:tasks.workunit.client.1.smithi044.stdout:9/862: truncate d2/dc/fb0 3724198 0 2024-04-22T23:46:24.417 INFO:tasks.workunit.client.1.smithi044.stdout:1/964: mkdir d1/d8/d1c/d7b/d80/dfd/dbf/ddb/d134/d153 0 2024-04-22T23:46:24.420 INFO:tasks.workunit.client.1.smithi044.stdout:2/807: mknod d8/d19/d36/d93/dbf/c124 0 2024-04-22T23:46:24.421 INFO:tasks.workunit.client.1.smithi044.stdout:2/808: readlink d8/d19/d36/d93/dae/d11e/l11a 0 2024-04-22T23:46:24.422 INFO:tasks.workunit.client.1.smithi044.stdout:9/863: fsync d2/dc/d94/f55 0 2024-04-22T23:46:24.424 INFO:tasks.workunit.client.1.smithi044.stdout:1/965: mkdir d1/d8/d20/dae/deb/d124/d154 0 2024-04-22T23:46:24.425 INFO:tasks.workunit.client.1.smithi044.stdout:6/898: dwrite d5/d3f/f22 [0,4194304] 0 2024-04-22T23:46:24.427 INFO:tasks.workunit.client.1.smithi044.stdout:2/809: dread - d8/d19/f51 zero size 2024-04-22T23:46:24.428 INFO:tasks.workunit.client.1.smithi044.stdout:6/899: write d5/d3f/d4c/d4f/d117/d67/f5d [314448,108996] 0 2024-04-22T23:46:24.431 INFO:tasks.workunit.client.1.smithi044.stdout:6/900: truncate d5/d3f/d50/d79/d96/ff5 335903 0 2024-04-22T23:46:24.434 INFO:tasks.workunit.client.1.smithi044.stdout:6/901: read d5/d3f/d4c/d4f/d117/fc5 [30827,79713] 0 2024-04-22T23:46:24.435 INFO:tasks.workunit.client.1.smithi044.stdout:2/810: creat d8/d27/d5b/d110/daf/f125 x:0 0 0 2024-04-22T23:46:24.436 INFO:tasks.workunit.client.1.smithi044.stdout:6/902: write d5/d3f/d4c/df1/dfb/f10e [859058,93934] 0 2024-04-22T23:46:24.443 INFO:tasks.workunit.client.1.smithi044.stdout:2/811: dwrite d8/d19/f39 [0,4194304] 0 2024-04-22T23:46:24.444 INFO:tasks.workunit.client.1.smithi044.stdout:9/864: dread d2/dc/d1d/d34/d8e/dbd/f10e [0,4194304] 0 2024-04-22T23:46:24.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:24 smithi031 ceph-mon[26721]: pgmap v15: 65 pgs: 65 active+clean; 2.8 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 122 MiB/s wr, 281 op/s 2024-04-22T23:46:24.447 INFO:tasks.workunit.client.1.smithi044.stdout:6/903: rename d5/d3f/d4c/d92/d97/de5 to d5/d3f/d4c/d4f/d117/d28/d75/d88/dcd/de3/d11b 0 2024-04-22T23:46:24.451 INFO:tasks.workunit.client.1.smithi044.stdout:9/865: chown d2/dc/c2a 427976 1 2024-04-22T23:46:24.452 INFO:tasks.workunit.client.1.smithi044.stdout:6/904: write d5/d3f/d4c/d40/d53/d56/db7/fe2 [284817,14166] 0 2024-04-22T23:46:24.453 INFO:tasks.workunit.client.1.smithi044.stdout:2/812: dwrite d8/d19/d36/fcd [0,4194304] 0 2024-04-22T23:46:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:24 smithi044 ceph-mon[34463]: pgmap v15: 65 pgs: 65 active+clean; 2.8 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail; 44 MiB/s rd, 122 MiB/s wr, 281 op/s 2024-04-22T23:46:24.463 INFO:tasks.workunit.client.1.smithi044.stdout:6/905: dwrite d5/f4d [0,4194304] 0 2024-04-22T23:46:24.464 INFO:tasks.workunit.client.1.smithi044.stdout:2/813: dread - d8/d27/d5b/fb5 zero size 2024-04-22T23:46:24.466 INFO:tasks.workunit.client.1.smithi044.stdout:6/906: dread - d5/d3f/d4c/f72 zero size 2024-04-22T23:46:24.466 INFO:tasks.workunit.client.1.smithi044.stdout:9/866: dwrite d2/d6/d42/d8b/f106 [0,4194304] 0 2024-04-22T23:46:24.471 INFO:tasks.workunit.client.1.smithi044.stdout:2/814: mknod d8/d27/d5b/d110/daf/c126 0 2024-04-22T23:46:24.475 INFO:tasks.workunit.client.1.smithi044.stdout:2/815: mknod d8/d60/c127 0 2024-04-22T23:46:24.476 INFO:tasks.workunit.client.1.smithi044.stdout:6/907: dwrite d5/d3f/d4c/d4f/d117/d28/f10a [0,4194304] 0 2024-04-22T23:46:24.488 INFO:tasks.workunit.client.1.smithi044.stdout:9/867: link d2/dc/d1d/d3f/f4c d2/dc/f11d 0 2024-04-22T23:46:24.493 INFO:tasks.workunit.client.1.smithi044.stdout:9/868: creat d2/dc/d1d/d34/d38/d4a/dca/f11e x:0 0 0 2024-04-22T23:46:24.498 INFO:tasks.workunit.client.1.smithi044.stdout:6/908: link d5/d3f/fa d5/d3f/da7/f11c 0 2024-04-22T23:46:24.503 INFO:tasks.workunit.client.1.smithi044.stdout:2/816: write d8/d27/d5b/f7b [499979,67479] 0 2024-04-22T23:46:24.504 INFO:tasks.workunit.client.1.smithi044.stdout:2/817: chown d8/d19/f39 919 1 2024-04-22T23:46:24.509 INFO:tasks.workunit.client.1.smithi044.stdout:2/818: dread d8/d19/d36/d46/d63/d65/d72/f86 [0,4194304] 0 2024-04-22T23:46:24.509 INFO:tasks.workunit.client.1.smithi044.stdout:2/819: chown d8/d19/d96/ff3 8353199 1 2024-04-22T23:46:24.510 INFO:tasks.workunit.client.1.smithi044.stdout:9/869: write d2/dc/d1d/d34/d38/d5c/d69/d75/f7c [38220,83445] 0 2024-04-22T23:46:24.515 INFO:tasks.workunit.client.1.smithi044.stdout:6/909: dwrite d5/d3f/d4c/d23/f3b [0,4194304] 0 2024-04-22T23:46:24.516 INFO:tasks.workunit.client.1.smithi044.stdout:6/910: readlink d5/d3f/lfd 0 2024-04-22T23:46:24.521 INFO:tasks.workunit.client.1.smithi044.stdout:9/870: creat d2/dc/d1d/d34/d38/d5c/d69/f11f x:0 0 0 2024-04-22T23:46:24.526 INFO:tasks.workunit.client.1.smithi044.stdout:6/911: dwrite d5/d3f/d4c/d92/fea [0,4194304] 0 2024-04-22T23:46:24.526 INFO:tasks.workunit.client.1.smithi044.stdout:2/820: rename d8/d19/d36/d46/l66 to d8/d19/d36/d46/d63/l128 0 2024-04-22T23:46:24.529 INFO:tasks.workunit.client.1.smithi044.stdout:9/871: read d2/d6/fa8 [195473,42038] 0 2024-04-22T23:46:24.547 INFO:tasks.workunit.client.1.smithi044.stdout:2/821: dwrite d8/d19/d36/d93/dae/d11e/f103 [0,4194304] 0 2024-04-22T23:46:24.549 INFO:tasks.workunit.client.1.smithi044.stdout:6/912: dread f1 [0,4194304] 0 2024-04-22T23:46:24.552 INFO:tasks.workunit.client.1.smithi044.stdout:2/822: creat d8/d19/d36/d46/d63/d65/d72/f129 x:0 0 0 2024-04-22T23:46:24.552 INFO:tasks.workunit.client.1.smithi044.stdout:2/823: dread - d8/d27/d5b/fb5 zero size 2024-04-22T23:46:24.554 INFO:tasks.workunit.client.1.smithi044.stdout:6/913: symlink d5/d3f/da7/l11d 0 2024-04-22T23:46:24.555 INFO:tasks.workunit.client.1.smithi044.stdout:2/824: mkdir d8/d19/d36/d46/df0/d12a 0 2024-04-22T23:46:24.555 INFO:tasks.workunit.client.1.smithi044.stdout:2/825: fdatasync d8/fe 0 2024-04-22T23:46:24.557 INFO:tasks.workunit.client.1.smithi044.stdout:6/914: read d5/d30/d73/fc1 [1573343,17586] 0 2024-04-22T23:46:24.570 INFO:tasks.workunit.client.1.smithi044.stdout:6/915: write d5/d30/d73/f9f [303107,21063] 0 2024-04-22T23:46:24.573 INFO:tasks.workunit.client.1.smithi044.stdout:6/916: fsync d5/d3f/d4c/d4f/d117/d28/f2e 0 2024-04-22T23:46:24.574 INFO:tasks.workunit.client.1.smithi044.stdout:2/826: dwrite d8/d19/d36/fa5 [0,4194304] 0 2024-04-22T23:46:24.578 INFO:tasks.workunit.client.1.smithi044.stdout:6/917: mkdir d5/d3f/d4c/d92/d97/d11e 0 2024-04-22T23:46:24.579 INFO:tasks.workunit.client.1.smithi044.stdout:6/918: dread - d5/d3f/d4c/d40/d53/d56/f6e zero size 2024-04-22T23:46:24.597 INFO:tasks.workunit.client.1.smithi044.stdout:2/827: dwrite d8/d19/d36/d44/f97 [0,4194304] 0 2024-04-22T23:46:24.598 INFO:tasks.workunit.client.1.smithi044.stdout:6/919: dwrite d5/d3f/d50/f6d [0,4194304] 0 2024-04-22T23:46:24.601 INFO:tasks.workunit.client.1.smithi044.stdout:2/828: creat d8/d19/d36/f12b x:0 0 0 2024-04-22T23:46:24.605 INFO:tasks.workunit.client.1.smithi044.stdout:6/920: creat d5/d3f/d4c/d40/db9/dc0/f11f x:0 0 0 2024-04-22T23:46:24.606 INFO:tasks.workunit.client.1.smithi044.stdout:2/829: rename d8/d19/c6e to d8/d19/d36/d46/df0/d12a/c12c 0 2024-04-22T23:46:24.608 INFO:tasks.workunit.client.1.smithi044.stdout:6/921: symlink d5/d3f/da7/l120 0 2024-04-22T23:46:24.611 INFO:tasks.workunit.client.1.smithi044.stdout:6/922: symlink d5/d3f/d4c/df1/dfb/l121 0 2024-04-22T23:46:24.612 INFO:tasks.workunit.client.1.smithi044.stdout:6/923: dread - d5/d3f/d4c/f72 zero size 2024-04-22T23:46:24.615 INFO:tasks.workunit.client.1.smithi044.stdout:6/924: mkdir d5/d3f/d50/d122 0 2024-04-22T23:46:24.629 INFO:tasks.workunit.client.1.smithi044.stdout:2/830: dwrite d8/d19/f39 [4194304,4194304] 0 2024-04-22T23:46:24.636 INFO:tasks.workunit.client.1.smithi044.stdout:2/831: fdatasync d8/d27/fc8 0 2024-04-22T23:46:24.641 INFO:tasks.workunit.client.1.smithi044.stdout:2/832: unlink d8/fd 0 2024-04-22T23:46:24.646 INFO:tasks.workunit.client.1.smithi044.stdout:2/833: creat d8/d60/f12d x:0 0 0 2024-04-22T23:46:24.652 INFO:tasks.workunit.client.1.smithi044.stdout:2/834: rename d8/d19/d36/d44/cd4 to d8/d19/d36/d46/ddb/ddd/c12e 0 2024-04-22T23:46:24.674 INFO:tasks.workunit.client.1.smithi044.stdout:1/966: sync 2024-04-22T23:46:24.686 INFO:tasks.workunit.client.1.smithi044.stdout:1/967: dread d1/d8/d1c/f5b [8388608,4194304] 0 2024-04-22T23:46:24.693 INFO:tasks.workunit.client.1.smithi044.stdout:1/968: write d1/d8/d1c/d7b/d80/dfd/dad/fbb [1022427,16576] 0 2024-04-22T23:46:24.697 INFO:tasks.workunit.client.1.smithi044.stdout:1/969: creat d1/d8/d20/d45/f155 x:0 0 0 2024-04-22T23:46:24.703 INFO:tasks.workunit.client.1.smithi044.stdout:1/970: write d1/d8/d1c/d23/d3e/f7e [600440,114501] 0 2024-04-22T23:46:24.706 INFO:tasks.workunit.client.1.smithi044.stdout:1/971: mkdir d1/d8/d1c/d7b/d80/de3/d156 0 2024-04-22T23:46:24.709 INFO:tasks.workunit.client.1.smithi044.stdout:1/972: mknod d1/d8/d20/d45/c157 0 2024-04-22T23:46:24.714 INFO:tasks.workunit.client.1.smithi044.stdout:1/973: write d1/d8/d1c/d7b/fd9 [620371,112781] 0 2024-04-22T23:46:24.716 INFO:tasks.workunit.client.1.smithi044.stdout:1/974: dread d1/d8/d20/d45/d75/df2/ff5 [0,4194304] 0 2024-04-22T23:46:24.719 INFO:tasks.workunit.client.1.smithi044.stdout:1/975: creat d1/d8/d20/dae/deb/f158 x:0 0 0 2024-04-22T23:46:24.723 INFO:tasks.workunit.client.1.smithi044.stdout:1/976: mkdir d1/d8/d1c/d7b/d80/dfd/d144/d159 0 2024-04-22T23:46:24.725 INFO:tasks.workunit.client.1.smithi044.stdout:1/977: symlink d1/d8/d20/d45/d75/dfb/l15a 0 2024-04-22T23:46:24.728 INFO:tasks.workunit.client.1.smithi044.stdout:1/978: creat d1/d8/d1c/d7b/d80/dfd/dbf/ddb/f15b x:0 0 0 2024-04-22T23:46:24.729 INFO:tasks.workunit.client.1.smithi044.stdout:1/979: fsync d1/d8/d20/dae/f143 0 2024-04-22T23:46:24.730 INFO:tasks.workunit.client.1.smithi044.stdout:1/980: read d1/d8/f1a [952740,66184] 0 2024-04-22T23:46:24.743 INFO:tasks.workunit.client.1.smithi044.stdout:1/981: dread d1/d8/d20/dae/fe2 [0,4194304] 0 2024-04-22T23:46:24.755 INFO:tasks.workunit.client.1.smithi044.stdout:1/982: dwrite d1/d8/d1c/d7b/f65 [4194304,4194304] 0 2024-04-22T23:46:24.762 INFO:tasks.workunit.client.1.smithi044.stdout:1/983: symlink d1/d8/d1c/d7b/d80/dfd/dad/dcb/l15c 0 2024-04-22T23:46:24.768 INFO:tasks.workunit.client.1.smithi044.stdout:1/984: getdents d1/d8/d20/d45/d75/df2/d99 0 2024-04-22T23:46:24.770 INFO:tasks.workunit.client.1.smithi044.stdout:1/985: rmdir d1/d8/d1c/d7b/d80/dfd/dbf/ddb 39 2024-04-22T23:46:24.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/986: readlink d1/d8/d20/d45/d5d/l91 0 2024-04-22T23:46:24.771 INFO:tasks.workunit.client.1.smithi044.stdout:1/987: stat d1/d8/d20/d47/d4a/d60/l8f 0 2024-04-22T23:46:24.773 INFO:tasks.workunit.client.1.smithi044.stdout:1/988: mknod d1/d8/d34/da1/d123/c15d 0 2024-04-22T23:46:24.779 INFO:tasks.workunit.client.1.smithi044.stdout:1/989: getdents d1/d8/d34/da1/d123 0 2024-04-22T23:46:24.782 INFO:tasks.workunit.client.1.smithi044.stdout:1/990: mknod d1/d8/d20/dae/d11c/c15e 0 2024-04-22T23:46:24.784 INFO:tasks.workunit.client.1.smithi044.stdout:1/991: creat d1/d8/d34/da1/dab/de9/d116/f15f x:0 0 0 2024-04-22T23:46:24.793 INFO:tasks.workunit.client.1.smithi044.stdout:1/992: write d1/d8/d1c/d10d/f11a [915930,71353] 0 2024-04-22T23:46:24.813 INFO:tasks.workunit.client.1.smithi044.stdout:1/993: truncate d1/d8/d20/d45/d5d/fdf 2551936 0 2024-04-22T23:46:24.817 INFO:tasks.workunit.client.1.smithi044.stdout:1/994: creat d1/d8/d20/d47/d92/f160 x:0 0 0 2024-04-22T23:46:24.819 INFO:tasks.workunit.client.1.smithi044.stdout:1/995: write d1/d8/d1c/d7b/d80/f142 [902152,108751] 0 2024-04-22T23:46:24.826 INFO:tasks.workunit.client.1.smithi044.stdout:1/996: getdents d1/d8/d20/d47/d4a/d60 0 2024-04-22T23:46:24.826 INFO:tasks.workunit.client.1.smithi044.stdout:1/997: chown d1/d8/d34/cd8 112167 1 2024-04-22T23:46:24.829 INFO:tasks.workunit.client.1.smithi044.stdout:1/998: read d1/d8/d20/d45/d5d/f61 [3682913,130402] 0 2024-04-22T23:46:24.829 INFO:tasks.workunit.client.1.smithi044.stdout:9/872: sync 2024-04-22T23:46:24.831 INFO:tasks.workunit.client.1.smithi044.stdout:1/999: dread - d1/d8/d20/fdd zero size 2024-04-22T23:46:24.836 INFO:tasks.workunit.client.1.smithi044.stdout:9/873: dread d2/f7f [0,4194304] 0 2024-04-22T23:46:24.844 INFO:tasks.workunit.client.1.smithi044.stdout:9/874: creat d2/dc/d1d/d34/d38/d5c/d69/d75/d78/f120 x:0 0 0 2024-04-22T23:46:24.851 INFO:tasks.workunit.client.1.smithi044.stdout:9/875: truncate d2/dc/d1d/d3f/fd9 20077 0 2024-04-22T23:46:24.859 INFO:tasks.workunit.client.1.smithi044.stdout:9/876: dwrite d2/dc/d1d/d34/d38/d5c/f76 [0,4194304] 0 2024-04-22T23:46:24.871 INFO:tasks.workunit.client.1.smithi044.stdout:6/925: sync 2024-04-22T23:46:24.871 INFO:tasks.workunit.client.1.smithi044.stdout:2/835: sync 2024-04-22T23:46:24.873 INFO:tasks.workunit.client.1.smithi044.stdout:6/926: dread - d5/d3f/d4c/d40/db9/dc0/f11f zero size 2024-04-22T23:46:24.874 INFO:tasks.workunit.client.1.smithi044.stdout:2/836: write d8/d19/d36/d44/f70 [2823133,111639] 0 2024-04-22T23:46:24.875 INFO:tasks.workunit.client.1.smithi044.stdout:6/927: fdatasync d5/d3f/f22 0 2024-04-22T23:46:24.879 INFO:tasks.workunit.client.1.smithi044.stdout:2/837: symlink d8/d19/d36/d93/dbf/l12f 0 2024-04-22T23:46:24.880 INFO:tasks.workunit.client.1.smithi044.stdout:2/838: readlink d8/d27/d5b/d110/l3a 0 2024-04-22T23:46:24.883 INFO:tasks.workunit.client.1.smithi044.stdout:6/928: rename d5/d3f/d4c/d83/fd1 to d5/d3f/d50/d122/f123 0 2024-04-22T23:46:24.889 INFO:tasks.workunit.client.1.smithi044.stdout:6/929: unlink d5/d3f/d4c/d23/d3d/d64/f94 0 2024-04-22T23:46:24.890 INFO:tasks.workunit.client.1.smithi044.stdout:6/930: chown d5/d3f/d4c/d4f/d117/d112 2489104 1 2024-04-22T23:46:24.893 INFO:tasks.workunit.client.1.smithi044.stdout:9/877: dread d2/dc/d1d/d34/d38/d5c/d69/f96 [0,4194304] 0 2024-04-22T23:46:24.894 INFO:tasks.workunit.client.1.smithi044.stdout:6/931: creat d5/d3f/d4c/df1/dfb/f124 x:0 0 0 2024-04-22T23:46:24.895 INFO:tasks.workunit.client.1.smithi044.stdout:6/932: rename d5/d3f/d4c to d5/d3f/d4c/d40/db9/dc0/d125 22 2024-04-22T23:46:24.897 INFO:tasks.workunit.client.1.smithi044.stdout:9/878: mkdir d2/dc/d1d/d34/d38/d4a/db7/d121 0 2024-04-22T23:46:24.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/933: mknod d5/d3f/d4c/d83/c126 0 2024-04-22T23:46:24.900 INFO:tasks.workunit.client.1.smithi044.stdout:6/934: chown d5/d3f/d4c/d40/f71 96 1 2024-04-22T23:46:24.901 INFO:tasks.workunit.client.1.smithi044.stdout:9/879: mknod d2/d6/d42/c122 0 2024-04-22T23:46:24.904 INFO:tasks.workunit.client.1.smithi044.stdout:2/839: dread d8/d19/d36/d46/f67 [0,4194304] 0 2024-04-22T23:46:24.905 INFO:tasks.workunit.client.1.smithi044.stdout:2/840: write d8/d19/d36/d44/f109 [332670,66462] 0 2024-04-22T23:46:24.908 INFO:tasks.workunit.client.1.smithi044.stdout:9/880: rename d2/d9 to d2/dc/d1d/d34/d38/d4a/db7/d123 0 2024-04-22T23:46:24.912 INFO:tasks.workunit.client.1.smithi044.stdout:9/881: truncate d2/dc/d1d/d34/d8e/dbd/f10e 4491141 0 2024-04-22T23:46:24.913 INFO:tasks.workunit.client.1.smithi044.stdout:6/935: rename d5/d3f/d4c/d23/cd8 to d5/d3f/d4c/d4f/d117/d28/d75/d88/c127 0 2024-04-22T23:46:24.913 INFO:tasks.workunit.client.1.smithi044.stdout:2/841: dwrite d8/d19/d36/d46/d63/d65/d94/f11d [0,4194304] 0 2024-04-22T23:46:24.915 INFO:tasks.workunit.client.1.smithi044.stdout:2/842: write f6 [4276701,114731] 0 2024-04-22T23:46:24.917 INFO:tasks.workunit.client.1.smithi044.stdout:6/936: write d5/d3f/d4c/d4f/d117/f68 [2659663,27515] 0 2024-04-22T23:46:24.923 INFO:tasks.workunit.client.1.smithi044.stdout:6/937: rmdir d5/d3f/d4c/d23 39 2024-04-22T23:46:24.930 INFO:tasks.workunit.client.1.smithi044.stdout:6/938: mkdir d5/d3f/d4c/d4f/d117/d28/d75/d88/d128 0 2024-04-22T23:46:24.931 INFO:tasks.workunit.client.1.smithi044.stdout:6/939: stat d5/d3f/d4c/d4f/d117/fd2 0 2024-04-22T23:46:24.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/940: rename d5/d30/d73 to d5/d3f/d4c/d92/d97/d129 0 2024-04-22T23:46:24.938 INFO:tasks.workunit.client.1.smithi044.stdout:9/882: getdents d2/dc/d1d/d34/d8e 0 2024-04-22T23:46:24.938 INFO:tasks.workunit.client.1.smithi044.stdout:6/941: dread d5/d3f/d4c/d4f/d117/d28/fe7 [0,4194304] 0 2024-04-22T23:46:24.942 INFO:tasks.workunit.client.1.smithi044.stdout:2/843: getdents d8/d19/d36/d46/d63 0 2024-04-22T23:46:24.945 INFO:tasks.workunit.client.1.smithi044.stdout:6/942: fsync d5/d3f/d4c/d4f/d117/fc5 0 2024-04-22T23:46:24.955 INFO:tasks.workunit.client.1.smithi044.stdout:9/883: link d2/dc/d1d/d3f/d9d/f113 d2/dc/d1d/d34/d38/d4a/f124 0 2024-04-22T23:46:24.960 INFO:tasks.workunit.client.1.smithi044.stdout:9/884: symlink d2/dc/d1d/d34/d38/d5c/d69/dcb/l125 0 2024-04-22T23:46:24.961 INFO:tasks.workunit.client.1.smithi044.stdout:2/844: creat d8/d60/f130 x:0 0 0 2024-04-22T23:46:24.964 INFO:tasks.workunit.client.1.smithi044.stdout:6/943: getdents d5/d3f/d4c/df1 0 2024-04-22T23:46:24.975 INFO:tasks.workunit.client.1.smithi044.stdout:6/944: symlink d5/d3f/d4c/d4f/d117/d28/d111/l12a 0 2024-04-22T23:46:24.976 INFO:tasks.workunit.client.1.smithi044.stdout:2/845: creat d8/d19/f131 x:0 0 0 2024-04-22T23:46:24.980 INFO:tasks.workunit.client.1.smithi044.stdout:2/846: creat d8/d19/d36/d44/dd8/f132 x:0 0 0 2024-04-22T23:46:24.982 INFO:tasks.workunit.client.1.smithi044.stdout:2/847: fdatasync d8/d19/f10a 0 2024-04-22T23:46:24.985 INFO:tasks.workunit.client.1.smithi044.stdout:9/885: dwrite d2/dc/d1d/d34/d38/d5c/d69/fb8 [0,4194304] 0 2024-04-22T23:46:24.985 INFO:tasks.workunit.client.1.smithi044.stdout:2/848: fdatasync d8/d19/d36/d46/d63/d65/d94/fa8 0 2024-04-22T23:46:24.993 INFO:tasks.workunit.client.1.smithi044.stdout:9/886: mknod d2/dc/d1d/d3f/de3/d101/c126 0 2024-04-22T23:46:24.994 INFO:tasks.workunit.client.1.smithi044.stdout:2/849: dwrite d8/d19/d5d/f7e [0,4194304] 0 2024-04-22T23:46:24.999 INFO:tasks.workunit.client.1.smithi044.stdout:6/945: dwrite d5/d3f/d50/d79/d96/fa1 [0,4194304] 0 2024-04-22T23:46:24.999 INFO:tasks.workunit.client.1.smithi044.stdout:9/887: write d2/dc/d1d/d3f/d9d/f117 [117039,75530] 0 2024-04-22T23:46:25.001 INFO:tasks.workunit.client.1.smithi044.stdout:2/850: symlink d8/d19/d36/d46/df0/l133 0 2024-04-22T23:46:25.005 INFO:tasks.workunit.client.1.smithi044.stdout:6/946: mkdir d5/d3f/d12b 0 2024-04-22T23:46:25.009 INFO:tasks.workunit.client.1.smithi044.stdout:9/888: rmdir d2/dc/d1d/d34/d38/d4a 39 2024-04-22T23:46:25.016 INFO:tasks.workunit.client.1.smithi044.stdout:2/851: dwrite d8/d19/d36/d46/f67 [0,4194304] 0 2024-04-22T23:46:25.020 INFO:tasks.workunit.client.1.smithi044.stdout:9/889: dread d2/d6/f108 [0,4194304] 0 2024-04-22T23:46:25.022 INFO:tasks.workunit.client.1.smithi044.stdout:9/890: chown d2/dc/d1d/d34/d38/cb6 16 1 2024-04-22T23:46:25.029 INFO:tasks.workunit.client.1.smithi044.stdout:2/852: rename d8/d19/d36/d46/ddb/ca2 to d8/d27/d5b/d110/c134 0 2024-04-22T23:46:25.029 INFO:tasks.workunit.client.1.smithi044.stdout:2/853: stat d8/d19/d117 0 2024-04-22T23:46:25.030 INFO:tasks.workunit.client.1.smithi044.stdout:2/854: readlink d8/d19/d36/d46/d63/d116/lb3 0 2024-04-22T23:46:25.033 INFO:tasks.workunit.client.1.smithi044.stdout:9/891: creat d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/f127 x:0 0 0 2024-04-22T23:46:25.034 INFO:tasks.workunit.client.1.smithi044.stdout:2/855: symlink d8/d19/d117/l135 0 2024-04-22T23:46:25.045 INFO:tasks.workunit.client.1.smithi044.stdout:6/947: dwrite d5/d3f/d50/d79/d96/fac [0,4194304] 0 2024-04-22T23:46:25.180 INFO:tasks.workunit.client.1.smithi044.stdout:2/856: mknod d8/d19/d36/d93/dae/d11e/c136 0 2024-04-22T23:46:25.185 INFO:tasks.workunit.client.1.smithi044.stdout:9/892: rename d2/dc/d1d/d34/d38/d4a/d72/l68 to d2/dc/d1d/d34/d38/d5c/d69/dcb/dfd/l128 0 2024-04-22T23:46:25.191 INFO:tasks.workunit.client.1.smithi044.stdout:2/857: dwrite d8/d19/f131 [0,4194304] 0 2024-04-22T23:46:25.192 INFO:tasks.workunit.client.1.smithi044.stdout:6/948: dwrite d5/d3f/d4c/d40/df9/ffc [0,4194304] 0 2024-04-22T23:46:25.196 INFO:tasks.workunit.client.1.smithi044.stdout:2/858: write d8/d27/d5b/d74/d81/fba [4966614,108370] 0 2024-04-22T23:46:25.198 INFO:tasks.workunit.client.1.smithi044.stdout:9/893: getdents d2/d6/d42/df2/d100 0 2024-04-22T23:46:25.202 INFO:tasks.workunit.client.1.smithi044.stdout:6/949: creat d5/d3f/d50/d79/d96/f12c x:0 0 0 2024-04-22T23:46:25.209 INFO:tasks.workunit.client.1.smithi044.stdout:6/950: fsync d5/d3f/d4c/d23/d3d/f107 0 2024-04-22T23:46:25.209 INFO:tasks.workunit.client.1.smithi044.stdout:2/859: symlink d8/d27/de5/l137 0 2024-04-22T23:46:25.218 INFO:tasks.workunit.client.1.smithi044.stdout:2/860: symlink d8/d27/d5b/d111/l138 0 2024-04-22T23:46:25.220 INFO:tasks.workunit.client.1.smithi044.stdout:6/951: mkdir d5/d3f/d4c/d4f/d117/d115/d12d 0 2024-04-22T23:46:25.221 INFO:tasks.workunit.client.1.smithi044.stdout:6/952: chown d5/d3f/d4c/db4/dc4/fff 16279339 1 2024-04-22T23:46:25.222 INFO:tasks.workunit.client.1.smithi044.stdout:2/861: rename d8/d19/d96/fef to d8/d19/d36/d46/ddb/ddd/df5/f139 0 2024-04-22T23:46:25.222 INFO:tasks.workunit.client.1.smithi044.stdout:6/953: readlink d5/d3f/d4c/d40/d53/l6f 0 2024-04-22T23:46:25.223 INFO:tasks.workunit.client.1.smithi044.stdout:2/862: chown d8/fe 0 1 2024-04-22T23:46:25.226 INFO:tasks.workunit.client.1.smithi044.stdout:6/954: fdatasync d5/d3f/d4c/d23/d3d/d64/fa3 0 2024-04-22T23:46:25.227 INFO:tasks.workunit.client.1.smithi044.stdout:6/955: read - d5/d3f/d4c/db4/dc4/fff zero size 2024-04-22T23:46:25.228 INFO:tasks.workunit.client.1.smithi044.stdout:2/863: write d8/d19/d36/f9f [1866114,25281] 0 2024-04-22T23:46:25.229 INFO:tasks.workunit.client.1.smithi044.stdout:2/864: read d8/d19/f131 [697685,95712] 0 2024-04-22T23:46:25.231 INFO:tasks.workunit.client.1.smithi044.stdout:2/865: fdatasync d8/d19/d117/f11c 0 2024-04-22T23:46:25.237 INFO:tasks.workunit.client.1.smithi044.stdout:2/866: mkdir d8/d19/d36/d46/d63/d65/d6c/d13a 0 2024-04-22T23:46:25.301 INFO:tasks.workunit.client.1.smithi044.stdout:2/867: mkdir d8/d27/d5b/d110/daf/d13b 0 2024-04-22T23:46:25.301 INFO:tasks.workunit.client.1.smithi044.stdout:2/868: stat d8/d19/f51 0 2024-04-22T23:46:25.302 INFO:tasks.workunit.client.1.smithi044.stdout:2/869: truncate d8/d27/d5b/d110/ff8 916071 0 2024-04-22T23:46:25.303 INFO:tasks.workunit.client.1.smithi044.stdout:2/870: write d8/d19/d96/ff3 [801183,89958] 0 2024-04-22T23:46:25.307 INFO:tasks.workunit.client.1.smithi044.stdout:2/871: creat d8/d19/d36/d46/ddb/ddd/df5/f13c x:0 0 0 2024-04-22T23:46:25.313 INFO:tasks.workunit.client.1.smithi044.stdout:2/872: dread d8/d19/d96/fad [0,4194304] 0 2024-04-22T23:46:25.334 INFO:tasks.workunit.client.1.smithi044.stdout:9/894: dwrite d2/f7f [0,4194304] 0 2024-04-22T23:46:25.338 INFO:tasks.workunit.client.1.smithi044.stdout:9/895: dread d2/dc/d94/f55 [0,4194304] 0 2024-04-22T23:46:25.341 INFO:tasks.workunit.client.1.smithi044.stdout:9/896: symlink d2/d6/d42/d8b/ddc/l129 0 2024-04-22T23:46:25.348 INFO:tasks.workunit.client.1.smithi044.stdout:9/897: getdents d2/dc/d94 0 2024-04-22T23:46:25.351 INFO:tasks.workunit.client.1.smithi044.stdout:9/898: mknod d2/dc/d1d/d3f/d9d/c12a 0 2024-04-22T23:46:25.352 INFO:tasks.workunit.client.1.smithi044.stdout:9/899: chown d2/dc/d94/ld 446147028 1 2024-04-22T23:46:25.356 INFO:tasks.workunit.client.1.smithi044.stdout:9/900: creat d2/dc/d1d/d3f/d107/f12b x:0 0 0 2024-04-22T23:46:25.376 INFO:tasks.workunit.client.1.smithi044.stdout:6/956: rename d5/d3f/d50/f6d to d5/d3f/d4c/d40/f12e 0 2024-04-22T23:46:25.379 INFO:tasks.workunit.client.1.smithi044.stdout:6/957: unlink d5/d3f/d4c/d40/d74/daf/ffa 0 2024-04-22T23:46:25.380 INFO:tasks.workunit.client.1.smithi044.stdout:6/958: write d5/d3f/d4c/d4f/d117/f68 [769962,66291] 0 2024-04-22T23:46:25.384 INFO:tasks.workunit.client.1.smithi044.stdout:6/959: creat d5/d3f/d4c/d4f/d117/d28/d75/df0/f12f x:0 0 0 2024-04-22T23:46:25.392 INFO:tasks.workunit.client.1.smithi044.stdout:9/901: write d2/dc/fb0 [1201581,14830] 0 2024-04-22T23:46:25.841 INFO:tasks.workunit.client.1.smithi044.stdout:2/873: sync 2024-04-22T23:46:25.841 INFO:tasks.workunit.client.1.smithi044.stdout:6/960: sync 2024-04-22T23:46:25.841 INFO:tasks.workunit.client.1.smithi044.stdout:9/902: sync 2024-04-22T23:46:25.846 INFO:tasks.workunit.client.1.smithi044.stdout:6/961: write d5/d3f/d4c/d4f/d117/d28/f10a [1351695,126373] 0 2024-04-22T23:46:25.846 INFO:tasks.workunit.client.1.smithi044.stdout:2/874: write d8/d19/d5d/f7e [1415055,64856] 0 2024-04-22T23:46:25.848 INFO:tasks.workunit.client.1.smithi044.stdout:6/962: chown d5/d3f/d50/d122 39227254 1 2024-04-22T23:46:25.852 INFO:tasks.workunit.client.1.smithi044.stdout:9/903: dwrite d2/dc/d1d/d3f/d9d/f112 [0,4194304] 0 2024-04-22T23:46:25.858 INFO:tasks.workunit.client.1.smithi044.stdout:2/875: creat d8/d27/d5b/d74/d80/f13d x:0 0 0 2024-04-22T23:46:25.860 INFO:tasks.workunit.client.1.smithi044.stdout:6/963: dwrite d5/d3f/d4c/f3a [0,4194304] 0 2024-04-22T23:46:25.862 INFO:tasks.workunit.client.1.smithi044.stdout:9/904: mknod d2/dc/d1d/d34/d38/d5c/d69/d9f/c12c 0 2024-04-22T23:46:25.885 INFO:tasks.workunit.client.1.smithi044.stdout:9/905: write d2/d6/f43 [2553573,77405] 0 2024-04-22T23:46:25.886 INFO:tasks.workunit.client.1.smithi044.stdout:6/964: truncate d5/d3f/d50/d79/f81 244208 0 2024-04-22T23:46:25.888 INFO:tasks.workunit.client.1.smithi044.stdout:2/876: write d8/d27/d5b/d110/fcb [391,52684] 0 2024-04-22T23:46:25.889 INFO:tasks.workunit.client.1.smithi044.stdout:6/965: creat d5/d3f/d4c/d40/d53/d56/db7/f130 x:0 0 0 2024-04-22T23:46:25.891 INFO:tasks.workunit.client.1.smithi044.stdout:9/906: mkdir d2/dc/d1d/d34/d38/d5c/d69/d75/d78/d12d 0 2024-04-22T23:46:25.892 INFO:tasks.workunit.client.1.smithi044.stdout:9/907: chown d2/dc/d51/l102 129556 1 2024-04-22T23:46:25.893 INFO:tasks.workunit.client.1.smithi044.stdout:2/877: stat d8/d19/d36/d44/f55 0 2024-04-22T23:46:25.897 INFO:tasks.workunit.client.1.smithi044.stdout:6/966: creat d5/d3f/d4c/d4f/d117/d28/d75/d88/f131 x:0 0 0 2024-04-22T23:46:25.898 INFO:tasks.workunit.client.1.smithi044.stdout:6/967: chown d5/d3f/d4c/d92/d97/lab 876378544 1 2024-04-22T23:46:25.899 INFO:tasks.workunit.client.1.smithi044.stdout:6/968: fsync d5/d3f/d50/d79/f119 0 2024-04-22T23:46:25.905 INFO:tasks.workunit.client.1.smithi044.stdout:6/969: mkdir d5/d3f/d4c/d40/df9/d132 0 2024-04-22T23:46:25.906 INFO:tasks.workunit.client.1.smithi044.stdout:6/970: readlink d5/d3f/d4c/d92/d97/l118 0 2024-04-22T23:46:25.907 INFO:tasks.workunit.client.1.smithi044.stdout:9/908: truncate d2/dc/f66 2749857 0 2024-04-22T23:46:25.910 INFO:tasks.workunit.client.1.smithi044.stdout:2/878: getdents d8/d19/d36/d46/d63/d65/d94 0 2024-04-22T23:46:25.912 INFO:tasks.workunit.client.1.smithi044.stdout:6/971: mknod d5/d3f/d4c/d4f/d117/d28/d75/d88/c133 0 2024-04-22T23:46:25.914 INFO:tasks.workunit.client.1.smithi044.stdout:9/909: unlink d2/d6/fa9 0 2024-04-22T23:46:25.920 INFO:tasks.workunit.client.1.smithi044.stdout:9/910: dread - d2/d6/d42/daa/fcc zero size 2024-04-22T23:46:25.920 INFO:tasks.workunit.client.1.smithi044.stdout:6/972: dwrite d5/d3f/d50/d79/fae [0,4194304] 0 2024-04-22T23:46:25.922 INFO:tasks.workunit.client.1.smithi044.stdout:9/911: write d2/dc/d1d/d34/d38/d4a/dca/f11e [455675,10424] 0 2024-04-22T23:46:25.922 INFO:tasks.workunit.client.1.smithi044.stdout:6/973: read - d5/d3f/d4c/d40/d53/d56/f6e zero size 2024-04-22T23:46:25.925 INFO:tasks.workunit.client.1.smithi044.stdout:9/912: dread - d2/dc/d1d/d34/d38/d5c/d69/d9f/f10d zero size 2024-04-22T23:46:25.925 INFO:tasks.workunit.client.1.smithi044.stdout:2/879: dread d8/f92 [0,4194304] 0 2024-04-22T23:46:25.927 INFO:tasks.workunit.client.1.smithi044.stdout:6/974: fdatasync d5/d3f/d4c/d40/d53/d56/f6e 0 2024-04-22T23:46:25.928 INFO:tasks.workunit.client.1.smithi044.stdout:2/880: mknod d8/d19/d36/d46/c13e 0 2024-04-22T23:46:25.934 INFO:tasks.workunit.client.1.smithi044.stdout:9/913: rename d2/dc/d1d/d3f/de3 to d2/dc/d1d/d34/d38/d5c/d7b/d12e 0 2024-04-22T23:46:25.934 INFO:tasks.workunit.client.1.smithi044.stdout:6/975: dwrite d5/d3f/d4c/d40/db9/dc0/f11f [0,4194304] 0 2024-04-22T23:46:25.936 INFO:tasks.workunit.client.1.smithi044.stdout:6/976: fdatasync d5/d3f/d4c/d92/d97/d129/f9f 0 2024-04-22T23:46:25.939 INFO:tasks.workunit.client.1.smithi044.stdout:9/914: dread d2/dc/d1d/d3f/d9d/f117 [0,4194304] 0 2024-04-22T23:46:25.940 INFO:tasks.workunit.client.1.smithi044.stdout:9/915: chown d2/dc/d1d/d3f/c86 11912 1 2024-04-22T23:46:25.949 INFO:tasks.workunit.client.1.smithi044.stdout:2/881: write d8/d27/f56 [284370,73891] 0 2024-04-22T23:46:25.951 INFO:tasks.workunit.client.1.smithi044.stdout:2/882: stat d8/d19/d36/d46/ddb/ddd/df5/l121 0 2024-04-22T23:46:25.957 INFO:tasks.workunit.client.1.smithi044.stdout:9/916: rename d2/dc/d1d/d3f/f4c to d2/f12f 0 2024-04-22T23:46:25.957 INFO:tasks.workunit.client.1.smithi044.stdout:2/883: dwrite d8/d19/d36/f9f [4194304,4194304] 0 2024-04-22T23:46:25.958 INFO:tasks.workunit.client.1.smithi044.stdout:6/977: write d5/d30/f80 [1120583,3675] 0 2024-04-22T23:46:25.959 INFO:tasks.workunit.client.1.smithi044.stdout:9/917: chown d2/d6/d42/daa/ff9 435 1 2024-04-22T23:46:25.959 INFO:tasks.workunit.client.1.smithi044.stdout:6/978: chown d5/d3f/d4c/d40/f78 3112 1 2024-04-22T23:46:25.962 INFO:tasks.workunit.client.1.smithi044.stdout:2/884: mkdir d8/d19/d36/d93/dae/d13f 0 2024-04-22T23:46:25.963 INFO:tasks.workunit.client.1.smithi044.stdout:6/979: rmdir d5/d3f/d4c/d40/db9/dc0 39 2024-04-22T23:46:25.965 INFO:tasks.workunit.client.1.smithi044.stdout:6/980: write d5/d30/f80 [2046818,114109] 0 2024-04-22T23:46:25.966 INFO:tasks.workunit.client.1.smithi044.stdout:9/918: unlink d2/dc/d1d/d34/d38/d5c/d69/d75/fef 0 2024-04-22T23:46:25.968 INFO:tasks.workunit.client.1.smithi044.stdout:9/919: stat d2/dc/d1d/d34/d38/d4a/db7/l11b 0 2024-04-22T23:46:25.973 INFO:tasks.workunit.client.1.smithi044.stdout:6/981: dwrite d5/d3f/d4c/d23/f9b [0,4194304] 0 2024-04-22T23:46:25.977 INFO:tasks.workunit.client.1.smithi044.stdout:2/885: mkdir d8/d19/d36/d93/dae/d13f/d140 0 2024-04-22T23:46:25.979 INFO:tasks.workunit.client.1.smithi044.stdout:9/920: dwrite d2/dc/d1d/d3f/d9d/f112 [0,4194304] 0 2024-04-22T23:46:25.985 INFO:tasks.workunit.client.1.smithi044.stdout:2/886: mkdir d8/d19/d36/d93/dae/d108/d141 0 2024-04-22T23:46:25.988 INFO:tasks.workunit.client.1.smithi044.stdout:9/921: symlink d2/dc/d1d/d34/d38/d5c/d7b/d12e/l130 0 2024-04-22T23:46:25.995 INFO:tasks.workunit.client.1.smithi044.stdout:9/922: getdents d2/dc/d1d/d34/d38/d4a/db7/d123 0 2024-04-22T23:46:25.997 INFO:tasks.workunit.client.1.smithi044.stdout:9/923: chown d2/d6/d42/d8b/ddc/l129 111869 1 2024-04-22T23:46:25.997 INFO:tasks.workunit.client.1.smithi044.stdout:6/982: dwrite d5/f21 [0,4194304] 0 2024-04-22T23:46:26.001 INFO:tasks.workunit.client.1.smithi044.stdout:6/983: mknod d5/d3f/d4c/c134 0 2024-04-22T23:46:26.004 INFO:tasks.workunit.client.1.smithi044.stdout:6/984: mkdir d5/d3f/d4c/d4f/d135 0 2024-04-22T23:46:26.005 INFO:tasks.workunit.client.1.smithi044.stdout:9/924: link d2/dc/d1d/d34/ce2 d2/dc/d1d/d34/d38/d4a/db7/d123/c131 0 2024-04-22T23:46:26.010 INFO:tasks.workunit.client.1.smithi044.stdout:9/925: creat d2/dc/d1d/d34/d38/d4a/db7/d123/f132 x:0 0 0 2024-04-22T23:46:26.012 INFO:tasks.workunit.client.1.smithi044.stdout:6/985: dwrite d5/d3f/d4c/d4f/d117/f6a [4194304,4194304] 0 2024-04-22T23:46:26.015 INFO:tasks.workunit.client.1.smithi044.stdout:9/926: truncate d2/d6/d42/d8b/f98 3012173 0 2024-04-22T23:46:26.017 INFO:tasks.workunit.client.1.smithi044.stdout:6/986: mknod d5/d3f/d4c/d4f/d117/d28/d75/d88/d128/c136 0 2024-04-22T23:46:26.019 INFO:tasks.workunit.client.1.smithi044.stdout:6/987: stat d5/d3f/d4c/d40/d53/f8e 0 2024-04-22T23:46:26.019 INFO:tasks.workunit.client.1.smithi044.stdout:2/887: dread d8/d27/f4b [0,4194304] 0 2024-04-22T23:46:26.021 INFO:tasks.workunit.client.1.smithi044.stdout:2/888: truncate d8/d19/d36/d44/dd8/fea 488680 0 2024-04-22T23:46:26.023 INFO:tasks.workunit.client.1.smithi044.stdout:2/889: creat d8/d27/d5b/d74/d81/f142 x:0 0 0 2024-04-22T23:46:26.025 INFO:tasks.workunit.client.1.smithi044.stdout:2/890: truncate d8/d27/f28 1093985 0 2024-04-22T23:46:26.025 INFO:tasks.workunit.client.1.smithi044.stdout:6/988: unlink d5/d3f/c48 0 2024-04-22T23:46:26.034 INFO:tasks.workunit.client.1.smithi044.stdout:2/891: dwrite d8/d19/d36/d46/d63/d65/d94/f11d [0,4194304] 0 2024-04-22T23:46:26.038 INFO:tasks.workunit.client.1.smithi044.stdout:2/892: symlink d8/d19/d36/d46/df0/d12a/l143 0 2024-04-22T23:46:26.041 INFO:tasks.workunit.client.1.smithi044.stdout:2/893: symlink d8/d19/d96/l144 0 2024-04-22T23:46:26.048 INFO:tasks.workunit.client.1.smithi044.stdout:2/894: unlink d8/d19/d36/d46/ddb/ddd/df5/l121 0 2024-04-22T23:46:26.049 INFO:tasks.workunit.client.1.smithi044.stdout:2/895: chown d8/d19/d36/d93/dae/de8/feb 47 1 2024-04-22T23:46:26.061 INFO:tasks.workunit.client.1.smithi044.stdout:6/989: truncate d5/f4d 4164008 0 2024-04-22T23:46:26.068 INFO:tasks.workunit.client.1.smithi044.stdout:6/990: dwrite d5/d3f/d50/d79/d96/fa1 [0,4194304] 0 2024-04-22T23:46:26.068 INFO:tasks.workunit.client.1.smithi044.stdout:6/991: stat d5/d3f/d4c/lcc 0 2024-04-22T23:46:26.127 INFO:tasks.workunit.client.1.smithi044.stdout:2/896: write d8/d27/f4b [4839297,121667] 0 2024-04-22T23:46:26.128 INFO:tasks.workunit.client.1.smithi044.stdout:6/992: write f1 [8626772,107559] 0 2024-04-22T23:46:26.133 INFO:tasks.workunit.client.1.smithi044.stdout:6/993: dread - d5/d3f/d4c/d23/d3d/fcf zero size 2024-04-22T23:46:26.134 INFO:tasks.workunit.client.1.smithi044.stdout:2/897: symlink d8/d19/d36/d44/l145 0 2024-04-22T23:46:26.135 INFO:tasks.workunit.client.1.smithi044.stdout:6/994: symlink d5/d3f/d50/l137 0 2024-04-22T23:46:26.137 INFO:tasks.workunit.client.1.smithi044.stdout:2/898: unlink d8/d19/d36/d46/f67 0 2024-04-22T23:46:26.139 INFO:tasks.workunit.client.1.smithi044.stdout:6/995: write d5/d3f/d4c/d4f/d117/d67/f24 [3466871,119216] 0 2024-04-22T23:46:26.145 INFO:tasks.workunit.client.1.smithi044.stdout:2/899: getdents d8/d19/d5d/db0/dff 0 2024-04-22T23:46:26.150 INFO:tasks.workunit.client.1.smithi044.stdout:2/900: symlink d8/d27/d5b/d110/l146 0 2024-04-22T23:46:26.152 INFO:tasks.workunit.client.1.smithi044.stdout:2/901: fsync d8/d19/d5d/fec 0 2024-04-22T23:46:26.154 INFO:tasks.workunit.client.1.smithi044.stdout:2/902: mknod d8/d19/d36/d93/dae/d13f/c147 0 2024-04-22T23:46:26.156 INFO:tasks.workunit.client.1.smithi044.stdout:2/903: fdatasync d8/f1a 0 2024-04-22T23:46:26.160 INFO:tasks.workunit.client.1.smithi044.stdout:2/904: rename d8/d19/d36/d44/dd8 to d8/d27/d5b/d74/d80/d148 0 2024-04-22T23:46:26.161 INFO:tasks.workunit.client.1.smithi044.stdout:2/905: dread - d8/d19/d117/f11c zero size 2024-04-22T23:46:26.164 INFO:tasks.workunit.client.1.smithi044.stdout:2/906: creat d8/d19/d36/d46/d11b/f149 x:0 0 0 2024-04-22T23:46:26.166 INFO:tasks.workunit.client.1.smithi044.stdout:2/907: read d8/d19/d5d/f7c [126963,98282] 0 2024-04-22T23:46:26.171 INFO:tasks.workunit.client.1.smithi044.stdout:2/908: mknod d8/d19/d36/d46/df0/c14a 0 2024-04-22T23:46:26.173 INFO:tasks.workunit.client.1.smithi044.stdout:2/909: creat d8/d19/d36/d46/d11b/f14b x:0 0 0 2024-04-22T23:46:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:26 smithi031 ceph-mon[26721]: pgmap v16: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 63 MiB/s rd, 169 MiB/s wr, 411 op/s 2024-04-22T23:46:26.431 INFO:tasks.workunit.client.1.smithi044.stdout:9/927: sync 2024-04-22T23:46:26.433 INFO:tasks.workunit.client.1.smithi044.stdout:9/928: mkdir d2/d6/d133 0 2024-04-22T23:46:26.433 INFO:tasks.workunit.client.1.smithi044.stdout:9/929: fsync d2/f7f 0 2024-04-22T23:46:26.437 INFO:tasks.workunit.client.1.smithi044.stdout:9/930: rename d2/d6/d4d/l8a to d2/d6/d42/d8b/l134 0 2024-04-22T23:46:26.439 INFO:tasks.workunit.client.1.smithi044.stdout:9/931: dread d2/dc/d94/f55 [0,4194304] 0 2024-04-22T23:46:26.439 INFO:tasks.workunit.client.1.smithi044.stdout:9/932: chown d2/d6/f108 1 1 2024-04-22T23:46:26.447 INFO:tasks.workunit.client.1.smithi044.stdout:9/933: dwrite d2/dc/d1d/d34/d8e/dbd/f119 [0,4194304] 0 2024-04-22T23:46:26.457 INFO:tasks.workunit.client.1.smithi044.stdout:9/934: getdents d2/d6/d42/d8b 0 2024-04-22T23:46:26.460 INFO:tasks.workunit.client.1.smithi044.stdout:9/935: chown d2/d6/d42/df2/lf6 1 1 2024-04-22T23:46:26.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:26 smithi044 ceph-mon[34463]: pgmap v16: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 63 MiB/s rd, 169 MiB/s wr, 411 op/s 2024-04-22T23:46:26.462 INFO:tasks.workunit.client.1.smithi044.stdout:9/936: symlink d2/d6/d42/daa/l135 0 2024-04-22T23:46:26.463 INFO:tasks.workunit.client.1.smithi044.stdout:9/937: chown d2/dc/d1d/d34/d8e/dbd/dd1/d11a 218023 1 2024-04-22T23:46:26.465 INFO:tasks.workunit.client.1.smithi044.stdout:9/938: rmdir d2/d6/d42/d8b 39 2024-04-22T23:46:26.466 INFO:tasks.workunit.client.1.smithi044.stdout:9/939: fdatasync d2/d6/d42/daa/fae 0 2024-04-22T23:46:26.472 INFO:tasks.workunit.client.1.smithi044.stdout:9/940: truncate d2/dc/f18 5108707 0 2024-04-22T23:46:26.473 INFO:tasks.workunit.client.1.smithi044.stdout:9/941: readlink d2/dc/d51/lc0 0 2024-04-22T23:46:26.474 INFO:tasks.workunit.client.1.smithi044.stdout:9/942: creat d2/dc/d1d/d34/d38/d5c/d69/d75/d78/f136 x:0 0 0 2024-04-22T23:46:26.476 INFO:tasks.workunit.client.1.smithi044.stdout:9/943: write d2/dc/d1d/d34/f9c [2197492,128377] 0 2024-04-22T23:46:26.477 INFO:tasks.workunit.client.1.smithi044.stdout:9/944: chown d2/dc/d1d/d34/d38/d5c/d69/d75/cff 272 1 2024-04-22T23:46:26.479 INFO:tasks.workunit.client.1.smithi044.stdout:9/945: mknod d2/d6/d133/c137 0 2024-04-22T23:46:26.480 INFO:tasks.workunit.client.1.smithi044.stdout:9/946: dread - d2/dc/d1d/d34/d38/d5c/d69/dcb/ff4 zero size 2024-04-22T23:46:26.481 INFO:tasks.workunit.client.1.smithi044.stdout:9/947: chown d2/dc/d1d/d34/d38/d5c/d69/d75/fb5 5814 1 2024-04-22T23:46:26.482 INFO:tasks.workunit.client.1.smithi044.stdout:9/948: fsync d2/dc/d1d/d34/d38/d4a/d72/f8f 0 2024-04-22T23:46:26.491 INFO:tasks.workunit.client.1.smithi044.stdout:9/949: link d2/dc/d1d/d34/d38/d5c/d69/dcb/ff4 d2/dc/d1d/d34/d38/d5c/d69/dcb/dfd/f138 0 2024-04-22T23:46:26.493 INFO:tasks.workunit.client.1.smithi044.stdout:9/950: truncate d2/dc/d1d/d34/d38/d5c/d69/d75/f7c 240607 0 2024-04-22T23:46:26.496 INFO:tasks.workunit.client.1.smithi044.stdout:2/910: sync 2024-04-22T23:46:26.497 INFO:tasks.workunit.client.1.smithi044.stdout:6/996: sync 2024-04-22T23:46:26.498 INFO:tasks.workunit.client.1.smithi044.stdout:9/951: symlink d2/dc/d1d/d34/d38/d5c/d69/d75/l139 0 2024-04-22T23:46:26.498 INFO:tasks.workunit.client.1.smithi044.stdout:2/911: readlink d8/d19/d36/d46/ddb/ddd/df5/lf9 0 2024-04-22T23:46:26.502 INFO:tasks.workunit.client.1.smithi044.stdout:6/997: mknod d5/d3f/d50/d79/df8/c138 0 2024-04-22T23:46:26.502 INFO:tasks.workunit.client.1.smithi044.stdout:2/912: dread d8/d19/d36/d44/f109 [0,4194304] 0 2024-04-22T23:46:26.503 INFO:tasks.workunit.client.1.smithi044.stdout:6/998: chown d5/d3f/d4c/d4f/d117/d28/d75/fee 116788188 1 2024-04-22T23:46:26.507 INFO:tasks.workunit.client.1.smithi044.stdout:9/952: unlink d2/dc/f66 0 2024-04-22T23:46:26.509 INFO:tasks.workunit.client.1.smithi044.stdout:6/999: symlink d5/d30/l139 0 2024-04-22T23:46:26.513 INFO:tasks.workunit.client.1.smithi044.stdout:9/953: write d2/d6/d42/d8b/f106 [583460,70624] 0 2024-04-22T23:46:26.516 INFO:tasks.workunit.client.1.smithi044.stdout:2/913: mknod d8/d19/c14c 0 2024-04-22T23:46:26.520 INFO:tasks.workunit.client.1.smithi044.stdout:9/954: rename d2/d6/d42/d8b/ddc/fe4 to d2/dc/d1d/f13a 0 2024-04-22T23:46:26.521 INFO:tasks.workunit.client.1.smithi044.stdout:2/914: mknod d8/d19/d36/d93/dae/d13f/d140/c14d 0 2024-04-22T23:46:26.526 INFO:tasks.workunit.client.1.smithi044.stdout:9/955: creat d2/dc/f13b x:0 0 0 2024-04-22T23:46:26.536 INFO:tasks.workunit.client.1.smithi044.stdout:9/956: write d2/dc/d94/f30 [471969,118169] 0 2024-04-22T23:46:26.537 INFO:tasks.workunit.client.1.smithi044.stdout:2/915: dwrite d8/d19/d36/d46/ddb/ddd/fde [0,4194304] 0 2024-04-22T23:46:26.540 INFO:tasks.workunit.client.1.smithi044.stdout:9/957: readlink d2/dc/d1d/d34/d38/d4a/db7/d123/lb2 0 2024-04-22T23:46:26.540 INFO:tasks.workunit.client.1.smithi044.stdout:2/916: mknod d8/d60/c14e 0 2024-04-22T23:46:26.547 INFO:tasks.workunit.client.1.smithi044.stdout:9/958: dwrite d2/f3 [4194304,4194304] 0 2024-04-22T23:46:26.563 INFO:tasks.workunit.client.1.smithi044.stdout:2/917: write d8/d19/f51 [247153,21466] 0 2024-04-22T23:46:26.565 INFO:tasks.workunit.client.1.smithi044.stdout:2/918: chown d8/d27/d5b/fb5 21175 1 2024-04-22T23:46:26.566 INFO:tasks.workunit.client.1.smithi044.stdout:9/959: dread d2/dc/d94/fc2 [0,4194304] 0 2024-04-22T23:46:26.567 INFO:tasks.workunit.client.1.smithi044.stdout:2/919: dread d8/d27/d5b/d110/daf/dd0/f106 [0,4194304] 0 2024-04-22T23:46:26.568 INFO:tasks.workunit.client.1.smithi044.stdout:2/920: chown d8/d60/ffa 38535 1 2024-04-22T23:46:26.571 INFO:tasks.workunit.client.1.smithi044.stdout:2/921: dread d8/d19/f51 [0,4194304] 0 2024-04-22T23:46:26.571 INFO:tasks.workunit.client.1.smithi044.stdout:9/960: rename d2/d6/d42/la2 to d2/d6/d4d/l13c 0 2024-04-22T23:46:26.580 INFO:tasks.workunit.client.1.smithi044.stdout:9/961: link d2/dc/d1d/d34/d38/d5c/d69/d7a/deb/fec d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/f13d 0 2024-04-22T23:46:26.582 INFO:tasks.workunit.client.1.smithi044.stdout:9/962: creat d2/dc/d1d/d34/d38/d5c/d69/dcb/f13e x:0 0 0 2024-04-22T23:46:26.583 INFO:tasks.workunit.client.1.smithi044.stdout:2/922: getdents d8/d27/d5b 0 2024-04-22T23:46:26.584 INFO:tasks.workunit.client.1.smithi044.stdout:2/923: write d8/d27/d5b/d110/f1d [1150238,4375] 0 2024-04-22T23:46:26.586 INFO:tasks.workunit.client.1.smithi044.stdout:2/924: write d8/d19/d36/d93/fbc [2256525,36694] 0 2024-04-22T23:46:26.591 INFO:tasks.workunit.client.1.smithi044.stdout:9/963: dwrite d2/dc/d1d/d34/d8e/dbd/f10e [0,4194304] 0 2024-04-22T23:46:26.596 INFO:tasks.workunit.client.1.smithi044.stdout:9/964: fsync d2/d6/d42/d8b/f98 0 2024-04-22T23:46:26.601 INFO:tasks.workunit.client.1.smithi044.stdout:9/965: mknod d2/dc/d1d/d34/d38/d4a/da6/c13f 0 2024-04-22T23:46:26.604 INFO:tasks.workunit.client.1.smithi044.stdout:9/966: mkdir d2/dc/d1d/d34/d38/d5c/d69/dcb/d140 0 2024-04-22T23:46:26.605 INFO:tasks.workunit.client.1.smithi044.stdout:9/967: chown d2/dc/d1d/d34/d38/d4a/db7/d123/cf 158857039 1 2024-04-22T23:46:26.606 INFO:tasks.workunit.client.1.smithi044.stdout:9/968: chown d2/dc/d1d/d34/d8e/dbd/dd1/d11a 14 1 2024-04-22T23:46:26.610 INFO:tasks.workunit.client.1.smithi044.stdout:9/969: mkdir d2/dc/d1d/d34/d8e/d141 0 2024-04-22T23:46:26.618 INFO:tasks.workunit.client.1.smithi044.stdout:9/970: write d2/dc/d1d/d3f/d9d/f113 [251174,44413] 0 2024-04-22T23:46:26.620 INFO:tasks.workunit.client.1.smithi044.stdout:9/971: symlink d2/dc/d1d/d34/d38/d5c/d69/dcb/dfd/l142 0 2024-04-22T23:46:26.622 INFO:tasks.workunit.client.1.smithi044.stdout:9/972: write d2/dc/d1d/d34/d38/d5c/d69/d75/f7c [1207432,94008] 0 2024-04-22T23:46:26.626 INFO:tasks.workunit.client.1.smithi044.stdout:9/973: rename d2/dc/d1d/d34/f92 to d2/dc/d1d/d3f/d9d/f143 0 2024-04-22T23:46:26.628 INFO:tasks.workunit.client.1.smithi044.stdout:9/974: mknod d2/dc/d1d/d34/d38/d4a/db7/c144 0 2024-04-22T23:46:26.630 INFO:tasks.workunit.client.1.smithi044.stdout:9/975: truncate d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/f127 988047 0 2024-04-22T23:46:26.635 INFO:tasks.workunit.client.1.smithi044.stdout:9/976: dread d2/dc/d1d/d34/d8e/dbd/fee [0,4194304] 0 2024-04-22T23:46:26.656 INFO:tasks.workunit.client.1.smithi044.stdout:9/977: dwrite d2/fa3 [0,4194304] 0 2024-04-22T23:46:26.658 INFO:tasks.workunit.client.1.smithi044.stdout:9/978: rmdir d2/d6/d42/d8b/ddc 39 2024-04-22T23:46:26.659 INFO:tasks.workunit.client.1.smithi044.stdout:9/979: chown d2/c31 376 1 2024-04-22T23:46:26.661 INFO:tasks.workunit.client.1.smithi044.stdout:9/980: read - d2/dc/d1d/d34/d38/d5c/d69/d75/d78/f120 zero size 2024-04-22T23:46:26.665 INFO:tasks.workunit.client.1.smithi044.stdout:9/981: symlink d2/dc/d1d/d34/d38/d5c/l145 0 2024-04-22T23:46:26.823 INFO:tasks.workunit.client.1.smithi044.stdout:2/925: sync 2024-04-22T23:46:26.824 INFO:tasks.workunit.client.1.smithi044.stdout:2/926: fdatasync d8/d27/d5b/d110/daf/f125 0 2024-04-22T23:46:26.829 INFO:tasks.workunit.client.1.smithi044.stdout:2/927: dread d8/d19/d36/d46/d63/de0/ffb [0,4194304] 0 2024-04-22T23:46:26.865 INFO:tasks.workunit.client.1.smithi044.stdout:2/928: write d8/d19/f1f [244526,53306] 0 2024-04-22T23:46:26.866 INFO:tasks.workunit.client.1.smithi044.stdout:2/929: readlink d8/d19/d36/d46/d63/d116/l114 0 2024-04-22T23:46:26.871 INFO:tasks.workunit.client.1.smithi044.stdout:2/930: write d8/d27/d5b/f9a [190772,88205] 0 2024-04-22T23:46:26.887 INFO:tasks.workunit.client.1.smithi044.stdout:2/931: dwrite d8/d27/d5b/d74/d80/fc0 [0,4194304] 0 2024-04-22T23:46:26.889 INFO:tasks.workunit.client.1.smithi044.stdout:9/982: sync 2024-04-22T23:46:26.893 INFO:tasks.workunit.client.1.smithi044.stdout:2/932: rename d8/fc to d8/d27/d5b/d111/f14f 0 2024-04-22T23:46:26.898 INFO:tasks.workunit.client.1.smithi044.stdout:2/933: chown d8/d19/d96/l105 67 1 2024-04-22T23:46:26.898 INFO:tasks.workunit.client.1.smithi044.stdout:9/983: symlink d2/dc/d1d/d34/d8e/d141/l146 0 2024-04-22T23:46:26.900 INFO:tasks.workunit.client.1.smithi044.stdout:9/984: stat d2/d6/d42/daa/l110 0 2024-04-22T23:46:26.903 INFO:tasks.workunit.client.1.smithi044.stdout:2/934: creat d8/f150 x:0 0 0 2024-04-22T23:46:26.904 INFO:tasks.workunit.client.1.smithi044.stdout:2/935: fdatasync d8/d27/f4b 0 2024-04-22T23:46:26.905 INFO:tasks.workunit.client.1.smithi044.stdout:9/985: symlink d2/dc/d1d/d34/d38/d4a/db7/d123/l147 0 2024-04-22T23:46:26.906 INFO:tasks.workunit.client.1.smithi044.stdout:2/936: read d8/d19/d36/d46/ddb/fca [3598710,22592] 0 2024-04-22T23:46:26.906 INFO:tasks.workunit.client.1.smithi044.stdout:9/986: write d2/dc/d1d/f103 [642167,104244] 0 2024-04-22T23:46:26.908 INFO:tasks.workunit.client.1.smithi044.stdout:9/987: write d2/dc/d1d/d34/d38/d4a/d72/f8f [2310,26282] 0 2024-04-22T23:46:26.910 INFO:tasks.workunit.client.1.smithi044.stdout:2/937: truncate d8/d19/d36/d44/f109 891930 0 2024-04-22T23:46:26.921 INFO:tasks.workunit.client.1.smithi044.stdout:9/988: getdents d2/dc/d1d/d34/d38/d5c/d69/d75/d78 0 2024-04-22T23:46:26.928 INFO:tasks.workunit.client.1.smithi044.stdout:2/938: dwrite d8/d27/d5b/d110/daf/dd0/f106 [0,4194304] 0 2024-04-22T23:46:26.933 INFO:tasks.workunit.client.1.smithi044.stdout:9/989: truncate d2/d6/fa 1422501 0 2024-04-22T23:46:26.938 INFO:tasks.workunit.client.1.smithi044.stdout:2/939: getdents d8/d19/d96 0 2024-04-22T23:46:26.940 INFO:tasks.workunit.client.1.smithi044.stdout:9/990: dread d2/dc/d1d/d34/d38/d5c/d69/d7a/da7/f13d [0,4194304] 0 2024-04-22T23:46:26.950 INFO:tasks.workunit.client.1.smithi044.stdout:9/991: truncate d2/dc/d1d/d34/d38/d5c/d69/d7a/deb/fec 1324670 0 2024-04-22T23:46:26.955 INFO:tasks.workunit.client.1.smithi044.stdout:9/992: dread d2/dc/d1d/d34/d38/d5c/f76 [0,4194304] 0 2024-04-22T23:46:26.957 INFO:tasks.workunit.client.1.smithi044.stdout:2/940: dread d8/f1a [0,4194304] 0 2024-04-22T23:46:26.959 INFO:tasks.workunit.client.1.smithi044.stdout:9/993: mkdir d2/dc/d1d/d34/d8e/d141/d148 0 2024-04-22T23:46:26.961 INFO:tasks.workunit.client.1.smithi044.stdout:9/994: truncate d2/dc/d94/f50 2589931 0 2024-04-22T23:46:26.963 INFO:tasks.workunit.client.1.smithi044.stdout:2/941: mknod d8/d19/d36/d44/c151 0 2024-04-22T23:46:26.966 INFO:tasks.workunit.client.1.smithi044.stdout:2/942: mknod d8/d27/d5b/d110/daf/c152 0 2024-04-22T23:46:26.966 INFO:tasks.workunit.client.1.smithi044.stdout:9/995: truncate d2/dc/d1d/d34/d38/d5c/d69/d75/d78/f136 1032591 0 2024-04-22T23:46:26.967 INFO:tasks.workunit.client.1.smithi044.stdout:2/943: chown d8/d27/d5b/d110/daf 1628440 1 2024-04-22T23:46:26.967 INFO:tasks.workunit.client.1.smithi044.stdout:2/944: fdatasync d8/f11 0 2024-04-22T23:46:26.969 INFO:tasks.workunit.client.1.smithi044.stdout:9/996: dread d2/dc/d1d/d3f/d9d/f113 [0,4194304] 0 2024-04-22T23:46:26.982 INFO:tasks.workunit.client.1.smithi044.stdout:2/945: link d8/d19/d36/d93/dbf/fd5 d8/d19/d36/d46/d63/f153 0 2024-04-22T23:46:26.983 INFO:tasks.workunit.client.1.smithi044.stdout:9/997: dread d2/dc/fb0 [0,4194304] 0 2024-04-22T23:46:26.983 INFO:tasks.workunit.client.1.smithi044.stdout:2/946: stat d8/d19/d96/fc6 0 2024-04-22T23:46:26.988 INFO:tasks.workunit.client.1.smithi044.stdout:9/998: creat d2/dc/d1d/d3f/f149 x:0 0 0 2024-04-22T23:46:26.989 INFO:tasks.workunit.client.1.smithi044.stdout:2/947: symlink d8/d19/d117/l154 0 2024-04-22T23:46:26.994 INFO:tasks.workunit.client.1.smithi044.stdout:9/999: mknod d2/dc/d1d/d34/d38/d5c/d69/c14a 0 2024-04-22T23:46:26.999 INFO:tasks.workunit.client.1.smithi044.stdout:2/948: write d8/d19/d96/fad [1579437,34031] 0 2024-04-22T23:46:27.251 INFO:tasks.workunit.client.1.smithi044.stdout:2/949: sync 2024-04-22T23:46:27.254 INFO:tasks.workunit.client.1.smithi044.stdout:2/950: creat d8/d19/f155 x:0 0 0 2024-04-22T23:46:27.255 INFO:tasks.workunit.client.1.smithi044.stdout:2/951: write d8/d27/f6f [936837,80040] 0 2024-04-22T23:46:27.262 INFO:tasks.workunit.client.1.smithi044.stdout:2/952: write d8/d19/d5d/f5f [1627424,9486] 0 2024-04-22T23:46:27.263 INFO:tasks.workunit.client.1.smithi044.stdout:2/953: fsync d8/d19/d36/d46/ddb/ddd/df5/f13c 0 2024-04-22T23:46:27.266 INFO:tasks.workunit.client.1.smithi044.stdout:2/954: mknod d8/d19/d36/d93/dae/de8/c156 0 2024-04-22T23:46:27.268 INFO:tasks.workunit.client.1.smithi044.stdout:2/955: mknod d8/d19/d36/d46/ddb/ddd/df5/c157 0 2024-04-22T23:46:27.273 INFO:tasks.workunit.client.1.smithi044.stdout:2/956: dread d8/d27/d5b/d74/d81/fbb [0,4194304] 0 2024-04-22T23:46:27.277 INFO:tasks.workunit.client.1.smithi044.stdout:2/957: creat d8/d27/f158 x:0 0 0 2024-04-22T23:46:27.279 INFO:tasks.workunit.client.1.smithi044.stdout:2/958: mkdir d8/d27/d5b/d110/daf/d13b/d159 0 2024-04-22T23:46:27.281 INFO:tasks.workunit.client.1.smithi044.stdout:2/959: rmdir d8/d19/d96 39 2024-04-22T23:46:27.482 INFO:tasks.workunit.client.1.smithi044.stdout:2/960: sync 2024-04-22T23:46:27.488 INFO:tasks.workunit.client.1.smithi044.stdout:2/961: getdents d8/d19/d36/d46/d63/de0 0 2024-04-22T23:46:27.490 INFO:tasks.workunit.client.1.smithi044.stdout:2/962: creat d8/d19/d36/d46/ddb/ddd/df5/f15a x:0 0 0 2024-04-22T23:46:27.491 INFO:tasks.workunit.client.1.smithi044.stdout:2/963: chown d8/d19/d36/d46/ddb/ddd/df5/lf9 75172 1 2024-04-22T23:46:27.507 INFO:tasks.workunit.client.1.smithi044.stdout:2/964: rmdir d8/d19/d36 39 2024-04-22T23:46:27.514 INFO:tasks.workunit.client.1.smithi044.stdout:2/965: mkdir d8/d19/d36/d46/df0/d15b 0 2024-04-22T23:46:27.515 INFO:tasks.workunit.client.1.smithi044.stdout:2/966: read d8/d27/d5b/d74/d81/fbb [742432,119725] 0 2024-04-22T23:46:27.516 INFO:tasks.workunit.client.1.smithi044.stdout:2/967: write d8/d27/f4f [335748,111745] 0 2024-04-22T23:46:27.517 INFO:tasks.workunit.client.1.smithi044.stdout:2/968: readlink d8/d19/d117/l154 0 2024-04-22T23:46:27.521 INFO:tasks.workunit.client.1.smithi044.stdout:2/969: symlink d8/d19/d5d/db0/dff/l15c 0 2024-04-22T23:46:27.531 INFO:tasks.workunit.client.1.smithi044.stdout:2/970: link d8/d19/d5d/db0/dff/l11f d8/d19/d36/d44/l15d 0 2024-04-22T23:46:27.533 INFO:tasks.workunit.client.1.smithi044.stdout:2/971: fdatasync d8/d19/d36/d46/d63/d65/f84 0 2024-04-22T23:46:27.534 INFO:tasks.workunit.client.1.smithi044.stdout:2/972: stat d8/d27/f158 0 2024-04-22T23:46:27.539 INFO:tasks.workunit.client.1.smithi044.stdout:2/973: creat d8/d19/d36/d93/dae/d108/d141/f15e x:0 0 0 2024-04-22T23:46:27.541 INFO:tasks.workunit.client.1.smithi044.stdout:2/974: mkdir d8/d27/d5b/d110/daf/d13b/d15f 0 2024-04-22T23:46:27.546 INFO:tasks.workunit.client.1.smithi044.stdout:2/975: getdents d8/d27/d5b/d110 0 2024-04-22T23:46:27.549 INFO:tasks.workunit.client.1.smithi044.stdout:2/976: creat d8/d19/d36/d46/df0/d15b/f160 x:0 0 0 2024-04-22T23:46:27.550 INFO:tasks.workunit.client.1.smithi044.stdout:2/977: chown d8 157 1 2024-04-22T23:46:27.571 INFO:tasks.workunit.client.1.smithi044.stdout:2/978: dwrite d8/d19/d96/f73 [0,4194304] 0 2024-04-22T23:46:27.580 INFO:tasks.workunit.client.1.smithi044.stdout:2/979: write d8/d27/d5b/d74/f79 [947136,36274] 0 2024-04-22T23:46:27.603 INFO:tasks.workunit.client.1.smithi044.stdout:2/980: dread d8/d27/d5b/d74/fab [0,4194304] 0 2024-04-22T23:46:27.604 INFO:tasks.workunit.client.1.smithi044.stdout:2/981: chown d8/d19/d36/d46/df0/d15b/f160 5871 1 2024-04-22T23:46:27.614 INFO:tasks.workunit.client.1.smithi044.stdout:2/982: dwrite d8/d19/d36/d46/fe4 [0,4194304] 0 2024-04-22T23:46:27.620 INFO:tasks.workunit.client.1.smithi044.stdout:2/983: dwrite d8/d19/d96/f10d [0,4194304] 0 2024-04-22T23:46:27.625 INFO:tasks.workunit.client.1.smithi044.stdout:2/984: dread d8/d19/d36/f3d [0,4194304] 0 2024-04-22T23:46:27.628 INFO:tasks.workunit.client.1.smithi044.stdout:2/985: unlink d8/d60/ffa 0 2024-04-22T23:46:27.635 INFO:tasks.workunit.client.1.smithi044.stdout:2/986: getdents d8/d60 0 2024-04-22T23:46:27.636 INFO:tasks.workunit.client.1.smithi044.stdout:2/987: fsync d8/d19/fed 0 2024-04-22T23:46:27.639 INFO:tasks.workunit.client.1.smithi044.stdout:2/988: rmdir d8/d19/d36/d93/dae/d108/d141 39 2024-04-22T23:46:27.640 INFO:tasks.workunit.client.1.smithi044.stdout:2/989: readlink d8/d27/d5b/d111/l138 0 2024-04-22T23:46:27.643 INFO:tasks.workunit.client.1.smithi044.stdout:2/990: rmdir d8/d19/d5d/db0/dff 39 2024-04-22T23:46:27.645 INFO:tasks.workunit.client.1.smithi044.stdout:2/991: symlink d8/d27/d5b/d110/daf/d13b/l161 0 2024-04-22T23:46:27.645 INFO:tasks.workunit.client.1.smithi044.stdout:2/992: stat d8/d19/d5d/l6a 0 2024-04-22T23:46:27.648 INFO:tasks.workunit.client.1.smithi044.stdout:2/993: rmdir d8/d19/d36/d46/ddb/ddd/df5 39 2024-04-22T23:46:27.654 INFO:tasks.workunit.client.1.smithi044.stdout:2/994: rename d8/d19/d36/d93/dae/de8/feb to d8/d19/d36/d93/dae/d108/d141/f162 0 2024-04-22T23:46:27.665 INFO:tasks.workunit.client.1.smithi044.stdout:2/995: link d8/d19/d36/d46/d63/d116/ce3 d8/d19/d36/d46/d63/c163 0 2024-04-22T23:46:27.665 INFO:tasks.workunit.client.1.smithi044.stdout:2/996: rmdir d8/d19/d36/d46/d63/d65/d94 39 2024-04-22T23:46:27.665 INFO:tasks.workunit.client.1.smithi044.stdout:2/997: stat d8/d19/d96/l105 0 2024-04-22T23:46:27.667 INFO:tasks.workunit.client.1.smithi044.stdout:2/998: chown d8/d19/f30 162986 1 2024-04-22T23:46:27.669 INFO:tasks.workunit.client.1.smithi044.stdout:2/999: creat d8/d19/d36/d46/d63/d65/d6c/d13a/f164 x:0 0 0 2024-04-22T23:46:27.673 INFO:tasks.workunit.client.1.smithi044.stderr:+ rm -rf -- ./tmp.vPCY9unfEj 2024-04-22T23:46:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 51 MiB/s rd, 122 MiB/s wr, 299 op/s 2024-04-22T23:46:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:27 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 51 MiB/s rd, 122 MiB/s wr, 299 op/s 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-22T23:46:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:27.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:27 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:28.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:28 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:28.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:28 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:29.357 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T23:46:29.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T23:46:29.780 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:29 smithi044 ceph-mon[34463]: pgmap v18: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 49 MiB/s rd, 111 MiB/s wr, 285 op/s 2024-04-22T23:46:29.780 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:29 smithi044 ceph-mon[34463]: Upgrade: Updating mgr.smithi044.iglcti 2024-04-22T23:46:29.780 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:46:29.780 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:46:29.780 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:29.780 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:29 smithi044 ceph-mon[34463]: Deploying daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:46:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:29 smithi031 ceph-mon[26721]: pgmap v18: 65 pgs: 65 active+clean; 3.0 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 49 MiB/s rd, 111 MiB/s wr, 285 op/s 2024-04-22T23:46:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:29 smithi031 ceph-mon[26721]: Upgrade: Updating mgr.smithi044.iglcti 2024-04-22T23:46:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:46:29.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:46:29.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:29.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:29 smithi031 ceph-mon[26721]: Deploying daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:46:32.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:31 smithi044 ceph-mon[34463]: pgmap v19: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 125 MiB/s wr, 369 op/s 2024-04-22T23:46:32.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:31 smithi031 ceph-mon[26721]: pgmap v19: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 125 MiB/s wr, 369 op/s 2024-04-22T23:46:34.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:34 smithi044 ceph-mon[34463]: pgmap v20: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 29 MiB/s rd, 71 MiB/s wr, 230 op/s 2024-04-22T23:46:34.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:34.187 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.188 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:34 smithi031 ceph-mon[26721]: pgmap v20: 65 pgs: 65 active+clean; 2.6 GiB data, 7.9 GiB used, 528 GiB / 536 GiB avail; 29 MiB/s rd, 71 MiB/s wr, 230 op/s 2024-04-22T23:46:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:34.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 -- 172.21.15.31:0/825549420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488104080 msgr2=0x7f5488106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 --2- 172.21.15.31:0/825549420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488104080 0x7f5488106470 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f547c009910 tx=0x7f547c009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 -- 172.21.15.31:0/825549420 shutdown_connections 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 --2- 172.21.15.31:0/825549420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488104080 0x7f5488106470 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 --2- 172.21.15.31:0/825549420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5488101750 0x7f5488103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 -- 172.21.15.31:0/825549420 >> 172.21.15.31:0/825549420 conn(0x7f54880fb160 msgr2=0x7f54880fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 -- 172.21.15.31:0/825549420 shutdown_connections 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 -- 172.21.15.31:0/825549420 wait complete. 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.462+0000 7f548dc45700 1 Processor -- start 2024-04-22T23:46:34.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f548dc45700 1 -- start start 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f548dc45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 0x7f54881967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f548dc45700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5488104080 0x7f5488196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f548dc45700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488197330 con 0x7f5488101750 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f548dc45700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488197470 con 0x7f5488104080 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f54877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 0x7f54881967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f54877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 0x7f54881967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58620/0 (socket says 172.21.15.31:58620) 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f54877fe700 1 -- 172.21.15.31:0/4194874974 learned_addr learned my addr 172.21.15.31:0/4194874974 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f5486ffd700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5488104080 0x7f5488196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f54877fe700 1 -- 172.21.15.31:0/4194874974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5488104080 msgr2=0x7f5488196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f54877fe700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5488104080 0x7f5488196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.463+0000 7f54877fe700 1 -- 172.21.15.31:0/4194874974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0095f0 con 0x7f5488101750 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.464+0000 7f54877fe700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 0x7f54881967d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f547800d900 tx=0x7f547800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.464+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5478004a00 con 0x7f5488101750 2024-04-22T23:46:34.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.464+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54780054d0 con 0x7f5488101750 2024-04-22T23:46:34.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.464+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548819bf20 con 0x7f5488101750 2024-04-22T23:46:34.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.464+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5478009d00 con 0x7f5488101750 2024-04-22T23:46:34.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.464+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548819c3e0 con 0x7f5488101750 2024-04-22T23:46:34.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.465+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488190650 con 0x7f5488101750 2024-04-22T23:46:34.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.466+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 29) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f5478005020 con 0x7f5488101750 2024-04-22T23:46:34.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.466+0000 7f5484ff9700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f547006ea70 0x7f5470070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:34.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.466+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f547808eae0 con 0x7f5488101750 2024-04-22T23:46:34.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.466+0000 7f5486ffd700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f547006ea70 0x7f5470070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:34.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.467+0000 7f5486ffd700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f547006ea70 0x7f5470070f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f547c006010 tx=0x7f547c00b560 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:34.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.471+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f547805b680 con 0x7f5488101750 2024-04-22T23:46:34.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.720+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54880611d0 con 0x7f547006ea70 2024-04-22T23:46:34.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.721+0000 7f5484ff9700 1 -- 172.21.15.31:0/4194874974 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f54880611d0 con 0x7f547006ea70 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f547006ea70 msgr2=0x7f5470070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f547006ea70 0x7f5470070f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f547c006010 tx=0x7f547c00b560 comp rx=0 tx=0).stop 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 msgr2=0x7f54881967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 0x7f54881967d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f547800d900 tx=0x7f547800dc10 comp rx=0 tx=0).stop 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 shutdown_connections 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f547006ea70 0x7f5470070f30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5488101750 0x7f54881967d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 --2- 172.21.15.31:0/4194874974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5488104080 0x7f5488196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.725+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 >> 172.21.15.31:0/4194874974 conn(0x7f54880fb160 msgr2=0x7f54880fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:34.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.727+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 shutdown_connections 2024-04-22T23:46:34.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.727+0000 7f548dc45700 1 -- 172.21.15.31:0/4194874974 wait complete. 2024-04-22T23:46:34.732 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:46:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 -- 172.21.15.31:0/1238958499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4100cb0 msgr2=0x7f26f41010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/1238958499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4100cb0 0x7f26f41010d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009910 tx=0x7f26f0009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 -- 172.21.15.31:0/1238958499 shutdown_connections 2024-04-22T23:46:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/1238958499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f4101eb0 0x7f26f4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/1238958499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4100cb0 0x7f26f41010d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 -- 172.21.15.31:0/1238958499 >> 172.21.15.31:0/1238958499 conn(0x7f26f40fc230 msgr2=0x7f26f40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 -- 172.21.15.31:0/1238958499 shutdown_connections 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.850+0000 7f26fc6b5700 1 -- 172.21.15.31:0/1238958499 wait complete. 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fc6b5700 1 Processor -- start 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fc6b5700 1 -- start start 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fc6b5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f4100cb0 0x7f26f41964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fc6b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 0x7f26f4196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fc6b5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4197050 con 0x7f26f4101eb0 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fc6b5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f4197190 con 0x7f26f4100cb0 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26f9c50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 0x7f26f4196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26f9c50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 0x7f26f4196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58634/0 (socket says 172.21.15.31:58634) 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26f9c50700 1 -- 172.21.15.31:0/2769261383 learned_addr learned my addr 172.21.15.31:0/2769261383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26fa451700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f4100cb0 0x7f26f41964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:34.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26f9c50700 1 -- 172.21.15.31:0/2769261383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f4100cb0 msgr2=0x7f26f41964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:34.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26f9c50700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f4100cb0 0x7f26f41964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:34.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.851+0000 7f26f9c50700 1 -- 172.21.15.31:0/2769261383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00095f0 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.852+0000 7f26f9c50700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 0x7f26f4196a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e920 tx=0x7f26e400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.852+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e400c9d0 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.852+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f26e4010430 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.852+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f419bc40 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.852+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26e4018650 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.852+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f419c190 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.853+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f4109810 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.854+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 29) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f26e40105a0 con 0x7f26f4101eb0 2024-04-22T23:46:34.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.854+0000 7f26eb7fe700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f26e006ea80 0x7f26e0070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:34.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.854+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f26e4014070 con 0x7f26f4101eb0 2024-04-22T23:46:34.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.854+0000 7f26fa451700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f26e006ea80 0x7f26e0070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:34.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.855+0000 7f26fa451700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f26e006ea80 0x7f26e0070f40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b6f0 tx=0x7f26f0005940 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:34.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:34.859+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f26e4053e20 con 0x7f26f4101eb0 2024-04-22T23:46:35.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.116+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26f4106800 con 0x7f26e006ea80 2024-04-22T23:46:35.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.118+0000 7f26eb7fe700 1 -- 172.21.15.31:0/2769261383 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f26f4106800 con 0x7f26e006ea80 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f26e006ea80 msgr2=0x7f26e0070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f26e006ea80 0x7f26e0070f40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26f000b6f0 tx=0x7f26f0005940 comp rx=0 tx=0).stop 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 msgr2=0x7f26f4196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 0x7f26f4196a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e920 tx=0x7f26e400ec30 comp rx=0 tx=0).stop 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 shutdown_connections 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26f4100cb0 0x7f26f41964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f26e006ea80 0x7f26e0070f40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 --2- 172.21.15.31:0/2769261383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26f4101eb0 0x7f26f4196a30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.122+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 >> 172.21.15.31:0/2769261383 conn(0x7f26f40fc230 msgr2=0x7f26f41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:35.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.123+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 shutdown_connections 2024-04-22T23:46:35.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.123+0000 7f26fc6b5700 1 -- 172.21.15.31:0/2769261383 wait complete. 2024-04-22T23:46:35.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 -- 172.21.15.31:0/456682077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 msgr2=0x7f2e3c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 --2- 172.21.15.31:0/456682077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c102310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009910 tx=0x7f2e2c009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 -- 172.21.15.31:0/456682077 shutdown_connections 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 --2- 172.21.15.31:0/456682077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c102310 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 --2- 172.21.15.31:0/456682077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e3c100cb0 0x7f2e3c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 -- 172.21.15.31:0/456682077 >> 172.21.15.31:0/456682077 conn(0x7f2e3c0fc230 msgr2=0x7f2e3c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 -- 172.21.15.31:0/456682077 shutdown_connections 2024-04-22T23:46:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.268+0000 7f2e43679700 1 -- 172.21.15.31:0/456682077 wait complete. 2024-04-22T23:46:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e43679700 1 Processor -- start 2024-04-22T23:46:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e43679700 1 -- start start 2024-04-22T23:46:35.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e43679700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e3c100cb0 0x7f2e3c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e43679700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e43679700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0736c0 con 0x7f2e3c101eb0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e43679700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c073800 con 0x7f2e3c100cb0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.269+0000 7f2e40c14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e40c14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58654/0 (socket says 172.21.15.31:58654) 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e40c14700 1 -- 172.21.15.31:0/807945100 learned_addr learned my addr 172.21.15.31:0/807945100 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e41415700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e3c100cb0 0x7f2e3c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e40c14700 1 -- 172.21.15.31:0/807945100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e3c100cb0 msgr2=0x7f2e3c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e40c14700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e3c100cb0 0x7f2e3c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e40c14700 1 -- 172.21.15.31:0/807945100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0095f0 con 0x7f2e3c101eb0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e40c14700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c073180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c006010 tx=0x7f2e2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2c01d070 con 0x7f2e3c101eb0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c073a80 con 0x7f2e3c101eb0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e2c00f460 con 0x7f2e3c101eb0 2024-04-22T23:46:35.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2c017630 con 0x7f2e3c101eb0 2024-04-22T23:46:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.270+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c073f70 con 0x7f2e3c101eb0 2024-04-22T23:46:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.271+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e20005320 con 0x7f2e3c101eb0 2024-04-22T23:46:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.272+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 29) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f2e2c003a60 con 0x7f2e3c101eb0 2024-04-22T23:46:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.272+0000 7f2e327fc700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2e2806ea80 0x7f2e28070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.272+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2e2c090000 con 0x7f2e3c101eb0 2024-04-22T23:46:35.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.272+0000 7f2e41415700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2e2806ea80 0x7f2e28070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:35.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.273+0000 7f2e41415700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2e2806ea80 0x7f2e28070f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005950 tx=0x7f2e380058e0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.277+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f2e2c0c4e20 con 0x7f2e3c101eb0 2024-04-22T23:46:35.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.504+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e20000bf0 con 0x7f2e2806ea80 2024-04-22T23:46:35.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.515+0000 7f2e327fc700 1 -- 172.21.15.31:0/807945100 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2e20000bf0 con 0x7f2e2806ea80 2024-04-22T23:46:35.507 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:46:35.507 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 30s ago 9m 25.6M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:46:35.507 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (9m) 30s ago 9m 7730k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (3m) 19s ago 8m 7596k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (9m) 30s ago 9m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (8m) 19s ago 8m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 30s ago 9m 87.4M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (4m) 30s ago 4m 13.6M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (3m) 30s ago 3m 270M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (4m) 19s ago 4m 234M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (3m) 19s ago 3m 17.5M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (67s) 30s ago 10m 477M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 starting - - - - 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (10m) 30s ago 11m 52.3M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (8m) 19s ago 8m 41.8M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 30s ago 9m 17.0M - 1.5.0 0da6a335fe13 4abc8631afd1 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 19s ago 8m 16.9M - 1.5.0 0da6a335fe13 b91f737832c2 2024-04-22T23:46:35.508 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 30s ago 7m 158M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:46:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 30s ago 7m 171M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:46:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 30s ago 6m 143M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:46:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (6m) 19s ago 6m 175M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:46:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (5m) 19s ago 5m 159M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:46:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (5m) 19s ago 5m 174M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:46:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 starting - - - - 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2e2806ea80 msgr2=0x7f2e28070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2e2806ea80 0x7f2e28070f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005950 tx=0x7f2e380058e0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 msgr2=0x7f2e3c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c073180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c006010 tx=0x7f2e2c00be20 comp rx=0 tx=0).stop 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 shutdown_connections 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e3c100cb0 0x7f2e3c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2e2806ea80 0x7f2e28070f40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 --2- 172.21.15.31:0/807945100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e3c101eb0 0x7f2e3c073180 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 >> 172.21.15.31:0/807945100 conn(0x7f2e3c0fc230 msgr2=0x7f2e3c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 shutdown_connections 2024-04-22T23:46:35.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.517+0000 7f2e43679700 1 -- 172.21.15.31:0/807945100 wait complete. 2024-04-22T23:46:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 -- 172.21.15.31:0/1805437319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 msgr2=0x7f32680ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 --2- 172.21.15.31:0/1805437319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f32680ff3a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3258009910 tx=0x7f3258009c20 comp rx=0 tx=0).stop 2024-04-22T23:46:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 -- 172.21.15.31:0/1805437319 shutdown_connections 2024-04-22T23:46:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 --2- 172.21.15.31:0/1805437319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 0x7f3268102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 --2- 172.21.15.31:0/1805437319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f32680ff3a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 -- 172.21.15.31:0/1805437319 >> 172.21.15.31:0/1805437319 conn(0x7f32680fa8c0 msgr2=0x7f32680fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:35.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 -- 172.21.15.31:0/1805437319 shutdown_connections 2024-04-22T23:46:35.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.642+0000 7f326ce74700 1 -- 172.21.15.31:0/1805437319 wait complete. 2024-04-22T23:46:35.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326ce74700 1 Processor -- start 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326ce74700 1 -- start start 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326ce74700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f3268196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326ce74700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 0x7f3268196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326ce74700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268197060 con 0x7f32680fef80 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326ce74700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32681971a0 con 0x7f3268102000 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f3268196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f3268196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58674/0 (socket says 172.21.15.31:58674) 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f326659c700 1 -- 172.21.15.31:0/1175655984 learned_addr learned my addr 172.21.15.31:0/1175655984 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.643+0000 7f325dd9b700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 0x7f3268196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325dd9b700 1 -- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 msgr2=0x7f3268196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325dd9b700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f3268196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325dd9b700 1 -- 172.21.15.31:0/1175655984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580095f0 con 0x7f3268102000 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f326659c700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f3268196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325dd9b700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 0x7f3268196a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f325000e970 tx=0x7f325000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325000c9d0 con 0x7f3268102000 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326819bc50 con 0x7f3268102000 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3250010430 con 0x7f3268102000 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326819c1a0 con 0x7f3268102000 2024-04-22T23:46:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.644+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32500185f0 con 0x7f3268102000 2024-04-22T23:46:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.645+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3268109800 con 0x7f3268102000 2024-04-22T23:46:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.646+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 29) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f32500105a0 con 0x7f3268102000 2024-04-22T23:46:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.646+0000 7f325ffff700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f325406ea80 0x7f3254070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.646+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3250014070 con 0x7f3268102000 2024-04-22T23:46:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.646+0000 7f326659c700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f325406ea80 0x7f3254070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.647+0000 7f326659c700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f325406ea80 0x7f3254070f40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3258009fd0 tx=0x7f3258005940 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:35.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:35.653+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f3250053e20 con 0x7f3268102000 2024-04-22T23:46:36.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.016+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f326819c800 con 0x7f3268102000 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.017+0000 7f325ffff700 1 -- 172.21.15.31:0/1175655984 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f325005b8f0 con 0x7f3268102000 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:46:36.010 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:46:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f325406ea80 msgr2=0x7f3254070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f325406ea80 0x7f3254070f40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3258009fd0 tx=0x7f3258005940 comp rx=0 tx=0).stop 2024-04-22T23:46:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 msgr2=0x7f3268196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:36.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 0x7f3268196a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f325000e970 tx=0x7f325000ec80 comp rx=0 tx=0).stop 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 shutdown_connections 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f325406ea80 0x7f3254070f40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32680fef80 0x7f3268196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 --2- 172.21.15.31:0/1175655984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3268102000 0x7f3268196a40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.019+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 >> 172.21.15.31:0/1175655984 conn(0x7f32680fa8c0 msgr2=0x7f3268105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.020+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 shutdown_connections 2024-04-22T23:46:36.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.020+0000 7f326ce74700 1 -- 172.21.15.31:0/1175655984 wait complete. 2024-04-22T23:46:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.146+0000 7fd71fc31700 1 -- 172.21.15.31:0/1623616389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 msgr2=0x7fd718100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.146+0000 7fd71fc31700 1 --2- 172.21.15.31:0/1623616389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 0x7fd718100030 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd71000b0a0 tx=0x7fd71000b3b0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.147+0000 7fd71fc31700 1 -- 172.21.15.31:0/1623616389 shutdown_connections 2024-04-22T23:46:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.147+0000 7fd71fc31700 1 --2- 172.21.15.31:0/1623616389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd718101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.147+0000 7fd71fc31700 1 --2- 172.21.15.31:0/1623616389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 0x7fd718100030 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.147+0000 7fd71fc31700 1 -- 172.21.15.31:0/1623616389 >> 172.21.15.31:0/1623616389 conn(0x7fd7180fb190 msgr2=0x7fd7180fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.148+0000 7fd71fc31700 1 -- 172.21.15.31:0/1623616389 shutdown_connections 2024-04-22T23:46:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.148+0000 7fd71fc31700 1 -- 172.21.15.31:0/1623616389 wait complete. 2024-04-22T23:46:36.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.149+0000 7fd71fc31700 1 Processor -- start 2024-04-22T23:46:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.149+0000 7fd71fc31700 1 -- start start 2024-04-22T23:46:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.149+0000 7fd71fc31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 0x7fd71819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.149+0000 7fd71fc31700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd71819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:36.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.150+0000 7fd71fc31700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71819d5e0 con 0x7fd718100e10 2024-04-22T23:46:36.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.150+0000 7fd71fc31700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71819d720 con 0x7fd7180ffc10 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.150+0000 7fd71d1cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd71819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.150+0000 7fd71d1cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd71819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58692/0 (socket says 172.21.15.31:58692) 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.150+0000 7fd71d1cc700 1 -- 172.21.15.31:0/328579807 learned_addr learned my addr 172.21.15.31:0/328579807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.150+0000 7fd71d9cd700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 0x7fd71819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.151+0000 7fd71d1cc700 1 -- 172.21.15.31:0/328579807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 msgr2=0x7fd71819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.151+0000 7fd71d1cc700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 0x7fd71819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.151+0000 7fd71d1cc700 1 -- 172.21.15.31:0/328579807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710009d00 con 0x7fd718100e10 2024-04-22T23:46:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.151+0000 7fd71d1cc700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd71819cfc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e920 tx=0x7fd70800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.152+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd70800c9d0 con 0x7fd718100e10 2024-04-22T23:46:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.152+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd708010430 con 0x7fd718100e10 2024-04-22T23:46:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.152+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7181a21d0 con 0x7fd718100e10 2024-04-22T23:46:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.152+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd708018650 con 0x7fd718100e10 2024-04-22T23:46:36.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.152+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7181a2720 con 0x7fd718100e10 2024-04-22T23:46:36.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.155+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 29) v1 ==== 92700+0+0 (secure 0 0 0) 0x7fd7080105a0 con 0x7fd718100e10 2024-04-22T23:46:36.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.155+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd718108700 con 0x7fd718100e10 2024-04-22T23:46:36.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.155+0000 7fd70effd700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd70406eb80 0x7fd704071040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:46:36.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.155+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd708014070 con 0x7fd718100e10 2024-04-22T23:46:36.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.156+0000 7fd71d9cd700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd70406eb80 0x7fd704071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:46:36.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.157+0000 7fd71d9cd700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd70406eb80 0x7fd704071040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd71000d010 tx=0x7fd710006000 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:46:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.163+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fd70805b950 con 0x7fd718100e10 2024-04-22T23:46:36.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:36 smithi031 ceph-mon[26721]: pgmap v21: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 29 MiB/s rd, 71 MiB/s wr, 295 op/s 2024-04-22T23:46:36.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:36 smithi031 ceph-mon[26721]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:36.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:36 smithi031 ceph-mon[26721]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:36.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:36 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1175655984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:36.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.447+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd718068c90 con 0x7fd70406eb80 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.448+0000 7fd70effd700 1 -- 172.21.15.31:0/328579807 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fd718068c90 con 0x7fd70406eb80 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-22T23:46:36.440 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "", 2024-04-22T23:46:36.441 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-04-22T23:46:36.441 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:46:36.441 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:46:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd70406eb80 msgr2=0x7fd704071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd70406eb80 0x7fd704071040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd71000d010 tx=0x7fd710006000 comp rx=0 tx=0).stop 2024-04-22T23:46:36.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 msgr2=0x7fd71819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd71819cfc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd70800e920 tx=0x7fd70800ec30 comp rx=0 tx=0).stop 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 shutdown_connections 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7180ffc10 0x7fd71819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd70406eb80 0x7fd704071040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 --2- 172.21.15.31:0/328579807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd718100e10 0x7fd71819cfc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.450+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 >> 172.21.15.31:0/328579807 conn(0x7fd7180fb190 msgr2=0x7fd718068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.451+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 shutdown_connections 2024-04-22T23:46:36.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:46:36.451+0000 7fd71fc31700 1 -- 172.21.15.31:0/328579807 wait complete. 2024-04-22T23:46:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:36 smithi044 ceph-mon[34463]: pgmap v21: 65 pgs: 65 active+clean; 2.0 GiB data, 6.2 GiB used, 530 GiB / 536 GiB avail; 29 MiB/s rd, 71 MiB/s wr, 295 op/s 2024-04-22T23:46:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:36 smithi044 ceph-mon[34463]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:36 smithi044 ceph-mon[34463]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:36 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1175655984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:37.355 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:37 smithi044 ceph-mon[34463]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:37.355 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:37 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:37.355 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:37 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:37.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:37 smithi031 ceph-mon[26721]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:37.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:37 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:37.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:37 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:38.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:38 smithi031 ceph-mon[26721]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:38.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:38 smithi031 ceph-mon[26721]: pgmap v22: 65 pgs: 65 active+clean; 1.8 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 9.9 MiB/s rd, 25 MiB/s wr, 175 op/s 2024-04-22T23:46:38.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:38 smithi044 ceph-mon[34463]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:46:38.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:38 smithi044 ceph-mon[34463]: pgmap v22: 65 pgs: 65 active+clean; 1.8 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 9.9 MiB/s rd, 25 MiB/s wr, 175 op/s 2024-04-22T23:46:39.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:39.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:39.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:39 smithi044 ceph-mon[34463]: pgmap v23: 65 pgs: 65 active+clean; 1.8 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 158 op/s 2024-04-22T23:46:39.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:39.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:39.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:39 smithi031 ceph-mon[26721]: pgmap v23: 65 pgs: 65 active+clean; 1.8 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 5.3 MiB/s rd, 14 MiB/s wr, 158 op/s 2024-04-22T23:46:40.612 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:40 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:40.612 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:40 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:40.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:40 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:40.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:40 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:41.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:41 smithi031 ceph-mon[26721]: pgmap v24: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 5.9 MiB/s rd, 15 MiB/s wr, 215 op/s 2024-04-22T23:46:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:41 smithi044 ceph-mon[34463]: pgmap v24: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 5.9 MiB/s rd, 15 MiB/s wr, 215 op/s 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all mgr 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.fazdcv"}]: dispatch 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.fazdcv"}]': finished 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.iglcti"}]: dispatch 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.iglcti"}]': finished 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all rgw 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all rbd-mirror 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all iscsi 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all nfs 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all nvmeof 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: pgmap v25: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Standby manager daemon smithi044.iglcti restarted 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:46:43.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:46:43.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:46:43.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:43 smithi031 ceph-mon[26721]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all mgr 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.fazdcv"}]: dispatch 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.fazdcv"}]': finished 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.iglcti"}]: dispatch 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi044.iglcti"}]': finished 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all rgw 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all rbd-mirror 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all iscsi 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all nfs 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all nvmeof 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: pgmap v25: 65 pgs: 65 active+clean; 1.3 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti restarted 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:46:43.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:43 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1445196508' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:46:44.786 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:44 smithi031 ceph-mon[26721]: mgrmap e30: smithi031.fazdcv(active, since 45s), standbys: smithi044.iglcti 2024-04-22T23:46:44.786 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:44 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:44.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:44 smithi044 ceph-mon[34463]: mgrmap e30: smithi031.fazdcv(active, since 45s), standbys: smithi044.iglcti 2024-04-22T23:46:44.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:44 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:46.172 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:45 smithi031 ceph-mon[26721]: Upgrade: Updating node-exporter.smithi031 (1/2) 2024-04-22T23:46:46.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:45 smithi031 ceph-mon[26721]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-22T23:46:46.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:45 smithi031 ceph-mon[26721]: pgmap v26: 65 pgs: 65 active+clean; 990 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-04-22T23:46:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:45 smithi044 ceph-mon[34463]: Upgrade: Updating node-exporter.smithi031 (1/2) 2024-04-22T23:46:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:45 smithi044 ceph-mon[34463]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-22T23:46:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:45 smithi044 ceph-mon[34463]: pgmap v26: 65 pgs: 65 active+clean; 990 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 182 op/s 2024-04-22T23:46:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:47 smithi044 ceph-mon[34463]: pgmap v27: 65 pgs: 65 active+clean; 990 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-04-22T23:46:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:47 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:47 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:48.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:47 smithi031 ceph-mon[26721]: pgmap v27: 65 pgs: 65 active+clean; 990 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-04-22T23:46:48.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:47 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:48.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:47 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:46:49.369 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T23:46:49.370 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-22T23:46:50.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:49 smithi031 ceph-mon[26721]: pgmap v28: 65 pgs: 65 active+clean; 990 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-04-22T23:46:50.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:49 smithi044 ceph-mon[34463]: pgmap v28: 65 pgs: 65 active+clean; 990 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-04-22T23:46:52.361 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:51 smithi044 ceph-mon[34463]: pgmap v29: 65 pgs: 65 active+clean; 521 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-04-22T23:46:52.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:51 smithi031 ceph-mon[26721]: pgmap v29: 65 pgs: 65 active+clean; 521 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-04-22T23:46:54.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:53 smithi031 ceph-mon[26721]: pgmap v30: 65 pgs: 65 active+clean; 521 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-04-22T23:46:54.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:53 smithi044 ceph-mon[34463]: pgmap v30: 65 pgs: 65 active+clean; 521 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-04-22T23:46:56.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:55 smithi031 ceph-mon[26721]: pgmap v31: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-04-22T23:46:56.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:55 smithi044 ceph-mon[34463]: pgmap v31: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-04-22T23:46:58.433 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:57 smithi044 ceph-mon[34463]: pgmap v32: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-04-22T23:46:58.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:57 smithi031 ceph-mon[26721]: pgmap v32: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-04-22T23:46:59.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:58 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:58 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:00.162 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:59 smithi044 ceph-mon[34463]: Upgrade: Updating node-exporter.smithi044 (2/2) 2024-04-22T23:47:00.162 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:59 smithi044 ceph-mon[34463]: Deploying daemon node-exporter.smithi044 on smithi044 2024-04-22T23:47:00.163 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:46:59 smithi044 ceph-mon[34463]: pgmap v33: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-04-22T23:47:00.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:59 smithi031 ceph-mon[26721]: Upgrade: Updating node-exporter.smithi044 (2/2) 2024-04-22T23:47:00.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:59 smithi031 ceph-mon[26721]: Deploying daemon node-exporter.smithi044 on smithi044 2024-04-22T23:47:00.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:46:59 smithi031 ceph-mon[26721]: pgmap v33: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-04-22T23:47:02.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:01 smithi031 ceph-mon[26721]: pgmap v34: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-04-22T23:47:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:01 smithi044 ceph-mon[34463]: pgmap v34: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-04-22T23:47:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:03 smithi031 ceph-mon[26721]: pgmap v35: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-04-22T23:47:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:03 smithi044 ceph-mon[34463]: pgmap v35: 65 pgs: 65 active+clean; 291 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-04-22T23:47:06.251 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:05 smithi044 ceph-mon[34463]: pgmap v36: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-04-22T23:47:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:05 smithi031 ceph-mon[26721]: pgmap v36: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 -- 172.21.15.31:0/1025470625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 msgr2=0x7f0ffc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 --2- 172.21.15.31:0/1025470625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc0fffc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009910 tx=0x7f0fec009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 -- 172.21.15.31:0/1025470625 shutdown_connections 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 --2- 172.21.15.31:0/1025470625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc100da0 0x7f0ffc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 --2- 172.21.15.31:0/1025470625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc0fffc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 -- 172.21.15.31:0/1025470625 >> 172.21.15.31:0/1025470625 conn(0x7f0ffc0fb120 msgr2=0x7f0ffc0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 -- 172.21.15.31:0/1025470625 shutdown_connections 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.586+0000 7f10023d9700 1 -- 172.21.15.31:0/1025470625 wait complete. 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.587+0000 7f10023d9700 1 Processor -- start 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f10023d9700 1 -- start start 2024-04-22T23:47:06.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f10023d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc1942a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f10023d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc100da0 0x7f0ffc1947e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f10023d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc194e00 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f10023d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc194f40 con 0x7f0ffc100da0 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f0ffbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc1942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f0ffbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc1942a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38514/0 (socket says 172.21.15.31:38514) 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.588+0000 7f0ffbfff700 1 -- 172.21.15.31:0/390408038 learned_addr learned my addr 172.21.15.31:0/390408038 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f0ffbfff700 1 -- 172.21.15.31:0/390408038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc100da0 msgr2=0x7f0ffc1947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f0ffbfff700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc100da0 0x7f0ffc1947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f0ffbfff700 1 -- 172.21.15.31:0/390408038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0095f0 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f0ffbfff700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc1942a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bda0 tx=0x7f0fec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fec01d070 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ffc199990 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc199e80 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.589+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0fec00f460 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.591+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fec02b610 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.592+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 30) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f0fec02b770 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.592+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe8005320 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.593+0000 7f0ff97fa700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f0fe406ead0 0x7f0fe4070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.593+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0fec08fe00 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.593+0000 7f0ffb7fe700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f0fe406ead0 0x7f0fe4070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.594+0000 7f0ffb7fe700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f0fe406ead0 0x7f0fe4070f90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005950 tx=0x7f0ff00058e0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.599+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f0fec05ca20 con 0x7f0ffc0ffba0 2024-04-22T23:47:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.868+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fe8000bf0 con 0x7f0fe406ead0 2024-04-22T23:47:06.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.870+0000 7f0ff97fa700 1 -- 172.21.15.31:0/390408038 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0fe8000bf0 con 0x7f0fe406ead0 2024-04-22T23:47:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f0fe406ead0 msgr2=0x7f0fe4070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f0fe406ead0 0x7f0fe4070f90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005950 tx=0x7f0ff00058e0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 msgr2=0x7f0ffc1942a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc1942a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bda0 tx=0x7f0fec00be80 comp rx=0 tx=0).stop 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 shutdown_connections 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f0fe406ead0 0x7f0fe4070f90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ffc0ffba0 0x7f0ffc1942a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 --2- 172.21.15.31:0/390408038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ffc100da0 0x7f0ffc1947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 >> 172.21.15.31:0/390408038 conn(0x7f0ffc0fb120 msgr2=0x7f0ffc103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 shutdown_connections 2024-04-22T23:47:06.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.872+0000 7f10023d9700 1 -- 172.21.15.31:0/390408038 wait complete. 2024-04-22T23:47:06.875 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:47:06.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/3282022599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e40ff590 msgr2=0x7ff1e41019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/3282022599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e40ff590 0x7ff1e41019c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009910 tx=0x7ff1e0009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/3282022599 shutdown_connections 2024-04-22T23:47:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/3282022599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e4101f00 0x7ff1e4104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/3282022599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e40ff590 0x7ff1e41019c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/3282022599 >> 172.21.15.31:0/3282022599 conn(0x7ff1e40fb190 msgr2=0x7ff1e40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/3282022599 shutdown_connections 2024-04-22T23:47:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.987+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/3282022599 wait complete. 2024-04-22T23:47:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1ec1e4700 1 Processor -- start 2024-04-22T23:47:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1ec1e4700 1 -- start start 2024-04-22T23:47:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1ec1e4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e40ff590 0x7ff1e419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1ec1e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 0x7ff1e419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1ec1e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419d5d0 con 0x7ff1e4101f00 2024-04-22T23:47:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1ec1e4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419d710 con 0x7ff1e40ff590 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.988+0000 7ff1e977f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 0x7ff1e419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e977f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 0x7ff1e419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38524/0 (socket says 172.21.15.31:38524) 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e9f80700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e40ff590 0x7ff1e419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e977f700 1 -- 172.21.15.31:0/1841606156 learned_addr learned my addr 172.21.15.31:0/1841606156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e977f700 1 -- 172.21.15.31:0/1841606156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e40ff590 msgr2=0x7ff1e419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e977f700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e40ff590 0x7ff1e419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e977f700 1 -- 172.21.15.31:0/1841606156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00095f0 con 0x7ff1e4101f00 2024-04-22T23:47:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.989+0000 7ff1e977f700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 0x7ff1e419cfb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400e920 tx=0x7ff1d400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.990+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d400c9d0 con 0x7ff1e4101f00 2024-04-22T23:47:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.990+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e41a21c0 con 0x7ff1e4101f00 2024-04-22T23:47:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.990+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff1d4010430 con 0x7ff1e4101f00 2024-04-22T23:47:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.990+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d4018650 con 0x7ff1e4101f00 2024-04-22T23:47:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.990+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e41a2710 con 0x7ff1e4101f00 2024-04-22T23:47:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.991+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e4196c00 con 0x7ff1e4101f00 2024-04-22T23:47:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.992+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 30) v1 ==== 92700+0+0 (secure 0 0 0) 0x7ff1d40105a0 con 0x7ff1e4101f00 2024-04-22T23:47:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.993+0000 7ff1daffd700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff1d006ea80 0x7ff1d0070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.993+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff1d4014070 con 0x7ff1e4101f00 2024-04-22T23:47:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.993+0000 7ff1e9f80700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff1d006ea80 0x7ff1d0070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.993+0000 7ff1e9f80700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff1d006ea80 0x7ff1d0070f40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009fd0 tx=0x7ff1e0005940 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:06.997+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7ff1d4053e20 con 0x7ff1e4101f00 2024-04-22T23:47:07.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.239+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e40611d0 con 0x7ff1d006ea80 2024-04-22T23:47:07.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.241+0000 7ff1daffd700 1 -- 172.21.15.31:0/1841606156 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7ff1e40611d0 con 0x7ff1d006ea80 2024-04-22T23:47:07.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff1d006ea80 msgr2=0x7ff1d0070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff1d006ea80 0x7ff1d0070f40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009fd0 tx=0x7ff1e0005940 comp rx=0 tx=0).stop 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 msgr2=0x7ff1e419cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 0x7ff1e419cfb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400e920 tx=0x7ff1d400ec30 comp rx=0 tx=0).stop 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 shutdown_connections 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1e40ff590 0x7ff1e419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff1d006ea80 0x7ff1d0070f40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 --2- 172.21.15.31:0/1841606156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1e4101f00 0x7ff1e419cfb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 >> 172.21.15.31:0/1841606156 conn(0x7ff1e40fb190 msgr2=0x7ff1e40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 shutdown_connections 2024-04-22T23:47:07.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.243+0000 7ff1ec1e4700 1 -- 172.21.15.31:0/1841606156 wait complete. 2024-04-22T23:47:07.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 -- 172.21.15.31:0/1947223929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e700ffc10 msgr2=0x7f6e70100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 --2- 172.21.15.31:0/1947223929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e700ffc10 0x7f6e70100030 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009910 tx=0x7f6e64009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 -- 172.21.15.31:0/1947223929 shutdown_connections 2024-04-22T23:47:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 --2- 172.21.15.31:0/1947223929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e70100e10 0x7f6e70101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 --2- 172.21.15.31:0/1947223929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e700ffc10 0x7f6e70100030 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 -- 172.21.15.31:0/1947223929 >> 172.21.15.31:0/1947223929 conn(0x7f6e700fb190 msgr2=0x7f6e700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.386+0000 7f6e76e76700 1 -- 172.21.15.31:0/1947223929 shutdown_connections 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 -- 172.21.15.31:0/1947223929 wait complete. 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 Processor -- start 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 -- start start 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 0x7f6e70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 0x7f6e70069090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70071ae0 con 0x7f6e70100e10 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.387+0000 7f6e76e76700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70071c50 con 0x7f6e700ffc10 2024-04-22T23:47:07.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 0x7f6e70069090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 0x7f6e70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 0x7f6e70068b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:50024/0 (socket says 172.21.15.31:50024) 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 0x7f6e70069090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38544/0 (socket says 172.21.15.31:38544) 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 -- 172.21.15.31:0/3702887960 learned_addr learned my addr 172.21.15.31:0/3702887960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 -- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 msgr2=0x7f6e70069090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 0x7f6e70069090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 -- 172.21.15.31:0/3702887960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640095f0 con 0x7f6e700ffc10 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e6ffff700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 0x7f6e70069090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e74c12700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 0x7f6e70068b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e6401ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e64021070 con 0x7f6e700ffc10 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e70071ed0 con 0x7f6e700ffc10 2024-04-22T23:47:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e64005dc0 con 0x7f6e700ffc10 2024-04-22T23:47:07.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.388+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6400f690 con 0x7f6e700ffc10 2024-04-22T23:47:07.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.389+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e700723c0 con 0x7f6e700ffc10 2024-04-22T23:47:07.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.390+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e700726a0 con 0x7f6e700ffc10 2024-04-22T23:47:07.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.390+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 30) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f6e64025070 con 0x7f6e700ffc10 2024-04-22T23:47:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.391+0000 7f6e6dffb700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6e6006ea80 0x7f6e60070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.391+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6e640941c0 con 0x7f6e700ffc10 2024-04-22T23:47:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.391+0000 7f6e6ffff700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6e6006ea80 0x7f6e60070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:07.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.392+0000 7f6e6ffff700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6e6006ea80 0x7f6e60070f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6e70072d50 tx=0x7f6e5c009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.397+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f6e6405d190 con 0x7f6e700ffc10 2024-04-22T23:47:07.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.678+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e70105760 con 0x7f6e6006ea80 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.691+0000 7f6e6dffb700 1 -- 172.21.15.31:0/3702887960 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6e70105760 con 0x7f6e6006ea80 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 28s ago 9m 25.7M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (10m) 28s ago 10m 7772k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (4m) 31s ago 9m 7646k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (10m) 28s ago 10m 6660k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (9m) 31s ago 9m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (8m) 28s ago 9m 87.8M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (4m) 28s ago 4m 14.0M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:47:07.683 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (4m) 28s ago 4m 290M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (4m) 31s ago 4m 244M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (4m) 31s ago 4m 17.9M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (99s) 28s ago 11m 497M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (35s) 31s ago 8m 63.8M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (11m) 28s ago 11m 53.1M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (8m) 31s ago 8m 40.9M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 starting - - - - 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 starting - - - - 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 28s ago 8m 174M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 28s ago 7m 173M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (7m) 28s ago 7m 149M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (6m) 31s ago 6m 209M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (6m) 31s ago 6m 188M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (6m) 31s ago 6m 185M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:47:07.684 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (41s) 28s ago 9m 47.4M - 2.43.0 a07b618ecd1d fa288861173d 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6e6006ea80 msgr2=0x7f6e60070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6e6006ea80 0x7f6e60070f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6e70072d50 tx=0x7f6e5c009260 comp rx=0 tx=0).stop 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 msgr2=0x7f6e70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 0x7f6e70068b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e6401ca50 comp rx=0 tx=0).stop 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 shutdown_connections 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e700ffc10 0x7f6e70068b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6e6006ea80 0x7f6e60070f40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 --2- 172.21.15.31:0/3702887960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100e10 0x7f6e70069090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 >> 172.21.15.31:0/3702887960 conn(0x7f6e700fb190 msgr2=0x7f6e70104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 shutdown_connections 2024-04-22T23:47:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.695+0000 7f6e76e76700 1 -- 172.21.15.31:0/3702887960 wait complete. 2024-04-22T23:47:07.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 -- 172.21.15.31:0/1461862907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300ffcd0 msgr2=0x7f6430100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 --2- 172.21.15.31:0/1461862907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300ffcd0 0x7f6430100150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f641c009910 tx=0x7f641c009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 -- 172.21.15.31:0/1461862907 shutdown_connections 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 --2- 172.21.15.31:0/1461862907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300ffcd0 0x7f6430100150 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 --2- 172.21.15.31:0/1461862907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64300fef60 0x7f64300ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 -- 172.21.15.31:0/1461862907 >> 172.21.15.31:0/1461862907 conn(0x7f64300fa8c0 msgr2=0x7f64300fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.822+0000 7f6436c76700 1 -- 172.21.15.31:0/1461862907 shutdown_connections 2024-04-22T23:47:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 -- 172.21.15.31:0/1461862907 wait complete. 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 Processor -- start 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 -- start start 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 0x7f643019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64300ffcd0 0x7f643019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019d610 con 0x7f64300fef60 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.823+0000 7f6436c76700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019d750 con 0x7f64300ffcd0 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 0x7f643019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 0x7f643019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38552/0 (socket says 172.21.15.31:38552) 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 -- 172.21.15.31:0/580311037 learned_addr learned my addr 172.21.15.31:0/580311037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f642ffff700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64300ffcd0 0x7f643019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 -- 172.21.15.31:0/580311037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64300ffcd0 msgr2=0x7f643019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64300ffcd0 0x7f643019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 -- 172.21.15.31:0/580311037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c0095f0 con 0x7f64300fef60 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.824+0000 7f6434a12700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 0x7f643019cab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f642400e920 tx=0x7f642400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.825+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642400c9d0 con 0x7f64300fef60 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.825+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6424010430 con 0x7f64300fef60 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.825+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6424018650 con 0x7f64300fef60 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.825+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64301a2200 con 0x7f64300fef60 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.825+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301a2700 con 0x7f64300fef60 2024-04-22T23:47:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.826+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643010fe10 con 0x7f64300fef60 2024-04-22T23:47:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.828+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 30) v1 ==== 92700+0+0 (secure 0 0 0) 0x7f64240105a0 con 0x7f64300fef60 2024-04-22T23:47:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.828+0000 7f642dffb700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f642006ea80 0x7f6420070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.828+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6424014070 con 0x7f64300fef60 2024-04-22T23:47:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.828+0000 7f642ffff700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f642006ea80 0x7f6420070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.829+0000 7f642ffff700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f642006ea80 0x7f6420070f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f641c009fd0 tx=0x7f641c005940 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:07.833+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f642405bf50 con 0x7f64300fef60 2024-04-22T23:47:08.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.174+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f64301a2d90 con 0x7f64300fef60 2024-04-22T23:47:08.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.175+0000 7f642dffb700 1 -- 172.21.15.31:0/580311037 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f64240262c0 con 0x7f64300fef60 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:47:08.168 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:47:08.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:08.169 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:47:08.169 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:47:08.169 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:47:08.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:47:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:47:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f642006ea80 msgr2=0x7f6420070f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f642006ea80 0x7f6420070f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f641c009fd0 tx=0x7f641c005940 comp rx=0 tx=0).stop 2024-04-22T23:47:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 msgr2=0x7f643019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 0x7f643019cab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f642400e920 tx=0x7f642400ec30 comp rx=0 tx=0).stop 2024-04-22T23:47:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 shutdown_connections 2024-04-22T23:47:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f642006ea80 0x7f6420070f40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300fef60 0x7f643019cab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 --2- 172.21.15.31:0/580311037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64300ffcd0 0x7f643019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 >> 172.21.15.31:0/580311037 conn(0x7f64300fa8c0 msgr2=0x7f6430102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 shutdown_connections 2024-04-22T23:47:08.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.178+0000 7f6436c76700 1 -- 172.21.15.31:0/580311037 wait complete. 2024-04-22T23:47:08.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:07 smithi031 ceph-mon[26721]: pgmap v37: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-22T23:47:08.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:07 smithi031 ceph-mon[26721]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:08.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:07 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:08.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:07 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:08.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:07 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:08.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.317+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1036148725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6040ffbb0 msgr2=0x7fd6040fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:08.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.317+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1036148725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6040ffbb0 0x7fd6040fffd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009910 tx=0x7fd5f4009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:08.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.317+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1036148725 shutdown_connections 2024-04-22T23:47:08.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.317+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1036148725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd604100db0 0x7fd604101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.317+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1036148725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6040ffbb0 0x7fd6040fffd0 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009910 tx=0x7fd5f4009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:08.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.317+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1036148725 >> 172.21.15.31:0/1036148725 conn(0x7fd6040fb150 msgr2=0x7fd6040fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:08.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.318+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1036148725 shutdown_connections 2024-04-22T23:47:08.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.318+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1036148725 wait complete. 2024-04-22T23:47:08.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.318+0000 7fd60a3bb700 1 Processor -- start 2024-04-22T23:47:08.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.318+0000 7fd60a3bb700 1 -- start start 2024-04-22T23:47:08.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd60a3bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 0x7fd6040699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:08.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd60a3bb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd604197230 0x7fd6041996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:08.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd60a3bb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604199c90 con 0x7fd604100db0 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd60a3bb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd604199e00 con 0x7fd604197230 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd603fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 0x7fd6040699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd603fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 0x7fd6040699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38566/0 (socket says 172.21.15.31:38566) 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd603fff700 1 -- 172.21.15.31:0/1898245270 learned_addr learned my addr 172.21.15.31:0/1898245270 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.319+0000 7fd6037fe700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd604197230 0x7fd6041996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd603fff700 1 -- 172.21.15.31:0/1898245270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd604197230 msgr2=0x7fd6041996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd603fff700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd604197230 0x7fd6041996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd603fff700 1 -- 172.21.15.31:0/1898245270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40095f0 con 0x7fd604100db0 2024-04-22T23:47:08.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd603fff700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 0x7fd6040699d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4000c00 tx=0x7fd5f400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:08.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f401d070 con 0x7fd604100db0 2024-04-22T23:47:08.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd5f4005dc0 con 0x7fd604100db0 2024-04-22T23:47:08.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5f4017630 con 0x7fd604100db0 2024-04-22T23:47:08.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.320+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60419a080 con 0x7fd604100db0 2024-04-22T23:47:08.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.321+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60419a520 con 0x7fd604100db0 2024-04-22T23:47:08.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.322+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd604108710 con 0x7fd604100db0 2024-04-22T23:47:08.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.323+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 30) v1 ==== 92700+0+0 (secure 0 0 0) 0x7fd5f40038a0 con 0x7fd604100db0 2024-04-22T23:47:08.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.323+0000 7fd6017fa700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd5ec06ead0 0x7fd5ec070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:08.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.323+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd5f408ff60 con 0x7fd604100db0 2024-04-22T23:47:08.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.324+0000 7fd6037fe700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd5ec06ead0 0x7fd5ec070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:08.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.324+0000 7fd6037fe700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd5ec06ead0 0x7fd5ec070f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009bf0 tx=0x7fd5f8009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.328+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fd5f405ca60 con 0x7fd604100db0 2024-04-22T23:47:08.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:07 smithi044 ceph-mon[34463]: pgmap v37: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-22T23:47:08.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:07 smithi044 ceph-mon[34463]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:08.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:07 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:08.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:07 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:08.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:07 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.661+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd60419b160 con 0x7fd5ec06ead0 2024-04-22T23:47:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.663+0000 7fd6017fa700 1 -- 172.21.15.31:0/1898245270 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fd60419b160 con 0x7fd5ec06ead0 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:47:08.655 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd5ec06ead0 msgr2=0x7fd5ec070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd5ec06ead0 0x7fd5ec070f90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009bf0 tx=0x7fd5f8009260 comp rx=0 tx=0).stop 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 msgr2=0x7fd6040699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 0x7fd6040699d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4000c00 tx=0x7fd5f400f740 comp rx=0 tx=0).stop 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 shutdown_connections 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd5ec06ead0 0x7fd5ec070f90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd604100db0 0x7fd6040699d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 --2- 172.21.15.31:0/1898245270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd604197230 0x7fd6041996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 >> 172.21.15.31:0/1898245270 conn(0x7fd6040fb150 msgr2=0x7fd604103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 shutdown_connections 2024-04-22T23:47:08.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:08.666+0000 7fd60a3bb700 1 -- 172.21.15.31:0/1898245270 wait complete. 2024-04-22T23:47:09.312 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:08 smithi044 ceph-mon[34463]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:09.312 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:08 smithi044 ceph-mon[34463]: from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:09.312 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:08 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/580311037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:09.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:08 smithi031 ceph-mon[26721]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:09.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:08 smithi031 ceph-mon[26721]: from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:09.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:08 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/580311037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:10.265 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:09 smithi044 ceph-mon[34463]: pgmap v38: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-22T23:47:10.265 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:09 smithi044 ceph-mon[34463]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:10.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:09 smithi031 ceph-mon[26721]: pgmap v38: 65 pgs: 65 active+clean; 296 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 92 op/s 2024-04-22T23:47:10.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:09 smithi031 ceph-mon[26721]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:12 smithi031 ceph-mon[26721]: pgmap v39: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 136 op/s 2024-04-22T23:47:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:12 smithi044 ceph-mon[34463]: pgmap v39: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 136 op/s 2024-04-22T23:47:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:13 smithi044 ceph-mon[34463]: pgmap v40: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-04-22T23:47:13.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:13 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:13 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:13 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:13 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:13 smithi031 ceph-mon[26721]: pgmap v40: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-04-22T23:47:13.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:13 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:13 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:13 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:13.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:13 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:14.941 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:14.941 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:14.941 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:14.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:14.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:14.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:16.272 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:15 smithi044 ceph-mon[34463]: pgmap v41: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 131 op/s 2024-04-22T23:47:16.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:15 smithi031 ceph-mon[26721]: pgmap v41: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 131 op/s 2024-04-22T23:47:18.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:18 smithi031 ceph-mon[26721]: pgmap v42: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 111 op/s 2024-04-22T23:47:18.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:18.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:18 smithi044 ceph-mon[34463]: pgmap v42: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 111 op/s 2024-04-22T23:47:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.526 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.526 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.526 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:47:19.526 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: pgmap v43: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.527 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: pgmap v43: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:19.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:22.118 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:21 smithi031 ceph-mon[26721]: pgmap v44: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-22T23:47:22.118 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:21 smithi031 ceph-mon[26721]: Upgrade: Updating prometheus.smithi031 2024-04-22T23:47:22.353 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:21 smithi044 ceph-mon[34463]: pgmap v44: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-22T23:47:22.353 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:21 smithi044 ceph-mon[34463]: Upgrade: Updating prometheus.smithi031 2024-04-22T23:47:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:22 smithi031 ceph-mon[26721]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-22T23:47:23.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:22 smithi044 ceph-mon[34463]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-22T23:47:24.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:23 smithi031 ceph-mon[26721]: pgmap v45: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-22T23:47:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:23 smithi044 ceph-mon[34463]: pgmap v45: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-04-22T23:47:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:25 smithi031 ceph-mon[26721]: pgmap v46: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2024-04-22T23:47:26.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:25 smithi044 ceph-mon[34463]: pgmap v46: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2024-04-22T23:47:28.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:27 smithi031 ceph-mon[26721]: pgmap v47: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-04-22T23:47:28.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:27 smithi044 ceph-mon[34463]: pgmap v47: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-04-22T23:47:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:29 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:29 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:30 smithi031 ceph-mon[26721]: pgmap v48: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-04-22T23:47:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:30 smithi031 ceph-mon[26721]: mgrmap e31: smithi031.fazdcv(active, since 90s), standbys: smithi044.iglcti 2024-04-22T23:47:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:30 smithi044 ceph-mon[34463]: pgmap v48: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-04-22T23:47:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:30 smithi044 ceph-mon[34463]: mgrmap e31: smithi031.fazdcv(active, since 90s), standbys: smithi044.iglcti 2024-04-22T23:47:32.391 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:32 smithi031 ceph-mon[26721]: pgmap v49: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-04-22T23:47:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:32 smithi044 ceph-mon[34463]: pgmap v49: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-04-22T23:47:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:34 smithi031 ceph-mon[26721]: pgmap v50: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-22T23:47:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:34 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:34.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:34 smithi044 ceph-mon[34463]: pgmap v50: 65 pgs: 65 active+clean; 313 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-04-22T23:47:34.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:34.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:34 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:36.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:35 smithi031 ceph-mon[26721]: pgmap v51: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-04-22T23:47:36.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:35 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:36.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:35 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:35 smithi044 ceph-mon[34463]: pgmap v51: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 205 op/s 2024-04-22T23:47:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:35 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:35 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:38.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:37 smithi031 ceph-mon[26721]: pgmap v52: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-04-22T23:47:38.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:37 smithi044 ceph-mon[34463]: pgmap v52: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-04-22T23:47:38.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 -- 172.21.15.31:0/2484091746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40780ffbc0 msgr2=0x7f40780fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:38.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2484091746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40780ffbc0 0x7f40780fffe0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f406c009910 tx=0x7f406c009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 -- 172.21.15.31:0/2484091746 shutdown_connections 2024-04-22T23:47:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2484091746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4078100dc0 0x7f4078101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2484091746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40780ffbc0 0x7f40780fffe0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 -- 172.21.15.31:0/2484091746 >> 172.21.15.31:0/2484091746 conn(0x7f40780fb160 msgr2=0x7f40780fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 -- 172.21.15.31:0/2484091746 shutdown_connections 2024-04-22T23:47:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.785+0000 7f407ef0d700 1 -- 172.21.15.31:0/2484091746 wait complete. 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407ef0d700 1 Processor -- start 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407ef0d700 1 -- start start 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407ef0d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40780ffbc0 0x7f40781967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407ef0d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 0x7f4078196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407ef0d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078197350 con 0x7f4078100dc0 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407ef0d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078197490 con 0x7f40780ffbc0 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f4077fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 0x7f4078196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:38.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407cca9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40780ffbc0 0x7f40781967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f4077fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 0x7f4078196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58980/0 (socket says 172.21.15.31:58980) 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f4077fff700 1 -- 172.21.15.31:0/2998939988 learned_addr learned my addr 172.21.15.31:0/2998939988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407cca9700 1 -- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 msgr2=0x7f4078196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407cca9700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 0x7f4078196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f407cca9700 1 -- 172.21.15.31:0/2998939988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0095f0 con 0x7f40780ffbc0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.786+0000 7f4077fff700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 0x7f4078196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.787+0000 7f407cca9700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40780ffbc0 0x7f40781967f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f406c000c00 tx=0x7f406c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.787+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406c01d070 con 0x7f40780ffbc0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.787+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407819bee0 con 0x7f40780ffbc0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.787+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f406c005dc0 con 0x7f40780ffbc0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.787+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f406c017400 con 0x7f40780ffbc0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.787+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407819c350 con 0x7f40780ffbc0 2024-04-22T23:47:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.788+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078068af0 con 0x7f40780ffbc0 2024-04-22T23:47:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.789+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f406c003810 con 0x7f40780ffbc0 2024-04-22T23:47:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.789+0000 7f4075ffb700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f406806e940 0x7f4068070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.789+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f406c08f770 con 0x7f40780ffbc0 2024-04-22T23:47:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.789+0000 7f4077fff700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f406806e940 0x7f4068070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.790+0000 7f4077fff700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f406806e940 0x7f4068070e00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4064011550 tx=0x7f4064009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:38.794+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f406c05c530 con 0x7f40780ffbc0 2024-04-22T23:47:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.122+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40780695e0 con 0x7f406806e940 2024-04-22T23:47:39.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.124+0000 7f4075ffb700 1 -- 172.21.15.31:0/2998939988 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f40780695e0 con 0x7f406806e940 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f406806e940 msgr2=0x7f4068070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f406806e940 0x7f4068070e00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4064011550 tx=0x7f4064009260 comp rx=0 tx=0).stop 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40780ffbc0 msgr2=0x7f40781967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40780ffbc0 0x7f40781967f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f406c000c00 tx=0x7f406c00f690 comp rx=0 tx=0).stop 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 shutdown_connections 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40780ffbc0 0x7f40781967f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f406806e940 0x7f4068070e00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 --2- 172.21.15.31:0/2998939988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4078100dc0 0x7f4078196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 >> 172.21.15.31:0/2998939988 conn(0x7f40780fb160 msgr2=0x7f4078068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 shutdown_connections 2024-04-22T23:47:39.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.128+0000 7f407ef0d700 1 -- 172.21.15.31:0/2998939988 wait complete. 2024-04-22T23:47:39.134 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:47:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 -- 172.21.15.31:0/3671369760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 msgr2=0x7fab78101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 --2- 172.21.15.31:0/3671369760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab78101270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009910 tx=0x7fab6c009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 -- 172.21.15.31:0/3671369760 shutdown_connections 2024-04-22T23:47:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 --2- 172.21.15.31:0/3671369760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab78101270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 --2- 172.21.15.31:0/3671369760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab78100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 -- 172.21.15.31:0/3671369760 >> 172.21.15.31:0/3671369760 conn(0x7fab780fb190 msgr2=0x7fab780fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 -- 172.21.15.31:0/3671369760 shutdown_connections 2024-04-22T23:47:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.258+0000 7fab7cf62700 1 -- 172.21.15.31:0/3671369760 wait complete. 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7cf62700 1 Processor -- start 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7cf62700 1 -- start start 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7cf62700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab780690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7cf62700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab7819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7cf62700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78197070 con 0x7fab78100e10 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7cf62700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7819db40 con 0x7fab780ffc10 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab75d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab7819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab780690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:39.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab75d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab7819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58994/0 (socket says 172.21.15.31:58994) 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab7659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab780690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.31:53526/0 (socket says 172.21.15.31:53526) 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.259+0000 7fab75d9b700 1 -- 172.21.15.31:0/2866113179 learned_addr learned my addr 172.21.15.31:0/2866113179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab7659c700 1 -- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 msgr2=0x7fab7819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab7659c700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab7819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab7659c700 1 -- 172.21.15.31:0/2866113179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0095f0 con 0x7fab780ffc10 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab7659c700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab780690e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e940 tx=0x7fab6800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab680041f0 con 0x7fab780ffc10 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7819ddf0 con 0x7fab780ffc10 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fab68004350 con 0x7fab780ffc10 2024-04-22T23:47:39.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7819e310 con 0x7fab780ffc10 2024-04-22T23:47:39.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.260+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab68003cb0 con 0x7fab780ffc10 2024-04-22T23:47:39.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.261+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab78108700 con 0x7fab780ffc10 2024-04-22T23:47:39.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.262+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fab6800c7c0 con 0x7fab780ffc10 2024-04-22T23:47:39.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.263+0000 7fab677fe700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fab6006e940 0x7fab60070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:39.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.263+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fab68014070 con 0x7fab780ffc10 2024-04-22T23:47:39.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.263+0000 7fab75d9b700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fab6006e940 0x7fab60070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:39.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.264+0000 7fab75d9b700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fab6006e940 0x7fab60070e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab78069660 tx=0x7fab6c005940 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:39.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.271+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fab6805b9a0 con 0x7fab780ffc10 2024-04-22T23:47:39.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.526+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab78105760 con 0x7fab6006e940 2024-04-22T23:47:39.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.528+0000 7fab677fe700 1 -- 172.21.15.31:0/2866113179 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fab78105760 con 0x7fab6006e940 2024-04-22T23:47:39.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fab6006e940 msgr2=0x7fab60070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fab6006e940 0x7fab60070e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab78069660 tx=0x7fab6c005940 comp rx=0 tx=0).stop 2024-04-22T23:47:39.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 msgr2=0x7fab780690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab780690e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e940 tx=0x7fab6800ec50 comp rx=0 tx=0).stop 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 shutdown_connections 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab780ffc10 0x7fab780690e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fab6006e940 0x7fab60070e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.530+0000 7fab7cf62700 1 --2- 172.21.15.31:0/2866113179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100e10 0x7fab7819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.531+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 >> 172.21.15.31:0/2866113179 conn(0x7fab780fb190 msgr2=0x7fab78104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.531+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 shutdown_connections 2024-04-22T23:47:39.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.531+0000 7fab7cf62700 1 -- 172.21.15.31:0/2866113179 wait complete. 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.654+0000 7f7dd6a50700 1 -- 172.21.15.31:0/3313646090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 msgr2=0x7f7dd0106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.654+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/3313646090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd0106560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009870 tx=0x7f7dc0009b80 comp rx=0 tx=0).stop 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.654+0000 7f7dd6a50700 1 -- 172.21.15.31:0/3313646090 shutdown_connections 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.654+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/3313646090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd0106560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.654+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/3313646090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd0101840 0x7f7dd0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.654+0000 7f7dd6a50700 1 -- 172.21.15.31:0/3313646090 >> 172.21.15.31:0/3313646090 conn(0x7f7dd00fb190 msgr2=0x7f7dd00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:39.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.655+0000 7f7dd6a50700 1 -- 172.21.15.31:0/3313646090 shutdown_connections 2024-04-22T23:47:39.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.655+0000 7f7dd6a50700 1 -- 172.21.15.31:0/3313646090 wait complete. 2024-04-22T23:47:39.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.655+0000 7f7dd6a50700 1 Processor -- start 2024-04-22T23:47:39.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.655+0000 7f7dd6a50700 1 -- start start 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dd6a50700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd0101840 0x7f7dd0196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dd6a50700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd01969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dd6a50700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd0196fe0 con 0x7f7dd0104170 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dd6a50700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd0197120 con 0x7f7dd0101840 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd01969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd01969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59012/0 (socket says 172.21.15.31:59012) 2024-04-22T23:47:39.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 -- 172.21.15.31:0/2545895160 learned_addr learned my addr 172.21.15.31:0/2545895160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcffff700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd0101840 0x7f7dd0196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 -- 172.21.15.31:0/2545895160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd0101840 msgr2=0x7f7dd0196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd0101840 0x7f7dd0196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 -- 172.21.15.31:0/2545895160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0009520 con 0x7f7dd0104170 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.656+0000 7f7dcf7fe700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd01969c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009230 tx=0x7f7dc000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.657+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc001c070 con 0x7f7dd0104170 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.657+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7dc0005d20 con 0x7f7dd0104170 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.657+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd019bb70 con 0x7f7dd0104170 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.657+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc00173f0 con 0x7f7dd0104170 2024-04-22T23:47:39.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.657+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd019bfa0 con 0x7f7dd0104170 2024-04-22T23:47:39.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.658+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd0190620 con 0x7f7dd0104170 2024-04-22T23:47:39.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.659+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f7dc0020070 con 0x7f7dd0104170 2024-04-22T23:47:39.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.659+0000 7f7dcd7fa700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f7dbc06e940 0x7f7dbc070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:39.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.659+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7dc008fb70 con 0x7f7dd0104170 2024-04-22T23:47:39.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.659+0000 7f7dcffff700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f7dbc06e940 0x7f7dbc070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:39.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.660+0000 7f7dcffff700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f7dbc06e940 0x7f7dbc070e00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8003ed0 tx=0x7f7dc80091b0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:39.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.665+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f7dc005ca60 con 0x7f7dd0104170 2024-04-22T23:47:39.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.956+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dd00611d0 con 0x7f7dbc06e940 2024-04-22T23:47:39.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.970+0000 7f7dcd7fa700 1 -- 172.21.15.31:0/2545895160 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7dd00611d0 con 0x7f7dbc06e940 2024-04-22T23:47:39.963 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:47:39.963 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (9m) 6s ago 10m 25.7M - 0.25.0 c8568f914cd2 8baaa13e95f7 2024-04-22T23:47:39.963 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (10m) 6s ago 10m 7843k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (4m) 28s ago 9m 7692k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (10m) 6s ago 10m 6665k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (9m) 28s ago 9m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (8m) 6s ago 10m 87.1M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (5m) 6s ago 5m 14.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (5m) 6s ago 5m 229M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (5m) 28s ago 5m 240M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (4m) 28s ago 4m 18.1M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (2m) 6s ago 11m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (67s) 28s ago 9m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (12m) 6s ago 12m 55.1M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (9m) 28s ago 9m 43.2M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (53s) 6s ago 10m 7776k - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:47:39.964 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (33s) 28s ago 9m 5440k - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 6s ago 8m 180M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (8m) 6s ago 8m 189M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (7m) 6s ago 7m 161M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (7m) 28s ago 7m 213M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (6m) 28s ago 6m 186M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (6m) 28s ago 6m 175M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:47:39.965 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (12s) 6s ago 10m 44.0M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:47:39.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.973+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f7dbc06e940 msgr2=0x7f7dbc070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.973+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f7dbc06e940 0x7f7dbc070e00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8003ed0 tx=0x7f7dc80091b0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.973+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 msgr2=0x7f7dd01969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:39.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.973+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd01969c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009230 tx=0x7f7dc000f690 comp rx=0 tx=0).stop 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 shutdown_connections 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7dd0101840 0x7f7dd0196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f7dbc06e940 0x7f7dbc070e00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 --2- 172.21.15.31:0/2545895160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dd0104170 0x7f7dd01969c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 >> 172.21.15.31:0/2545895160 conn(0x7f7dd00fb190 msgr2=0x7f7dd00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 shutdown_connections 2024-04-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:39.974+0000 7f7dd6a50700 1 -- 172.21.15.31:0/2545895160 wait complete. 2024-04-22T23:47:40.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 -- 172.21.15.31:0/3422323542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c100e10 msgr2=0x7f735c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 --2- 172.21.15.31:0/3422323542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c100e10 0x7f735c101270 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7350009910 tx=0x7f7350009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 -- 172.21.15.31:0/3422323542 shutdown_connections 2024-04-22T23:47:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 --2- 172.21.15.31:0/3422323542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c100e10 0x7f735c101270 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 --2- 172.21.15.31:0/3422323542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735c0ffc10 0x7f735c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 -- 172.21.15.31:0/3422323542 >> 172.21.15.31:0/3422323542 conn(0x7f735c0fb190 msgr2=0x7f735c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 -- 172.21.15.31:0/3422323542 shutdown_connections 2024-04-22T23:47:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.092+0000 7f73634b4700 1 -- 172.21.15.31:0/3422323542 wait complete. 2024-04-22T23:47:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: pgmap v53: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-04-22T23:47:40.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='client.24537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:39 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.093+0000 7f73634b4700 1 Processor -- start 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.093+0000 7f73634b4700 1 -- start start 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.093+0000 7f73634b4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 0x7f735c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.093+0000 7f73634b4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735c100e10 0x7f735c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f73634b4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c102730 con 0x7f735c0ffc10 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f73634b4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c102870 con 0x7f735c100e10 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 0x7f735c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 0x7f735c103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59028/0 (socket says 172.21.15.31:59028) 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 -- 172.21.15.31:0/2833218290 learned_addr learned my addr 172.21.15.31:0/2833218290 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7360a4f700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735c100e10 0x7f735c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 -- 172.21.15.31:0/2833218290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735c100e10 msgr2=0x7f735c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735c100e10 0x7f735c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 -- 172.21.15.31:0/2833218290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73500095f0 con 0x7f735c0ffc10 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f7361250700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 0x7f735c103ba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f734c00efc0 tx=0x7f734c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f734c018070 con 0x7f735c0ffc10 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f734c00f040 con 0x7f735c0ffc10 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.094+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f734c004490 con 0x7f735c0ffc10 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.095+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c102af0 con 0x7f735c0ffc10 2024-04-22T23:47:40.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.095+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c103040 con 0x7f735c0ffc10 2024-04-22T23:47:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.096+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c108700 con 0x7f735c0ffc10 2024-04-22T23:47:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.100+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f734c0046c0 con 0x7f735c0ffc10 2024-04-22T23:47:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.100+0000 7f735a7fc700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f734406e990 0x7f7344070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.101+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f734c08f480 con 0x7f735c0ffc10 2024-04-22T23:47:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.101+0000 7f7360a4f700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f734406e990 0x7f7344070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.102+0000 7f7360a4f700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f734406e990 0x7f7344070e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7350009fd0 tx=0x7f7350005940 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:40.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.107+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f734c05c2a0 con 0x7f735c0ffc10 2024-04-22T23:47:40.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: pgmap v53: 65 pgs: 65 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-04-22T23:47:40.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:40.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='client.24537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:39 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.484+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f735c19e9c0 con 0x7f735c0ffc10 2024-04-22T23:47:40.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.485+0000 7f735a7fc700 1 -- 172.21.15.31:0/2833218290 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f734c05be30 con 0x7f735c0ffc10 2024-04-22T23:47:40.478 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:47:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:47:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:47:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:47:40.479 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:47:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f734406e990 msgr2=0x7f7344070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f734406e990 0x7f7344070e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7350009fd0 tx=0x7f7350005940 comp rx=0 tx=0).stop 2024-04-22T23:47:40.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 msgr2=0x7f735c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 0x7f735c103ba0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f734c00efc0 tx=0x7f734c00c3a0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 shutdown_connections 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f734406e990 0x7f7344070e50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f735c0ffc10 0x7f735c103ba0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 --2- 172.21.15.31:0/2833218290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f735c100e10 0x7f735c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 >> 172.21.15.31:0/2833218290 conn(0x7f735c0fb190 msgr2=0x7f735c068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 shutdown_connections 2024-04-22T23:47:40.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.487+0000 7f73634b4700 1 -- 172.21.15.31:0/2833218290 wait complete. 2024-04-22T23:47:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 -- 172.21.15.31:0/3575660078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 msgr2=0x7f32c00fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 --2- 172.21.15.31:0/3575660078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c00fffe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009910 tx=0x7f32bc009c20 comp rx=0 tx=0).stop 2024-04-22T23:47:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 -- 172.21.15.31:0/3575660078 shutdown_connections 2024-04-22T23:47:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 --2- 172.21.15.31:0/3575660078 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0100dc0 0x7f32c0101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 --2- 172.21.15.31:0/3575660078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c00fffe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 -- 172.21.15.31:0/3575660078 >> 172.21.15.31:0/3575660078 conn(0x7f32c00fb160 msgr2=0x7f32c00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 -- 172.21.15.31:0/3575660078 shutdown_connections 2024-04-22T23:47:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.603+0000 7f32c79c0700 1 -- 172.21.15.31:0/3575660078 wait complete. 2024-04-22T23:47:40.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c79c0700 1 Processor -- start 2024-04-22T23:47:40.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c79c0700 1 -- start start 2024-04-22T23:47:40.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c79c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c0103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c79c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0100dc0 0x7f32c01021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c79c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c01026e0 con 0x7f32c00ffbc0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c79c0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c0102820 con 0x7f32c0100dc0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c575c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c0103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c575c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c0103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59044/0 (socket says 172.21.15.31:59044) 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c575c700 1 -- 172.21.15.31:0/1591402250 learned_addr learned my addr 172.21.15.31:0/1591402250 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.604+0000 7f32c4f5b700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0100dc0 0x7f32c01021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32c575c700 1 -- 172.21.15.31:0/1591402250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0100dc0 msgr2=0x7f32c01021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32c575c700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0100dc0 0x7f32c01021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32c575c700 1 -- 172.21.15.31:0/1591402250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0095f0 con 0x7f32c00ffbc0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32c575c700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c0103b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00fa00 tx=0x7f32bc00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32bc01d070 con 0x7f32c00ffbc0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c0102aa0 con 0x7f32c00ffbc0 2024-04-22T23:47:40.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32bc00fe90 con 0x7f32c00ffbc0 2024-04-22T23:47:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32bc017630 con 0x7f32c00ffbc0 2024-04-22T23:47:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.605+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c0102f90 con 0x7f32c00ffbc0 2024-04-22T23:47:40.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.606+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c0068af0 con 0x7f32c00ffbc0 2024-04-22T23:47:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.607+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f32bc003680 con 0x7f32c00ffbc0 2024-04-22T23:47:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.607+0000 7f32b67fc700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f32ac06e940 0x7f32ac070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.607+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f32bc08fca0 con 0x7f32c00ffbc0 2024-04-22T23:47:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.608+0000 7f32c4f5b700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f32ac06e940 0x7f32ac070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:40.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.608+0000 7f32c4f5b700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f32ac06e940 0x7f32ac070e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009c60 tx=0x7f32b0009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:40.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.612+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f32bc094070 con 0x7f32c00ffbc0 2024-04-22T23:47:40.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.860+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32c00695e0 con 0x7f32ac06e940 2024-04-22T23:47:40.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.862+0000 7f32b67fc700 1 -- 172.21.15.31:0/1591402250 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f32c00695e0 con 0x7f32ac06e940 2024-04-22T23:47:40.855 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:47:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:47:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:47:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:47:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:47:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-22T23:47:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:47:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-22T23:47:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-22T23:47:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:47:40.857 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:47:40.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f32ac06e940 msgr2=0x7f32ac070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f32ac06e940 0x7f32ac070e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009c60 tx=0x7f32b0009260 comp rx=0 tx=0).stop 2024-04-22T23:47:40.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 msgr2=0x7f32c0103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:40.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c0103b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00fa00 tx=0x7f32bc00fa30 comp rx=0 tx=0).stop 2024-04-22T23:47:40.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 shutdown_connections 2024-04-22T23:47:40.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f32ac06e940 0x7f32ac070e00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32c00ffbc0 0x7f32c0103b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 --2- 172.21.15.31:0/1591402250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0100dc0 0x7f32c01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:40.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 >> 172.21.15.31:0/1591402250 conn(0x7f32c00fb160 msgr2=0x7f32c0068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:40.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 shutdown_connections 2024-04-22T23:47:40.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:47:40.864+0000 7f32c79c0700 1 -- 172.21.15.31:0/1591402250 wait complete. 2024-04-22T23:47:41.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:40 smithi031 ceph-mon[26721]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:41.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:40 smithi031 ceph-mon[26721]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:41.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:40 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2833218290' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:41.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:40 smithi044 ceph-mon[34463]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:41.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:40 smithi044 ceph-mon[34463]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:41.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:40 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2833218290' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:42.310 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:41 smithi031 ceph-mon[26721]: pgmap v54: 65 pgs: 65 active+clean; 299 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-04-22T23:47:42.310 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:41 smithi031 ceph-mon[26721]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:42.310 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:41 smithi031 ceph-mon[26721]: Upgrade: Updating alertmanager.smithi031 2024-04-22T23:47:42.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:41 smithi044 ceph-mon[34463]: pgmap v54: 65 pgs: 65 active+clean; 299 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 228 op/s 2024-04-22T23:47:42.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:41 smithi044 ceph-mon[34463]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:42.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:41 smithi044 ceph-mon[34463]: Upgrade: Updating alertmanager.smithi031 2024-04-22T23:47:43.294 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:42 smithi031 ceph-mon[26721]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-22T23:47:43.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:42 smithi044 ceph-mon[34463]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-22T23:47:44.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:43 smithi031 ceph-mon[26721]: pgmap v55: 65 pgs: 65 active+clean; 299 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-22T23:47:44.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:43 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:44.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:43 smithi044 ceph-mon[34463]: pgmap v55: 65 pgs: 65 active+clean; 299 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-04-22T23:47:44.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:43 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:44.477 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-22T23:47:44.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T23:47:44.851 DEBUG:teuthology.parallel:result is None 2024-04-22T23:47:46.163 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:45 smithi031 ceph-mon[26721]: pgmap v56: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-22T23:47:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:45 smithi044 ceph-mon[34463]: pgmap v56: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-22T23:47:48.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:47 smithi031 ceph-mon[26721]: pgmap v57: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-22T23:47:48.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:47 smithi044 ceph-mon[34463]: pgmap v57: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-22T23:47:49.542 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:49 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.542 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:49 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.542 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:49 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:49.543 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:49 smithi031 ceph-mon[26721]: pgmap v58: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-22T23:47:49.543 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:49 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.543 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:49 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:49 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:49 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:49 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:47:49.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:49 smithi044 ceph-mon[34463]: pgmap v58: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-22T23:47:49.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:49 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:49.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:49 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:52.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:51 smithi031 ceph-mon[26721]: pgmap v59: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-04-22T23:47:52.412 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:51 smithi044 ceph-mon[34463]: pgmap v59: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-04-22T23:47:52.543 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-22T23:47:52.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-22T23:47:52.831 DEBUG:teuthology.parallel:result is None 2024-04-22T23:47:52.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T23:47:52.871 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T23:47:52.872 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-22T23:47:52.930 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-22T23:47:52.930 DEBUG:teuthology.parallel:result is None 2024-04-22T23:47:54.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:53 smithi031 ceph-mon[26721]: pgmap v60: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-22T23:47:54.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:53 smithi044 ceph-mon[34463]: pgmap v60: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 174 op/s 2024-04-22T23:47:56.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:55 smithi031 ceph-mon[26721]: pgmap v61: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-22T23:47:56.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:55 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:56.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:55 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:55 smithi044 ceph-mon[34463]: pgmap v61: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 252 op/s 2024-04-22T23:47:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:55 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:55 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:57.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:56 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:57.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:56 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:57.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:56 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:57.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:56 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:47:58.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:57 smithi031 ceph-mon[26721]: pgmap v62: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 957 KiB/s rd, 865 KiB/s wr, 176 op/s 2024-04-22T23:47:58.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:57 smithi044 ceph-mon[34463]: pgmap v62: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 957 KiB/s rd, 865 KiB/s wr, 176 op/s 2024-04-22T23:47:59.316 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:58 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:58 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:00.289 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: pgmap v63: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 957 KiB/s rd, 865 KiB/s wr, 176 op/s 2024-04-22T23:48:00.289 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:00.289 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:00.289 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:00.289 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:47:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: pgmap v63: 65 pgs: 65 active+clean; 264 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 957 KiB/s rd, 865 KiB/s wr, 176 op/s 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:00.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:47:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:00 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:48:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:01 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:48:02.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:02 smithi031 ceph-mon[26721]: pgmap v64: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 963 KiB/s rd, 871 KiB/s wr, 180 op/s 2024-04-22T23:48:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:02 smithi044 ceph-mon[34463]: pgmap v64: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 963 KiB/s rd, 871 KiB/s wr, 180 op/s 2024-04-22T23:48:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:03 smithi044 ceph-mon[34463]: Upgrade: Updating grafana.smithi031 2024-04-22T23:48:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:03 smithi044 ceph-mon[34463]: Deploying daemon grafana.smithi031 on smithi031 2024-04-22T23:48:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:03 smithi031 ceph-mon[26721]: Upgrade: Updating grafana.smithi031 2024-04-22T23:48:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:03 smithi031 ceph-mon[26721]: Deploying daemon grafana.smithi031 on smithi031 2024-04-22T23:48:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:04 smithi031 ceph-mon[26721]: pgmap v65: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 426 KiB/s rd, 386 KiB/s wr, 82 op/s 2024-04-22T23:48:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:04 smithi044 ceph-mon[34463]: pgmap v65: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 426 KiB/s rd, 386 KiB/s wr, 82 op/s 2024-04-22T23:48:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:06 smithi031 ceph-mon[26721]: pgmap v66: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 427 KiB/s rd, 386 KiB/s wr, 83 op/s 2024-04-22T23:48:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:06 smithi044 ceph-mon[34463]: pgmap v66: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 427 KiB/s rd, 386 KiB/s wr, 83 op/s 2024-04-22T23:48:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:08 smithi044 ceph-mon[34463]: pgmap v67: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-22T23:48:08.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:08 smithi031 ceph-mon[26721]: pgmap v67: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-22T23:48:10.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:10 smithi031 ceph-mon[26721]: pgmap v68: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-22T23:48:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:10 smithi044 ceph-mon[34463]: pgmap v68: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-22T23:48:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 -- 172.21.15.31:0/1659525923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 msgr2=0x7f4e6c103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 --2- 172.21.15.31:0/1659525923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c103b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009910 tx=0x7f4e68009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 -- 172.21.15.31:0/1659525923 shutdown_connections 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 --2- 172.21.15.31:0/1659525923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6c1040d0 0x7f4e6c1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 --2- 172.21.15.31:0/1659525923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c103b90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 -- 172.21.15.31:0/1659525923 >> 172.21.15.31:0/1659525923 conn(0x7f4e6c0fb190 msgr2=0x7f4e6c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 -- 172.21.15.31:0/1659525923 shutdown_connections 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.986+0000 7f4e73e52700 1 -- 172.21.15.31:0/1659525923 wait complete. 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e73e52700 1 Processor -- start 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e73e52700 1 -- start start 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e73e52700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e73e52700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6c1040d0 0x7f4e6c0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e73e52700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ffc90 con 0x7f4e6c1017a0 2024-04-22T23:48:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e73e52700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0ffdd0 con 0x7f4e6c1040d0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e71bee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e71bee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38982/0 (socket says 172.21.15.31:38982) 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.987+0000 7f4e71bee700 1 -- 172.21.15.31:0/2589420699 learned_addr learned my addr 172.21.15.31:0/2589420699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e713ed700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6c1040d0 0x7f4e6c0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e71bee700 1 -- 172.21.15.31:0/2589420699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6c1040d0 msgr2=0x7f4e6c0ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e71bee700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6c1040d0 0x7f4e6c0ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e71bee700 1 -- 172.21.15.31:0/2589420699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680095f0 con 0x7f4e6c1017a0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e71bee700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c101100 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005b40 tx=0x7f4e6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6801d070 con 0x7f4e6c1017a0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c100050 con 0x7f4e6c1017a0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e68022470 con 0x7f4e6c1017a0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6800f690 con 0x7f4e6c1017a0 2024-04-22T23:48:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.988+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c100540 con 0x7f4e6c1017a0 2024-04-22T23:48:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.989+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c190690 con 0x7f4e6c1017a0 2024-04-22T23:48:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.990+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f4e680225e0 con 0x7f4e6c1017a0 2024-04-22T23:48:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.990+0000 7f4e62ffd700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f4e5806e940 0x7f4e58070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.990+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4e68090130 con 0x7f4e6c1017a0 2024-04-22T23:48:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.990+0000 7f4e713ed700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f4e5806e940 0x7f4e58070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.991+0000 7f4e713ed700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f4e5806e940 0x7f4e58070e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006fd0 tx=0x7f4e5c008040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:10.995+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f4e6805cf70 con 0x7f4e6c1017a0 2024-04-22T23:48:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.233+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e6c0611d0 con 0x7f4e5806e940 2024-04-22T23:48:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.234+0000 7f4e62ffd700 1 -- 172.21.15.31:0/2589420699 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f4e6c0611d0 con 0x7f4e5806e940 2024-04-22T23:48:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f4e5806e940 msgr2=0x7f4e58070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f4e5806e940 0x7f4e58070e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006fd0 tx=0x7f4e5c008040 comp rx=0 tx=0).stop 2024-04-22T23:48:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 msgr2=0x7f4e6c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c101100 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005b40 tx=0x7f4e6800be20 comp rx=0 tx=0).stop 2024-04-22T23:48:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 shutdown_connections 2024-04-22T23:48:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f4e5806e940 0x7f4e58070e00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6c1017a0 0x7f4e6c101100 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 --2- 172.21.15.31:0/2589420699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e6c1040d0 0x7f4e6c0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.236+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 >> 172.21.15.31:0/2589420699 conn(0x7f4e6c0fb190 msgr2=0x7f4e6c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.237+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 shutdown_connections 2024-04-22T23:48:11.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.237+0000 7f4e73e52700 1 -- 172.21.15.31:0/2589420699 wait complete. 2024-04-22T23:48:11.239 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:48:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.344+0000 7f6c4fec4700 1 -- 172.21.15.31:0/2106664168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48104090 msgr2=0x7f6c48106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.344+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/2106664168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48104090 0x7f6c48106480 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009910 tx=0x7f6c38009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 -- 172.21.15.31:0/2106664168 shutdown_connections 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/2106664168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48104090 0x7f6c48106480 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/2106664168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c48101760 0x7f6c48103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 -- 172.21.15.31:0/2106664168 >> 172.21.15.31:0/2106664168 conn(0x7f6c480fb150 msgr2=0x7f6c480fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 -- 172.21.15.31:0/2106664168 shutdown_connections 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 -- 172.21.15.31:0/2106664168 wait complete. 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 Processor -- start 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 -- start start 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.345+0000 7f6c4fec4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 0x7f6c48196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4fec4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c48104090 0x7f6c48196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4fec4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48197000 con 0x7f6c48101760 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4fec4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48197140 con 0x7f6c48104090 2024-04-22T23:48:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 0x7f6c48196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 0x7f6c48196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38996/0 (socket says 172.21.15.31:38996) 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 -- 172.21.15.31:0/1851814637 learned_addr learned my addr 172.21.15.31:0/1851814637 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4d45f700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c48104090 0x7f6c48196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 -- 172.21.15.31:0/1851814637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c48104090 msgr2=0x7f6c48196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c48104090 0x7f6c48196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 -- 172.21.15.31:0/1851814637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380095f0 con 0x7f6c48101760 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4dc60700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 0x7f6c48196530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400c930 tx=0x7f6c4400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c44007ad0 con 0x7f6c48101760 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6c44007c30 con 0x7f6c48101760 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4819bc00 con 0x7f6c48101760 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c44018630 con 0x7f6c48101760 2024-04-22T23:48:11.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.346+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4819c120 con 0x7f6c48101760 2024-04-22T23:48:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.347+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c48190640 con 0x7f6c48101760 2024-04-22T23:48:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.348+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f6c4400f450 con 0x7f6c48101760 2024-04-22T23:48:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.348+0000 7f6c3effd700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6c3406e940 0x7f6c34070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.349+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6c4408eb30 con 0x7f6c48101760 2024-04-22T23:48:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.349+0000 7f6c4d45f700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6c3406e940 0x7f6c34070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.349+0000 7f6c4d45f700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6c3406e940 0x7f6c34070e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800b5c0 tx=0x7f6c38005fb0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.353+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f6c4405b8f0 con 0x7f6c48101760 2024-04-22T23:48:11.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.587+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c480611d0 con 0x7f6c3406e940 2024-04-22T23:48:11.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.588+0000 7f6c3effd700 1 -- 172.21.15.31:0/1851814637 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f6c480611d0 con 0x7f6c3406e940 2024-04-22T23:48:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6c3406e940 msgr2=0x7f6c34070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6c3406e940 0x7f6c34070e00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800b5c0 tx=0x7f6c38005fb0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 msgr2=0x7f6c48196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 0x7f6c48196530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400c930 tx=0x7f6c4400ccf0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 shutdown_connections 2024-04-22T23:48:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f6c3406e940 0x7f6c34070e00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c48101760 0x7f6c48196530 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 --2- 172.21.15.31:0/1851814637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c48104090 0x7f6c48196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 >> 172.21.15.31:0/1851814637 conn(0x7f6c480fb150 msgr2=0x7f6c480fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 shutdown_connections 2024-04-22T23:48:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.590+0000 7f6c4fec4700 1 -- 172.21.15.31:0/1851814637 wait complete. 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 -- 172.21.15.31:0/2269994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 msgr2=0x7fe67c0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 --2- 172.21.15.31:0/2269994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c0ff380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009910 tx=0x7fe66c009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 -- 172.21.15.31:0/2269994161 shutdown_connections 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 --2- 172.21.15.31:0/2269994161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101f40 0x7fe67c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 --2- 172.21.15.31:0/2269994161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c0ff380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 -- 172.21.15.31:0/2269994161 >> 172.21.15.31:0/2269994161 conn(0x7fe67c0fa8c0 msgr2=0x7fe67c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 -- 172.21.15.31:0/2269994161 shutdown_connections 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.698+0000 7fe68413c700 1 -- 172.21.15.31:0/2269994161 wait complete. 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe68413c700 1 Processor -- start 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe68413c700 1 -- start start 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe68413c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe68413c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101f40 0x7fe67c196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe68413c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c197090 con 0x7fe67c0fef60 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe68413c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c1971d0 con 0x7fe67c101f40 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe681ed8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe681ed8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39010/0 (socket says 172.21.15.31:39010) 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe681ed8700 1 -- 172.21.15.31:0/1653675177 learned_addr learned my addr 172.21.15.31:0/1653675177 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe6816d7700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101f40 0x7fe67c196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe681ed8700 1 -- 172.21.15.31:0/1653675177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101f40 msgr2=0x7fe67c196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe681ed8700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101f40 0x7fe67c196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.699+0000 7fe681ed8700 1 -- 172.21.15.31:0/1653675177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0095f0 con 0x7fe67c0fef60 2024-04-22T23:48:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.700+0000 7fe681ed8700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c196530 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00bda0 tx=0x7fe66c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.700+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66c01d070 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.700+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe66c00f460 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.700+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c19bc20 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.700+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66c021620 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.700+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c19c110 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.701+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c109820 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.702+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fe66c00f5d0 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.702+0000 7fe672ffd700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fe66806e940 0x7fe668070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.702+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe66c08fb80 con 0x7fe67c0fef60 2024-04-22T23:48:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.702+0000 7fe6816d7700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fe66806e940 0x7fe668070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.703+0000 7fe6816d7700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fe66806e940 0x7fe668070e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe678004630 tx=0x7fe67800b3c0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:11.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.711+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fe66c05c940 con 0x7fe67c0fef60 2024-04-22T23:48:11.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.953+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe67c106890 con 0x7fe66806e940 2024-04-22T23:48:11.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.964+0000 7fe672ffd700 1 -- 172.21.15.31:0/1653675177 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe67c106890 con 0x7fe66806e940 2024-04-22T23:48:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:48:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (24s) 17s ago 10m 16.1M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:48:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (11m) 17s ago 11m 7872k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:48:11.956 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (5m) 60s ago 10m 7692k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (11m) 17s ago 11m 6665k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (10m) 60s ago 10m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (9m) 17s ago 10m 87.9M - 9.4.7 954c08fa6188 863259e8e386 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (5m) 17s ago 5m 14.5M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (5m) 17s ago 5m 157M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (5m) 60s ago 5m 240M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (5m) 60s ago 5m 18.1M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (2m) 17s ago 12m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (99s) 60s ago 10m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (12m) 17s ago 12m 55.9M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (9m) 60s ago 9m 43.2M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (85s) 17s ago 11m 8619k - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (65s) 60s ago 10m 5440k - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:48:11.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (9m) 17s ago 9m 196M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:48:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (8m) 17s ago 8m 204M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:48:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (8m) 17s ago 8m 173M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:48:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (7m) 60s ago 7m 213M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:48:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (7m) 60s ago 7m 186M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:48:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (7m) 60s ago 7m 175M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:48:11.958 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (44s) 17s ago 10m 53.5M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fe66806e940 msgr2=0x7fe668070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fe66806e940 0x7fe668070e00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe678004630 tx=0x7fe67800b3c0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 msgr2=0x7fe67c196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c196530 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00bda0 tx=0x7fe66c00bdd0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 shutdown_connections 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fe66806e940 0x7fe668070e00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe67c0fef60 0x7fe67c196530 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 --2- 172.21.15.31:0/1653675177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe67c101f40 0x7fe67c196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.966+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 >> 172.21.15.31:0/1653675177 conn(0x7fe67c0fa8c0 msgr2=0x7fe67c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.967+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 shutdown_connections 2024-04-22T23:48:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:11.967+0000 7fe68413c700 1 -- 172.21.15.31:0/1653675177 wait complete. 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 -- 172.21.15.31:0/4075018397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 msgr2=0x7ff2b00ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 --2- 172.21.15.31:0/4075018397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b00ff160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009910 tx=0x7ff2a4009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 -- 172.21.15.31:0/4075018397 shutdown_connections 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 --2- 172.21.15.31:0/4075018397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b00ff160 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 --2- 172.21.15.31:0/4075018397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 0x7ff2b01026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 -- 172.21.15.31:0/4075018397 >> 172.21.15.31:0/4075018397 conn(0x7ff2b00fa8c0 msgr2=0x7ff2b00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:12.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 -- 172.21.15.31:0/4075018397 shutdown_connections 2024-04-22T23:48:12.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.079+0000 7ff2b5041700 1 -- 172.21.15.31:0/4075018397 wait complete. 2024-04-22T23:48:12.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.080+0000 7ff2b5041700 1 Processor -- start 2024-04-22T23:48:12.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.080+0000 7ff2b5041700 1 -- start start 2024-04-22T23:48:12.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.080+0000 7ff2b5041700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b0196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:12.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2b5041700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 0x7ff2b0196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:12.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2b5041700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0197020 con 0x7ff2b00fed00 2024-04-22T23:48:12.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2b5041700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0197160 con 0x7ff2b0102280 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b0196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b0196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39020/0 (socket says 172.21.15.31:39020) 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 -- 172.21.15.31:0/1655774848 learned_addr learned my addr 172.21.15.31:0/1655774848 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2ae59c700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 0x7ff2b0196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 -- 172.21.15.31:0/1655774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 msgr2=0x7ff2b0196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 0x7ff2b0196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 -- 172.21.15.31:0/1655774848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40095f0 con 0x7ff2b00fed00 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2ae59c700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 0x7ff2b0196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.081+0000 7ff2aed9d700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b0196550 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000cfd0 tx=0x7ff2a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.082+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2a000ad70 con 0x7ff2b00fed00 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.082+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b019bc20 con 0x7ff2b00fed00 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.082+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2a0004020 con 0x7ff2b00fed00 2024-04-22T23:48:12.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.082+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2a0012650 con 0x7ff2b00fed00 2024-04-22T23:48:12.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.082+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b019c140 con 0x7ff2b00fed00 2024-04-22T23:48:12.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.083+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0109810 con 0x7ff2b00fed00 2024-04-22T23:48:12.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.083+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7ff2a00128d0 con 0x7ff2b00fed00 2024-04-22T23:48:12.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.084+0000 7ff297fff700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff29806e940 0x7ff298070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:12.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.084+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff2a008eb70 con 0x7ff2b00fed00 2024-04-22T23:48:12.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.084+0000 7ff2ae59c700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff29806e940 0x7ff298070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:12.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.084+0000 7ff2ae59c700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff29806e940 0x7ff298070e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b5c0 tx=0x7ff2a4005ab0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:12.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.088+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7ff2a005b930 con 0x7ff2b00fed00 2024-04-22T23:48:12.127 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:12 smithi031 ceph-mon[26721]: pgmap v69: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-04-22T23:48:12.393 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:48:12.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:48:12.393 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:48:12.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:48:12.393 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:48:12.394 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:48:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.400+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2b019c750 con 0x7ff2b00fed00 2024-04-22T23:48:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.401+0000 7ff297fff700 1 -- 172.21.15.31:0/1655774848 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff2a005b4c0 con 0x7ff2b00fed00 2024-04-22T23:48:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.403+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff29806e940 msgr2=0x7ff298070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.403+0000 7ff2b5041700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff29806e940 0x7ff298070e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b5c0 tx=0x7ff2a4005ab0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.403+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 msgr2=0x7ff2b0196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.403+0000 7ff2b5041700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b0196550 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000cfd0 tx=0x7ff2a000a340 comp rx=0 tx=0).stop 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 shutdown_connections 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff29806e940 0x7ff298070e00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2b00fed00 0x7ff2b0196550 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 --2- 172.21.15.31:0/1655774848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2b0102280 0x7ff2b0196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 >> 172.21.15.31:0/1655774848 conn(0x7ff2b00fa8c0 msgr2=0x7ff2b0105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 shutdown_connections 2024-04-22T23:48:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.404+0000 7ff2b5041700 1 -- 172.21.15.31:0/1655774848 wait complete. 2024-04-22T23:48:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:12 smithi044 ceph-mon[34463]: pgmap v69: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-04-22T23:48:12.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 -- 172.21.15.31:0/958238441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c104080 msgr2=0x7f251c106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 --2- 172.21.15.31:0/958238441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c104080 0x7f251c106470 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2510009910 tx=0x7f2510009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 -- 172.21.15.31:0/958238441 shutdown_connections 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 --2- 172.21.15.31:0/958238441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c104080 0x7f251c106470 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 --2- 172.21.15.31:0/958238441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f251c101750 0x7f251c103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 -- 172.21.15.31:0/958238441 >> 172.21.15.31:0/958238441 conn(0x7f251c0fb160 msgr2=0x7f251c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 -- 172.21.15.31:0/958238441 shutdown_connections 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.521+0000 7f2520c1a700 1 -- 172.21.15.31:0/958238441 wait complete. 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2520c1a700 1 Processor -- start 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2520c1a700 1 -- start start 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2520c1a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 0x7f251c1010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2520c1a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f251c104080 0x7f251c0ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:12.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2520c1a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0ffc40 con 0x7f251c101750 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2520c1a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0ffd80 con 0x7f251c104080 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f251a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 0x7f251c1010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f251a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 0x7f251c1010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39040/0 (socket says 172.21.15.31:39040) 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f251a59c700 1 -- 172.21.15.31:0/3617769867 learned_addr learned my addr 172.21.15.31:0/3617769867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f2519d9b700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f251c104080 0x7f251c0ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f251a59c700 1 -- 172.21.15.31:0/3617769867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f251c104080 msgr2=0x7f251c0ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f251a59c700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f251c104080 0x7f251c0ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.522+0000 7f251a59c700 1 -- 172.21.15.31:0/3617769867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25100095f0 con 0x7f251c101750 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.523+0000 7f251a59c700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 0x7f251c1010b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b4b0 tx=0x7f250c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.523+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250c011800 con 0x7f251c101750 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.523+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f250c011e10 con 0x7f251c101750 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.523+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250c00f2e0 con 0x7f251c101750 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.523+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c100060 con 0x7f251c101750 2024-04-22T23:48:12.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.523+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c19e8c0 con 0x7f251c101750 2024-04-22T23:48:12.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.524+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c1906b0 con 0x7f251c101750 2024-04-22T23:48:12.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.524+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f250c011960 con 0x7f251c101750 2024-04-22T23:48:12.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.525+0000 7f250b7fe700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f250406e940 0x7f2504070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:12.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.525+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f250c08e4e0 con 0x7f251c101750 2024-04-22T23:48:12.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.525+0000 7f2519d9b700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f250406e940 0x7f2504070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:12.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.526+0000 7f2519d9b700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f250406e940 0x7f2504070e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25100059d0 tx=0x7f2510005920 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.530+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f250c05b320 con 0x7f251c101750 2024-04-22T23:48:12.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.758+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f251c02d120 con 0x7f250406e940 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.759+0000 7f250b7fe700 1 -- 172.21.15.31:0/3617769867 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f251c02d120 con 0x7f250406e940 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:48:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading grafana daemons", 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f250406e940 msgr2=0x7f2504070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f250406e940 0x7f2504070e00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25100059d0 tx=0x7f2510005920 comp rx=0 tx=0).stop 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 msgr2=0x7f251c1010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:12.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 0x7f251c1010b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b4b0 tx=0x7f250c00b870 comp rx=0 tx=0).stop 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 shutdown_connections 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f250406e940 0x7f2504070e00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c101750 0x7f251c1010b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 --2- 172.21.15.31:0/3617769867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f251c104080 0x7f251c0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 >> 172.21.15.31:0/3617769867 conn(0x7f251c0fb160 msgr2=0x7f251c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 shutdown_connections 2024-04-22T23:48:12.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:12.762+0000 7f2520c1a700 1 -- 172.21.15.31:0/3617769867 wait complete. 2024-04-22T23:48:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:13 smithi044 ceph-mon[34463]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:13 smithi044 ceph-mon[34463]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:13 smithi044 ceph-mon[34463]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:13 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1655774848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:13.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:13 smithi031 ceph-mon[26721]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:13.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:13 smithi031 ceph-mon[26721]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:13.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:13 smithi031 ceph-mon[26721]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:13.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:13 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1655774848' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:14.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:14 smithi031 ceph-mon[26721]: pgmap v70: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-22T23:48:14.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:14 smithi031 ceph-mon[26721]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:14.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:14.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:14.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:14.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:14 smithi044 ceph-mon[34463]: pgmap v70: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-22T23:48:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:14 smithi044 ceph-mon[34463]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:16.367 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:16 smithi031 ceph-mon[26721]: pgmap v71: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-22T23:48:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:16 smithi044 ceph-mon[34463]: pgmap v71: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-22T23:48:18.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:18 smithi031 ceph-mon[26721]: pgmap v72: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:18.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:18 smithi044 ceph-mon[34463]: pgmap v72: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:19.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:19.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:19 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:19.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:19 smithi031 ceph-mon[26721]: pgmap v73: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:19.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:19.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:19 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:19.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:19 smithi044 ceph-mon[34463]: pgmap v73: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:20.805 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:20 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:20.805 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:20 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:20.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:20 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:20.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:20 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:21.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:21 smithi031 ceph-mon[26721]: pgmap v74: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:21.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:21 smithi044 ceph-mon[34463]: pgmap v74: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:24.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: pgmap v75: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:24.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:24.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:24.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:48:24.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:48:24.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:24.197 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.198 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: Upgrade: Finalizing container_image settings 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-22T23:48:24.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-22T23:48:24.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-22T23:48:24.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-22T23:48:24.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-22T23:48:24.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-22T23:48:24.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-22T23:48:24.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-22T23:48:24.202 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.202 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-22T23:48:24.202 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-22T23:48:24.202 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-22T23:48:24.202 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-22T23:48:24.202 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-22T23:48:24.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-22T23:48:24.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-22T23:48:24.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: Upgrade: Complete! 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:24.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:24.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:24.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:24.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:24.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:24.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.206 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:23 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: pgmap v75: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:24.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:24.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:24.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:48:24.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:48:24.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:24.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: Upgrade: Finalizing container_image settings 2024-04-22T23:48:24.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-22T23:48:24.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-22T23:48:24.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-22T23:48:24.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-22T23:48:24.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: Upgrade: Complete! 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-22T23:48:24.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:48:24.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:24.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:23 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:48:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:25 smithi031 ceph-mon[26721]: pgmap v76: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:26.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:25 smithi044 ceph-mon[34463]: pgmap v76: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:28.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:27 smithi031 ceph-mon[26721]: pgmap v77: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:28.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:27 smithi044 ceph-mon[34463]: pgmap v77: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:28 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:28 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:29 smithi031 ceph-mon[26721]: pgmap v78: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:29 smithi044 ceph-mon[34463]: pgmap v78: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:32.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:32 smithi031 ceph-mon[26721]: pgmap v79: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:32 smithi044 ceph-mon[34463]: pgmap v79: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:34.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:34 smithi031 ceph-mon[26721]: pgmap v80: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:34.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:34 smithi044 ceph-mon[34463]: pgmap v80: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:36.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:36 smithi031 ceph-mon[26721]: pgmap v81: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:36.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:36 smithi044 ceph-mon[34463]: pgmap v81: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:38.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:38 smithi031 ceph-mon[26721]: pgmap v82: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:38.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:38 smithi044 ceph-mon[34463]: pgmap v82: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:40.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:40 smithi031 ceph-mon[26721]: pgmap v83: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:40 smithi044 ceph-mon[34463]: pgmap v83: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:42.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:42 smithi031 ceph-mon[26721]: pgmap v84: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:42.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:42 smithi044 ceph-mon[34463]: pgmap v84: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 -- 172.21.15.31:0/4176109541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 msgr2=0x7f2a50103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 --2- 172.21.15.31:0/4176109541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a50103b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009910 tx=0x7f2a40009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 -- 172.21.15.31:0/4176109541 shutdown_connections 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 --2- 172.21.15.31:0/4176109541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a50104080 0x7f2a50106470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 --2- 172.21.15.31:0/4176109541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a50103b40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 -- 172.21.15.31:0/4176109541 >> 172.21.15.31:0/4176109541 conn(0x7f2a500fb160 msgr2=0x7f2a500fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 -- 172.21.15.31:0/4176109541 shutdown_connections 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.884+0000 7f2a54a47700 1 -- 172.21.15.31:0/4176109541 wait complete. 2024-04-22T23:48:42.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a54a47700 1 Processor -- start 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a54a47700 1 -- start start 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a54a47700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a501967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a54a47700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a50104080 0x7f2a50196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a54a47700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a50197340 con 0x7f2a50101750 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a54a47700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a50197480 con 0x7f2a50104080 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a501967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a501967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41754/0 (socket says 172.21.15.31:41754) 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4e59c700 1 -- 172.21.15.31:0/2537882125 learned_addr learned my addr 172.21.15.31:0/2537882125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4dd9b700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a50104080 0x7f2a50196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4e59c700 1 -- 172.21.15.31:0/2537882125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a50104080 msgr2=0x7f2a50196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4e59c700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a50104080 0x7f2a50196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.885+0000 7f2a4e59c700 1 -- 172.21.15.31:0/2537882125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400095f0 con 0x7f2a50101750 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.886+0000 7f2a4e59c700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a501967e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2a40006010 tx=0x7f2a4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.886+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a4001d070 con 0x7f2a50101750 2024-04-22T23:48:42.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.886+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a4000f460 con 0x7f2a50101750 2024-04-22T23:48:42.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.886+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5019bed0 con 0x7f2a50101750 2024-04-22T23:48:42.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.886+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a40021640 con 0x7f2a50101750 2024-04-22T23:48:42.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.886+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5019c340 con 0x7f2a50101750 2024-04-22T23:48:42.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.887+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a50190630 con 0x7f2a50101750 2024-04-22T23:48:42.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.888+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f2a4000f5d0 con 0x7f2a50101750 2024-04-22T23:48:42.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.888+0000 7f2a3f7fe700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2a3806e940 0x7f2a38070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:42.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.888+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2a4008fac0 con 0x7f2a50101750 2024-04-22T23:48:42.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.888+0000 7f2a4dd9b700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2a3806e940 0x7f2a38070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:42.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.889+0000 7f2a4dd9b700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2a3806e940 0x7f2a38070e00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005950 tx=0x7f2a4400b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:42.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:42.893+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f2a4005c900 con 0x7f2a50101750 2024-04-22T23:48:43.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.123+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a500611d0 con 0x7f2a3806e940 2024-04-22T23:48:43.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.124+0000 7f2a3f7fe700 1 -- 172.21.15.31:0/2537882125 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f2a500611d0 con 0x7f2a3806e940 2024-04-22T23:48:43.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2a3806e940 msgr2=0x7f2a38070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2a3806e940 0x7f2a38070e00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2a44005950 tx=0x7f2a4400b410 comp rx=0 tx=0).stop 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 msgr2=0x7f2a501967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a501967e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2a40006010 tx=0x7f2a4000bd70 comp rx=0 tx=0).stop 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 shutdown_connections 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f2a3806e940 0x7f2a38070e00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a50101750 0x7f2a501967e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 --2- 172.21.15.31:0/2537882125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a50104080 0x7f2a50196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 >> 172.21.15.31:0/2537882125 conn(0x7f2a500fb160 msgr2=0x7f2a500fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 shutdown_connections 2024-04-22T23:48:43.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:43.126+0000 7f2a54a47700 1 -- 172.21.15.31:0/2537882125 wait complete. 2024-04-22T23:48:43.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-22T23:48:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:48:44.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:44 smithi031 ceph-mon[26721]: pgmap v85: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:44.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:44 smithi031 ceph-mon[26721]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:44.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:44 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:44.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:44 smithi044 ceph-mon[34463]: pgmap v85: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:44.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:44 smithi044 ceph-mon[34463]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:44.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:44 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:45.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 -- 172.21.15.31:0/3805399634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 msgr2=0x7f9654105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:45.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 --2- 172.21.15.31:0/3805399634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f9654105890 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f963c009910 tx=0x7f963c009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:45.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 -- 172.21.15.31:0/3805399634 shutdown_connections 2024-04-22T23:48:45.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 --2- 172.21.15.31:0/3805399634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f9654105890 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 --2- 172.21.15.31:0/3805399634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9654100e90 0x7f9654101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 -- 172.21.15.31:0/3805399634 >> 172.21.15.31:0/3805399634 conn(0x7f96540fc740 msgr2=0x7f96540feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 -- 172.21.15.31:0/3805399634 shutdown_connections 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 -- 172.21.15.31:0/3805399634 wait complete. 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.568+0000 7f9659285700 1 Processor -- start 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f9659285700 1 -- start start 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f9659285700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9654100e90 0x7f965419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f9659285700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f965419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f9659285700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965419d3a0 con 0x7f9654101840 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f9659285700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654196850 con 0x7f9654100e90 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f965419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f965419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41772/0 (socket says 172.21.15.31:41772) 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 -- 172.21.15.31:0/419514558 learned_addr learned my addr 172.21.15.31:0/419514558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:45.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f9652ffd700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9654100e90 0x7f965419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 -- 172.21.15.31:0/419514558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9654100e90 msgr2=0x7f965419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9654100e90 0x7f965419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 -- 172.21.15.31:0/419514558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0095f0 con 0x7f9654101840 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.569+0000 7f96527fc700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f965419cd10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f963c0092e0 tx=0x7f963c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.570+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963c01d070 con 0x7f9654101840 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.570+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9654196ad0 con 0x7f9654101840 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.570+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f963c00f460 con 0x7f9654101840 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.570+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963c0175f0 con 0x7f9654101840 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.570+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9654196fc0 con 0x7f9654101840 2024-04-22T23:48:45.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.571+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96541091e0 con 0x7f9654101840 2024-04-22T23:48:45.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.571+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f963c00f5d0 con 0x7f9654101840 2024-04-22T23:48:45.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.572+0000 7f964bfff700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f964006e940 0x7f9640070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:45.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.572+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f963c08fef0 con 0x7f9654101840 2024-04-22T23:48:45.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.572+0000 7f9652ffd700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f964006e940 0x7f9640070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:45.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.572+0000 7f9652ffd700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f964006e940 0x7f9640070e00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9644009f20 tx=0x7f96440094f0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:45.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.576+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f963c05cc10 con 0x7f9654101840 2024-04-22T23:48:45.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.879+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9654108bc0 con 0x7f9654101840 2024-04-22T23:48:45.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.879+0000 7f964bfff700 1 -- 172.21.15.31:0/419514558 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f963c05c7a0 con 0x7f9654101840 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f964006e940 msgr2=0x7f9640070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f964006e940 0x7f9640070e00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9644009f20 tx=0x7f96440094f0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 msgr2=0x7f965419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f965419cd10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f963c0092e0 tx=0x7f963c00be20 comp rx=0 tx=0).stop 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 shutdown_connections 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9654100e90 0x7f965419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f964006e940 0x7f9640070e00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 --2- 172.21.15.31:0/419514558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9654101840 0x7f965419cd10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:45.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.881+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 >> 172.21.15.31:0/419514558 conn(0x7f96540fc740 msgr2=0x7f96540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:45.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.882+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 shutdown_connections 2024-04-22T23:48:45.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:45.882+0000 7f9659285700 1 -- 172.21.15.31:0/419514558 wait complete. 2024-04-22T23:48:45.884 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:48:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:46 smithi031 ceph-mon[26721]: pgmap v86: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:46 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/419514558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:46 smithi044 ceph-mon[34463]: pgmap v86: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:46 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/419514558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:46.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-22T23:48:46.734 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:48:48.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.154+0000 7f3ec3795700 1 -- 172.21.15.31:0/2063831191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc101850 msgr2=0x7f3ebc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.154+0000 7f3ec3795700 1 --2- 172.21.15.31:0/2063831191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc101850 0x7f3ebc1058a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009910 tx=0x7f3eb8009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.154+0000 7f3ec3795700 1 -- 172.21.15.31:0/2063831191 shutdown_connections 2024-04-22T23:48:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.154+0000 7f3ec3795700 1 --2- 172.21.15.31:0/2063831191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc101850 0x7f3ebc1058a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.154+0000 7f3ec3795700 1 --2- 172.21.15.31:0/2063831191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.154+0000 7f3ec3795700 1 -- 172.21.15.31:0/2063831191 >> 172.21.15.31:0/2063831191 conn(0x7f3ebc0fc750 msgr2=0x7f3ebc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.155+0000 7f3ec3795700 1 -- 172.21.15.31:0/2063831191 shutdown_connections 2024-04-22T23:48:48.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:48 smithi031 ceph-mon[26721]: pgmap v87: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.155+0000 7f3ec3795700 1 -- 172.21.15.31:0/2063831191 wait complete. 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec3795700 1 Processor -- start 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec3795700 1 -- start start 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec3795700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec3795700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ebc101850 0x7f3ebc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec3795700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc19d2d0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec3795700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc196780 con 0x7f3ebc101850 2024-04-22T23:48:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec1531700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec1531700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41786/0 (socket says 172.21.15.31:41786) 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec1531700 1 -- 172.21.15.31:0/1688919845 learned_addr learned my addr 172.21.15.31:0/1688919845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec0d30700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ebc101850 0x7f3ebc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec1531700 1 -- 172.21.15.31:0/1688919845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ebc101850 msgr2=0x7f3ebc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec1531700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ebc101850 0x7f3ebc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.156+0000 7f3ec1531700 1 -- 172.21.15.31:0/1688919845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80095f0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.157+0000 7f3ec1531700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc19c700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c8a0 tx=0x7f3eac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.157+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eac0078e0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.157+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3eac00f450 con 0x7f3ebc100ea0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.157+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc196a60 con 0x7f3ebc100ea0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.157+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eac00e5e0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.157+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc196fb0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.158+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc1092a0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.158+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f3eac00f5c0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.159+0000 7f3eb27fc700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f3ea806e940 0x7f3ea8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:48.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.159+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3eac08e7f0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.159+0000 7f3ec0d30700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f3ea806e940 0x7f3ea8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:48.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.159+0000 7f3ec0d30700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f3ea806e940 0x7f3ea8070e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc197bf0 tx=0x7f3eb801a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:48.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.164+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f3eac05b590 con 0x7f3ebc100ea0 2024-04-22T23:48:48.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.465+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ebc04f2e0 con 0x7f3ebc100ea0 2024-04-22T23:48:48.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.465+0000 7f3eb27fc700 1 -- 172.21.15.31:0/1688919845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f3eac05b120 con 0x7f3ebc100ea0 2024-04-22T23:48:48.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f3ea806e940 msgr2=0x7f3ea8070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:48.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f3ea806e940 0x7f3ea8070e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc197bf0 tx=0x7f3eb801a040 comp rx=0 tx=0).stop 2024-04-22T23:48:48.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 msgr2=0x7f3ebc19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc19c700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c8a0 tx=0x7f3eac00cbb0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 shutdown_connections 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f3ea806e940 0x7f3ea8070e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ebc100ea0 0x7f3ebc19c700 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 --2- 172.21.15.31:0/1688919845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ebc101850 0x7f3ebc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 >> 172.21.15.31:0/1688919845 conn(0x7f3ebc0fc750 msgr2=0x7f3ebc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 shutdown_connections 2024-04-22T23:48:48.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:48.468+0000 7f3ec3795700 1 -- 172.21.15.31:0/1688919845 wait complete. 2024-04-22T23:48:48.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:48 smithi044 ceph-mon[34463]: pgmap v87: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)" 2024-04-22T23:48:49.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-22T23:48:49.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:49 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/1688919845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:49.257 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:48:49.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:49 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/1688919845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:50.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:50 smithi031 ceph-mon[26721]: pgmap v88: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:50.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:50 smithi044 ceph-mon[34463]: pgmap v88: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2094020316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 msgr2=0x7f5fb8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:50.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2094020316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb8101270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009910 tx=0x7f5fa8009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2094020316 shutdown_connections 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2094020316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb8101840 0x7f5fb8105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2094020316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb8101270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2094020316 >> 172.21.15.31:0/2094020316 conn(0x7f5fb80fc740 msgr2=0x7f5fb80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2094020316 shutdown_connections 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2094020316 wait complete. 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.871+0000 7f5fbdbe8700 1 Processor -- start 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fbdbe8700 1 -- start start 2024-04-22T23:48:50.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fbdbe8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fbdbe8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb8101840 0x7f5fb819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fbdbe8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb819d270 con 0x7f5fb8100e90 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fbdbe8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb8196720 con 0x7f5fb8101840 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48592/0 (socket says 172.21.15.31:48592) 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb77fe700 1 -- 172.21.15.31:0/2668177025 learned_addr learned my addr 172.21.15.31:0/2668177025 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb6ffd700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb8101840 0x7f5fb819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb77fe700 1 -- 172.21.15.31:0/2668177025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb8101840 msgr2=0x7f5fb819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb77fe700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb8101840 0x7f5fb819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.872+0000 7f5fb77fe700 1 -- 172.21.15.31:0/2668177025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80095f0 con 0x7f5fb8100e90 2024-04-22T23:48:50.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.873+0000 7f5fb77fe700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb819c6a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006010 tx=0x7f5fa800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:50.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.873+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa801d070 con 0x7f5fb8100e90 2024-04-22T23:48:50.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.873+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa800f460 con 0x7f5fb8100e90 2024-04-22T23:48:50.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.873+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb81969a0 con 0x7f5fb8100e90 2024-04-22T23:48:50.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.873+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa8017630 con 0x7f5fb8100e90 2024-04-22T23:48:50.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.873+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb8196e90 con 0x7f5fb8100e90 2024-04-22T23:48:50.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.874+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb8109230 con 0x7f5fb8100e90 2024-04-22T23:48:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.875+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f5fa800f5d0 con 0x7f5fb8100e90 2024-04-22T23:48:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.875+0000 7f5fb4ff9700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f5fa006e940 0x7f5fa0070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.875+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5fa808fd40 con 0x7f5fb8100e90 2024-04-22T23:48:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.875+0000 7f5fb6ffd700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f5fa006e940 0x7f5fa0070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:50.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.876+0000 7f5fb6ffd700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f5fa006e940 0x7f5fa0070e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005950 tx=0x7f5fac0058e0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:50.880+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f5fa805cb80 con 0x7f5fb8100e90 2024-04-22T23:48:51.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.190+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5fb804f2e0 con 0x7f5fb8100e90 2024-04-22T23:48:51.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.190+0000 7f5fb4ff9700 1 -- 172.21.15.31:0/2668177025 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f5fa805c710 con 0x7f5fb8100e90 2024-04-22T23:48:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.192+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f5fa006e940 msgr2=0x7f5fa0070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.192+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f5fa006e940 0x7f5fa0070e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005950 tx=0x7f5fac0058e0 comp rx=0 tx=0).stop 2024-04-22T23:48:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.192+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 msgr2=0x7f5fb819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:51.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.192+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb819c6a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8006010 tx=0x7f5fa800be20 comp rx=0 tx=0).stop 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 shutdown_connections 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f5fa006e940 0x7f5fa0070e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fb8100e90 0x7f5fb819c6a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 --2- 172.21.15.31:0/2668177025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fb8101840 0x7f5fb819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 >> 172.21.15.31:0/2668177025 conn(0x7f5fb80fc740 msgr2=0x7f5fb80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 shutdown_connections 2024-04-22T23:48:51.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:51.193+0000 7f5fbdbe8700 1 -- 172.21.15.31:0/2668177025 wait complete. 2024-04-22T23:48:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:48:51.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-22T23:48:52.080 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:48:52.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:52 smithi031 ceph-mon[26721]: pgmap v89: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:52.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:52 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2668177025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:52.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:52 smithi044 ceph-mon[34463]: pgmap v89: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:52.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:52 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2668177025' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 -- 172.21.15.31:0/511279591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 msgr2=0x7f5270101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 --2- 172.21.15.31:0/511279591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f5270101ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5260009910 tx=0x7f5260009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 -- 172.21.15.31:0/511279591 shutdown_connections 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 --2- 172.21.15.31:0/511279591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 0x7f527010a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 --2- 172.21.15.31:0/511279591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f5270101ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 -- 172.21.15.31:0/511279591 >> 172.21.15.31:0/511279591 conn(0x7f52700fb370 msgr2=0x7f52700fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 -- 172.21.15.31:0/511279591 shutdown_connections 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.742+0000 7f526f59e700 1 -- 172.21.15.31:0/511279591 wait complete. 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526f59e700 1 Processor -- start 2024-04-22T23:48:53.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526f59e700 1 -- start start 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526f59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f52700ff760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526f59e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 0x7f52700ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526f59e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52701001e0 con 0x7f5270101b00 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526f59e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5270100320 con 0x7f5270102420 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f52700ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f52700ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48614/0 (socket says 172.21.15.31:48614) 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526e59c700 1 -- 172.21.15.31:0/716603463 learned_addr learned my addr 172.21.15.31:0/716603463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526dd9b700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 0x7f52700ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526e59c700 1 -- 172.21.15.31:0/716603463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 msgr2=0x7f52700ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526e59c700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 0x7f52700ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526e59c700 1 -- 172.21.15.31:0/716603463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600095f0 con 0x7f5270101b00 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.743+0000 7f526dd9b700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 0x7f52700ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.744+0000 7f526e59c700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f52700ff760 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5260006010 tx=0x7f526000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.744+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f526001d070 con 0x7f5270101b00 2024-04-22T23:48:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.744+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5260004fb0 con 0x7f5270101b00 2024-04-22T23:48:53.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.744+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52701005a0 con 0x7f5270101b00 2024-04-22T23:48:53.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.744+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5260022ea0 con 0x7f5270101b00 2024-04-22T23:48:53.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.744+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52701a2bb0 con 0x7f5270101b00 2024-04-22T23:48:53.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.745+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5270108010 con 0x7f5270101b00 2024-04-22T23:48:53.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.745+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f5260022470 con 0x7f5270101b00 2024-04-22T23:48:53.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.746+0000 7f525f7fe700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f525806e940 0x7f5258070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:53.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.746+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f526008fe60 con 0x7f5270101b00 2024-04-22T23:48:53.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.746+0000 7f526dd9b700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f525806e940 0x7f5258070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:53.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.746+0000 7f526dd9b700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f525806e940 0x7f5258070e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5270100f50 tx=0x7f5264008040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:53.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.751+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f52600591a0 con 0x7f5270101b00 2024-04-22T23:48:53.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:53.978+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}) v1 -- 0x7f5270061270 con 0x7f525806e940 2024-04-22T23:48:54.282 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:54 smithi031 ceph-mon[26721]: pgmap v90: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:54.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:54 smithi044 ceph-mon[34463]: pgmap v90: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:55.283 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:55 smithi031 ceph-mon[26721]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:55.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:55 smithi044 ceph-mon[34463]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:48:56.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.149+0000 7f525f7fe700 1 -- 172.21.15.31:0/716603463 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f5270061270 con 0x7f525806e940 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f525806e940 msgr2=0x7f5258070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f525806e940 0x7f5258070e00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5270100f50 tx=0x7f5264008040 comp rx=0 tx=0).stop 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 msgr2=0x7f52700ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f52700ff760 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5260006010 tx=0x7f526000be20 comp rx=0 tx=0).stop 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 shutdown_connections 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f525806e940 0x7f5258070e00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:56.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5270101b00 0x7f52700ff760 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 --2- 172.21.15.31:0/716603463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5270102420 0x7f52700ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.151+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 >> 172.21.15.31:0/716603463 conn(0x7f52700fb370 msgr2=0x7f5270105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.152+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 shutdown_connections 2024-04-22T23:48:56.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:56.152+0000 7f526f59e700 1 -- 172.21.15.31:0/716603463 wait complete. 2024-04-22T23:48:56.154 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:48:56.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:56 smithi031 ceph-mon[26721]: pgmap v91: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:56.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:56 smithi044 ceph-mon[34463]: pgmap v91: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:48:56.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch ps' 2024-04-22T23:48:56.992 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:48:58.281 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:58 smithi031 ceph-mon[26721]: pgmap v92: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:58.359 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:58 smithi044 ceph-mon[34463]: pgmap v92: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:48:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 -- 172.21.15.31:0/149422460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8101850 msgr2=0x7fc5e81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 --2- 172.21.15.31:0/149422460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8101850 0x7fc5e81058a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009910 tx=0x7fc5d4009c20 comp rx=0 tx=0).stop 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 -- 172.21.15.31:0/149422460 shutdown_connections 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 --2- 172.21.15.31:0/149422460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8101850 0x7fc5e81058a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 --2- 172.21.15.31:0/149422460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e8100ea0 0x7fc5e8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 -- 172.21.15.31:0/149422460 >> 172.21.15.31:0/149422460 conn(0x7fc5e80fc750 msgr2=0x7fc5e80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 -- 172.21.15.31:0/149422460 shutdown_connections 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 -- 172.21.15.31:0/149422460 wait complete. 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 Processor -- start 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.503+0000 7fc5eebee700 1 -- start start 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5eebee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 0x7fc5e819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5eebee700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e8101850 0x7fc5e819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5eebee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e819d2d0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5eebee700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e8196780 con 0x7fc5e8101850 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 0x7fc5e819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 0x7fc5e819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48638/0 (socket says 172.21.15.31:48638) 2024-04-22T23:48:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 -- 172.21.15.31:0/3491593200 learned_addr learned my addr 172.21.15.31:0/3491593200 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:48:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5e7fff700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e8101850 0x7fc5e819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 -- 172.21.15.31:0/3491593200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e8101850 msgr2=0x7fc5e819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e8101850 0x7fc5e819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 -- 172.21.15.31:0/3491593200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40095f0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5ec98a700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 0x7fc5e819c700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00d8d0 tx=0x7fc5dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5dc0098a0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5dc010460 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e8196a60 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.504+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5dc00f5f0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.505+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e8196fb0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.506+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e81092a0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.506+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fc5dc009a00 con 0x7fc5e8100ea0 2024-04-22T23:48:58.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.507+0000 7fc5e5ffb700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc5d806e940 0x7fc5d8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:48:58.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.507+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc5dc08e7c0 con 0x7fc5e8100ea0 2024-04-22T23:48:58.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.507+0000 7fc5e7fff700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc5d806e940 0x7fc5d8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:48:58.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.507+0000 7fc5e7fff700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc5d806e940 0x7fc5d8070e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8197bf0 tx=0x7fc5d401a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:48:58.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.512+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fc5dc05b580 con 0x7fc5e8100ea0 2024-04-22T23:48:58.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.748+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5e8197da0 con 0x7fc5d806e940 2024-04-22T23:48:58.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.759+0000 7fc5e5ffb700 1 -- 172.21.15.31:0/3491593200 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fc5e8197da0 con 0x7fc5d806e940 2024-04-22T23:48:58.751 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:48:58.751 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (71s) 40s ago 11m 16.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:48:58.751 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (12m) 40s ago 12m 7906k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:48:58.751 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (6m) 107s ago 11m 7692k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (12m) 40s ago 11m 6665k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (11m) 107s ago 10m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (46s) 40s ago 11m 52.9M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (6m) 40s ago 6m 14.7M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (6m) 40s ago 6m 92.8M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (6m) 107s ago 6m 240M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (6m) 107s ago 6m 18.1M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (3m) 40s ago 13m 510M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (2m) 107s ago 10m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (13m) 40s ago 13m 57.5M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (10m) 107s ago 10m 43.2M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (2m) 40s ago 11m 8669k - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (111s) 107s ago 10m 5440k - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (10m) 40s ago 10m 197M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:48:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (9m) 40s ago 9m 205M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:48:58.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (9m) 40s ago 9m 174M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:48:58.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (8m) 107s ago 8m 213M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:48:58.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (8m) 107s ago 8m 186M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:48:58.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (7m) 107s ago 7m 175M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:48:58.753 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (91s) 40s ago 11m 55.7M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.761+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc5d806e940 msgr2=0x7fc5d8070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.761+0000 7fc5eebee700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc5d806e940 0x7fc5d8070e00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8197bf0 tx=0x7fc5d401a040 comp rx=0 tx=0).stop 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.761+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 msgr2=0x7fc5e819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.761+0000 7fc5eebee700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 0x7fc5e819c700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00d8d0 tx=0x7fc5dc00dbe0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 shutdown_connections 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc5d806e940 0x7fc5d8070e00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e8100ea0 0x7fc5e819c700 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 --2- 172.21.15.31:0/3491593200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5e8101850 0x7fc5e819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 >> 172.21.15.31:0/3491593200 conn(0x7fc5e80fc750 msgr2=0x7fc5e80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 shutdown_connections 2024-04-22T23:48:58.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:48:58.762+0000 7fc5eebee700 1 -- 172.21.15.31:0/3491593200 wait complete. 2024-04-22T23:48:59.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:48:59 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:59.418 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:48:59.419 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:48:59.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-22T23:48:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:48:59 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:48:59.641 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:49:00.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:00 smithi031 ceph-mon[26721]: pgmap v93: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:00.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:00 smithi031 ceph-mon[26721]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:00.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:00 smithi044 ceph-mon[34463]: pgmap v93: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:00.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:00 smithi044 ceph-mon[34463]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 -- 172.21.15.31:0/1444613351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 msgr2=0x7fcce0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/1444613351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce0105fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009910 tx=0x7fccdc009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 -- 172.21.15.31:0/1444613351 shutdown_connections 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/1444613351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce00ffc20 0x7fcce0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/1444613351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce0105fb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 -- 172.21.15.31:0/1444613351 >> 172.21.15.31:0/1444613351 conn(0x7fcce0075670 msgr2=0x7fcce0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 -- 172.21.15.31:0/1444613351 shutdown_connections 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.205+0000 7fcce7d2d700 1 -- 172.21.15.31:0/1444613351 wait complete. 2024-04-22T23:49:01.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce7d2d700 1 Processor -- start 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce7d2d700 1 -- start start 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce7d2d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce00ffc20 0x7fcce019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce7d2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce7d2d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce01955e0 con 0x7fcce0105bd0 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce7d2d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce0195750 con 0x7fcce00ffc20 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce52c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.206+0000 7fcce52c8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54728/0 (socket says 172.21.15.31:54728) 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce52c8700 1 -- 172.21.15.31:0/2615512854 learned_addr learned my addr 172.21.15.31:0/2615512854 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce5ac9700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce00ffc20 0x7fcce019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce52c8700 1 -- 172.21.15.31:0/2615512854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce00ffc20 msgr2=0x7fcce019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:01.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce52c8700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce00ffc20 0x7fcce019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce52c8700 1 -- 172.21.15.31:0/2615512854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc0095f0 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce52c8700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce019bac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fccd000c8a0 tx=0x7fccd000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccd00078e0 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccd000f450 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce0195a30 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccd000e5e0 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.207+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce0195f80 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.208+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce01080d0 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.209+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fccd000f5c0 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.209+0000 7fccd6ffd700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fcccc06e940 0x7fcccc070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.209+0000 7fcce5ac9700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fcccc06e940 0x7fcccc070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.209+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fccd008e910 con 0x7fcce0105bd0 2024-04-22T23:49:01.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.210+0000 7fcce5ac9700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fcccc06e940 0x7fcccc070e00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fccdc006010 tx=0x7fccdc01a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:01.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.214+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fccd0057ba0 con 0x7fcce0105bd0 2024-04-22T23:49:01.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.435+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fcce0196b80 con 0x7fcce0105bd0 2024-04-22T23:49:01.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.440+0000 7fccd6ffd700 1 -- 172.21.15.31:0/2615512854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7fccd005b260 con 0x7fcce0105bd0 2024-04-22T23:49:01.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fcccc06e940 msgr2=0x7fcccc070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:01.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fcccc06e940 0x7fcccc070e00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fccdc006010 tx=0x7fccdc01a040 comp rx=0 tx=0).stop 2024-04-22T23:49:01.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 msgr2=0x7fcce019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:01.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce019bac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fccd000c8a0 tx=0x7fccd000cbb0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 shutdown_connections 2024-04-22T23:49:01.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce00ffc20 0x7fcce019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fcccc06e940 0x7fcccc070e00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 --2- 172.21.15.31:0/2615512854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcce0105bd0 0x7fcce019bac0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 >> 172.21.15.31:0/2615512854 conn(0x7fcce0075670 msgr2=0x7fcce00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.442+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 shutdown_connections 2024-04-22T23:49:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:01.443+0000 7fcce7d2d700 1 -- 172.21.15.31:0/2615512854 wait complete. 2024-04-22T23:49:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:02 smithi031 ceph-mon[26721]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:02 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2615512854' entity='client.admin' 2024-04-22T23:49:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:02 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:02 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:02 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:02 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:02.082 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:49:02.082 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:49:02.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-22T23:49:02.303 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:49:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:02 smithi044 ceph-mon[34463]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:02 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2615512854' entity='client.admin' 2024-04-22T23:49:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:02 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:02 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:02 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:02.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:02 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.802+0000 7f11d13d9700 1 -- 172.21.15.31:0/1684296643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 msgr2=0x7f11cc102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:03.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.802+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1684296643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc102240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009910 tx=0x7f11bc009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.802+0000 7f11d13d9700 1 -- 172.21.15.31:0/1684296643 shutdown_connections 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.802+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1684296643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc102780 0x7f11cc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.802+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1684296643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc102240 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.802+0000 7f11d13d9700 1 -- 172.21.15.31:0/1684296643 >> 172.21.15.31:0/1684296643 conn(0x7f11cc0fb650 msgr2=0x7f11cc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 -- 172.21.15.31:0/1684296643 shutdown_connections 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 -- 172.21.15.31:0/1684296643 wait complete. 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 Processor -- start 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 -- start start 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc102780 0x7f11cc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc19d220 con 0x7f11cc101e60 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.803+0000 7f11d13d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc196720 con 0x7f11cc102780 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:03.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54748/0 (socket says 172.21.15.31:54748) 2024-04-22T23:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 -- 172.21.15.31:0/1316968004 learned_addr learned my addr 172.21.15.31:0/1316968004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11ca7fc700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc102780 0x7f11cc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 -- 172.21.15.31:0/1316968004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc102780 msgr2=0x7f11cc19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc102780 0x7f11cc19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 -- 172.21.15.31:0/1316968004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc0095f0 con 0x7f11cc101e60 2024-04-22T23:49:03.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11caffd700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc19c650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11bc006010 tx=0x7f11bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc01d070 con 0x7f11cc101e60 2024-04-22T23:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11bc00f460 con 0x7f11cc101e60 2024-04-22T23:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc1969a0 con 0x7f11cc101e60 2024-04-22T23:49:03.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc017630 con 0x7f11cc101e60 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.804+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc196e90 con 0x7f11cc101e60 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.805+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc108310 con 0x7f11cc101e60 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.806+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f11bc00f5d0 con 0x7f11cc101e60 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.806+0000 7f11b3fff700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f11b406e940 0x7f11b4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.806+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f11bc08fd20 con 0x7f11cc101e60 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.807+0000 7f11ca7fc700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f11b406e940 0x7f11b4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:03.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.807+0000 7f11ca7fc700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f11b406e940 0x7f11b4070e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11c0005950 tx=0x7f11c00058e0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:03.811+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f11bc05cb60 con 0x7f11cc101e60 2024-04-22T23:49:04.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.017+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f11cc197ce0 con 0x7f11cc101e60 2024-04-22T23:49:04.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.017+0000 7f11b3fff700 1 -- 172.21.15.31:0/1316968004 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7f11cc197ce0 con 0x7f11cc101e60 2024-04-22T23:49:04.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f11b406e940 msgr2=0x7f11b4070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:04.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f11b406e940 0x7f11b4070e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f11c0005950 tx=0x7f11c00058e0 comp rx=0 tx=0).stop 2024-04-22T23:49:04.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 msgr2=0x7f11cc19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:04.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc19c650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11bc006010 tx=0x7f11bc00be20 comp rx=0 tx=0).stop 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 shutdown_connections 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f11b406e940 0x7f11b4070e00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11cc101e60 0x7f11cc19c650 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.019+0000 7f11d13d9700 1 --2- 172.21.15.31:0/1316968004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11cc102780 0x7f11cc19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.020+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 >> 172.21.15.31:0/1316968004 conn(0x7f11cc0fb650 msgr2=0x7f11cc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.020+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 shutdown_connections 2024-04-22T23:49:04.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:04.020+0000 7f11d13d9700 1 -- 172.21.15.31:0/1316968004 wait complete. 2024-04-22T23:49:04.295 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:04 smithi031 ceph-mon[26721]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:04 smithi044 ceph-mon[34463]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:04.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-22T23:49:04.797 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:49:06.091 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:06 smithi031 ceph-mon[26721]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 -- 172.21.15.31:0/4202136476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 msgr2=0x7f42dc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 --2- 172.21.15.31:0/4202136476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc101270 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f42d400b0a0 tx=0x7f42d400b3b0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 -- 172.21.15.31:0/4202136476 shutdown_connections 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 --2- 172.21.15.31:0/4202136476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42dc101840 0x7f42dc105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 --2- 172.21.15.31:0/4202136476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc101270 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 -- 172.21.15.31:0/4202136476 >> 172.21.15.31:0/4202136476 conn(0x7f42dc0fc740 msgr2=0x7f42dc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:06.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 -- 172.21.15.31:0/4202136476 shutdown_connections 2024-04-22T23:49:06.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.361+0000 7f42e2087700 1 -- 172.21.15.31:0/4202136476 wait complete. 2024-04-22T23:49:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42e2087700 1 Processor -- start 2024-04-22T23:49:06.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42e2087700 1 -- start start 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42e2087700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42e2087700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42dc101840 0x7f42dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42e2087700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc073b10 con 0x7f42dc100e90 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42e2087700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc073c50 con 0x7f42dc101840 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42db7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54768/0 (socket says 172.21.15.31:54768) 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42db7fe700 1 -- 172.21.15.31:0/2760139758 learned_addr learned my addr 172.21.15.31:0/2760139758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.362+0000 7f42daffd700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42dc101840 0x7f42dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42db7fe700 1 -- 172.21.15.31:0/2760139758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42dc101840 msgr2=0x7f42dc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:06.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42db7fe700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42dc101840 0x7f42dc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42db7fe700 1 -- 172.21.15.31:0/2760139758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d4009d00 con 0x7f42dc100e90 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42db7fe700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc073090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000f80 tx=0x7f42d400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d400e040 con 0x7f42dc100e90 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc073ed0 con 0x7f42dc100e90 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42d4003de0 con 0x7f42dc100e90 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d401d7d0 con 0x7f42dc100e90 2024-04-22T23:49:06.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.363+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1a2d90 con 0x7f42dc100e90 2024-04-22T23:49:06.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.364+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc109230 con 0x7f42dc100e90 2024-04-22T23:49:06.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.365+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f42d4004040 con 0x7f42dc100e90 2024-04-22T23:49:06.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.365+0000 7f42d8ff9700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f42c806e940 0x7f42c8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:06.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.365+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f42d40902f0 con 0x7f42dc100e90 2024-04-22T23:49:06.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.365+0000 7f42daffd700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f42c806e940 0x7f42c8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:06.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.366+0000 7f42daffd700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f42c806e940 0x7f42c8070e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42dc074880 tx=0x7f42c400a300 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:06.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.370+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f42d40594e0 con 0x7f42dc100e90 2024-04-22T23:49:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:06 smithi044 ceph-mon[34463]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:06.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.577+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f42dc04f2e0 con 0x7f42dc100e90 2024-04-22T23:49:06.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.578+0000 7f42d8ff9700 1 -- 172.21.15.31:0/2760139758 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7f42d405cba0 con 0x7f42dc100e90 2024-04-22T23:49:06.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f42c806e940 msgr2=0x7f42c8070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:06.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f42c806e940 0x7f42c8070e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42dc074880 tx=0x7f42c400a300 comp rx=0 tx=0).stop 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 msgr2=0x7f42dc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc073090 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000f80 tx=0x7f42d400cd20 comp rx=0 tx=0).stop 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 shutdown_connections 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f42c806e940 0x7f42c8070e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42dc100e90 0x7f42dc073090 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 --2- 172.21.15.31:0/2760139758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42dc101840 0x7f42dc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 >> 172.21.15.31:0/2760139758 conn(0x7f42dc0fc740 msgr2=0x7f42dc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 shutdown_connections 2024-04-22T23:49:06.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:06.580+0000 7f42e2087700 1 -- 172.21.15.31:0/2760139758 wait complete. 2024-04-22T23:49:07.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-22T23:49:07.423 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:49:08.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:08 smithi031 ceph-mon[26721]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:08.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:08 smithi044 ceph-mon[34463]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:09.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 -- 172.21.15.31:0/3162803358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 msgr2=0x7f6474105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 --2- 172.21.15.31:0/3162803358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f6474105d30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6464009910 tx=0x7f6464009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 -- 172.21.15.31:0/3162803358 shutdown_connections 2024-04-22T23:49:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 --2- 172.21.15.31:0/3162803358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 0x7f64740ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 --2- 172.21.15.31:0/3162803358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f6474105d30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 -- 172.21.15.31:0/3162803358 >> 172.21.15.31:0/3162803358 conn(0x7f6474075460 msgr2=0x7f6474075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:09.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 -- 172.21.15.31:0/3162803358 shutdown_connections 2024-04-22T23:49:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.114+0000 7f6478b9f700 1 -- 172.21.15.31:0/3162803358 wait complete. 2024-04-22T23:49:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.115+0000 7f6478b9f700 1 Processor -- start 2024-04-22T23:49:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.115+0000 7f6478b9f700 1 -- start start 2024-04-22T23:49:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.115+0000 7f6478b9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 0x7f6474197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.115+0000 7f6478b9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f64741979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:09.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.115+0000 7f6478b9f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474197ff0 con 0x7f6474105950 2024-04-22T23:49:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.115+0000 7f6478b9f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474198130 con 0x7f64740ffa20 2024-04-22T23:49:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f64741979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f64741979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54784/0 (socket says 172.21.15.31:54784) 2024-04-22T23:49:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 -- 172.21.15.31:0/1825172938 learned_addr learned my addr 172.21.15.31:0/1825172938 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f64737fe700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 0x7f6474197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:09.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 -- 172.21.15.31:0/1825172938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 msgr2=0x7f6474197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 0x7f6474197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 -- 172.21.15.31:0/1825172938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64640095f0 con 0x7f6474105950 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f64737fe700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 0x7f6474197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6472ffd700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f64741979d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f646800d8d0 tx=0x7f646800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6468009940 con 0x7f6474105950 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6468010460 con 0x7f6474105950 2024-04-22T23:49:09.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64741a0500 con 0x7f6474105950 2024-04-22T23:49:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f646800f5f0 con 0x7f6474105950 2024-04-22T23:49:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.116+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64741a09c0 con 0x7f6474105950 2024-04-22T23:49:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.117+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6474107e70 con 0x7f6474105950 2024-04-22T23:49:09.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.118+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f6468009aa0 con 0x7f6474105950 2024-04-22T23:49:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.118+0000 7f6470ff9700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f645c06e940 0x7f645c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:09.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.118+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f646808e8e0 con 0x7f6474105950 2024-04-22T23:49:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.118+0000 7f64737fe700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f645c06e940 0x7f645c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.119+0000 7f64737fe700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f645c06e940 0x7f645c070e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f646400b5c0 tx=0x7f64640053a0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:09.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.123+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f6468057b70 con 0x7f6474105950 2024-04-22T23:49:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.331+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f64740623c0 con 0x7f6474105950 2024-04-22T23:49:09.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.331+0000 7f6470ff9700 1 -- 172.21.15.31:0/1825172938 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f646805b230 con 0x7f6474105950 2024-04-22T23:49:09.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f645c06e940 msgr2=0x7f645c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f645c06e940 0x7f645c070e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f646400b5c0 tx=0x7f64640053a0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 msgr2=0x7f64741979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f64741979d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f646800d8d0 tx=0x7f646800dc90 comp rx=0 tx=0).stop 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 shutdown_connections 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64740ffa20 0x7f6474197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f645c06e940 0x7f645c070e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 --2- 172.21.15.31:0/1825172938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6474105950 0x7f64741979d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 >> 172.21.15.31:0/1825172938 conn(0x7f6474075460 msgr2=0x7f64740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 shutdown_connections 2024-04-22T23:49:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:09.334+0000 7f6478b9f700 1 -- 172.21.15.31:0/1825172938 wait complete. 2024-04-22T23:49:09.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-22T23:49:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:49:10.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:10 smithi031 ceph-mon[26721]: pgmap v98: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:10 smithi044 ceph-mon[34463]: pgmap v98: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.783+0000 7ff564a73700 1 -- 172.21.15.31:0/208309119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560101850 msgr2=0x7ff5601058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.783+0000 7ff564a73700 1 --2- 172.21.15.31:0/208309119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560101850 0x7ff5601058a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff554009910 tx=0x7ff554009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.784+0000 7ff564a73700 1 -- 172.21.15.31:0/208309119 shutdown_connections 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.784+0000 7ff564a73700 1 --2- 172.21.15.31:0/208309119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560101850 0x7ff5601058a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.784+0000 7ff564a73700 1 --2- 172.21.15.31:0/208309119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560100ea0 0x7ff560101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.784+0000 7ff564a73700 1 -- 172.21.15.31:0/208309119 >> 172.21.15.31:0/208309119 conn(0x7ff5600fc750 msgr2=0x7ff5600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.784+0000 7ff564a73700 1 -- 172.21.15.31:0/208309119 shutdown_connections 2024-04-22T23:49:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.784+0000 7ff564a73700 1 -- 172.21.15.31:0/208309119 wait complete. 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff564a73700 1 Processor -- start 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff564a73700 1 -- start start 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff564a73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 0x7ff56019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff564a73700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560101850 0x7ff56019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff564a73700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019d360 con 0x7ff560100ea0 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff564a73700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff560196810 con 0x7ff560101850 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 0x7ff56019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 0x7ff56019c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32894/0 (socket says 172.21.15.31:32894) 2024-04-22T23:49:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55e59c700 1 -- 172.21.15.31:0/2281674918 learned_addr learned my addr 172.21.15.31:0/2281674918 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55dd9b700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560101850 0x7ff56019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55e59c700 1 -- 172.21.15.31:0/2281674918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560101850 msgr2=0x7ff56019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55e59c700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560101850 0x7ff56019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.785+0000 7ff55e59c700 1 -- 172.21.15.31:0/2281674918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540095f0 con 0x7ff560100ea0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.786+0000 7ff55e59c700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 0x7ff56019c790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b4b0 tx=0x7ff55000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.786+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff550011800 con 0x7ff560100ea0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.786+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff560196af0 con 0x7ff560100ea0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.786+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff550011e20 con 0x7ff560100ea0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.786+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff55000f2e0 con 0x7ff560100ea0 2024-04-22T23:49:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.786+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff560197010 con 0x7ff560100ea0 2024-04-22T23:49:11.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.787+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5601091d0 con 0x7ff560100ea0 2024-04-22T23:49:11.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.787+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7ff550010380 con 0x7ff560100ea0 2024-04-22T23:49:11.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.788+0000 7ff54f7fe700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff54806e940 0x7ff548070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:11.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.788+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff55008e410 con 0x7ff560100ea0 2024-04-22T23:49:11.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.788+0000 7ff55dd9b700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff54806e940 0x7ff548070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:11.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.789+0000 7ff55dd9b700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff54806e940 0x7ff548070e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff554006010 tx=0x7ff55401a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:11.793+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7ff55005b1d0 con 0x7ff560100ea0 2024-04-22T23:49:12.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.023+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff560197290 con 0x7ff54806e940 2024-04-22T23:49:12.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.029+0000 7ff54f7fe700 1 -- 172.21.15.31:0/2281674918 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7ff560197290 con 0x7ff54806e940 2024-04-22T23:49:12.020 INFO:teuthology.orchestra.run.smithi031.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:49:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff54806e940 msgr2=0x7ff548070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff54806e940 0x7ff548070e00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff554006010 tx=0x7ff55401a040 comp rx=0 tx=0).stop 2024-04-22T23:49:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 msgr2=0x7ff56019c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 0x7ff56019c790 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b4b0 tx=0x7ff55000b7c0 comp rx=0 tx=0).stop 2024-04-22T23:49:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 shutdown_connections 2024-04-22T23:49:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7ff54806e940 0x7ff548070e00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff560100ea0 0x7ff56019c790 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 --2- 172.21.15.31:0/2281674918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff560101850 0x7ff56019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 >> 172.21.15.31:0/2281674918 conn(0x7ff5600fc750 msgr2=0x7ff5600fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 shutdown_connections 2024-04-22T23:49:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:12.031+0000 7ff564a73700 1 -- 172.21.15.31:0/2281674918 wait complete. 2024-04-22T23:49:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:12 smithi031 ceph-mon[26721]: pgmap v99: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:12.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:12 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:12 smithi044 ceph-mon[34463]: pgmap v99: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:12 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:12.752 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:49:12.753 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:49:12.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-22T23:49:12.979 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:49:13.359 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:13 smithi031 ceph-mon[26721]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:13.359 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:13 smithi031 ceph-mon[26721]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:49:13.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:13 smithi044 ceph-mon[34463]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:13.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:13 smithi044 ceph-mon[34463]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:49:14.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:14 smithi031 ceph-mon[26721]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:49:14.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:14 smithi031 ceph-mon[26721]: pgmap v100: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:14.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:14 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:49:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:14 smithi044 ceph-mon[34463]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:49:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:14 smithi044 ceph-mon[34463]: pgmap v100: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:14.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:14 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 -- 172.21.15.31:0/2275089938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0068580 msgr2=0x7fd5b00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/2275089938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0068580 0x7fd5b00689e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009910 tx=0x7fd5a0009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 -- 172.21.15.31:0/2275089938 shutdown_connections 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/2275089938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0068580 0x7fd5b00689e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/2275089938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0106cd0 0x7fd5b01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 -- 172.21.15.31:0/2275089938 >> 172.21.15.31:0/2275089938 conn(0x7fd5b0075670 msgr2=0x7fd5b0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 -- 172.21.15.31:0/2275089938 shutdown_connections 2024-04-22T23:49:15.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.209+0000 7fd5b79ba700 1 -- 172.21.15.31:0/2275089938 wait complete. 2024-04-22T23:49:15.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b79ba700 1 Processor -- start 2024-04-22T23:49:15.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b79ba700 1 -- start start 2024-04-22T23:49:15.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b79ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0068580 0x7fd5b019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b79ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 0x7fd5b019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b79ba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b019f4f0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b79ba700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b01989a0 con 0x7fd5b0068580 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b4f55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 0x7fd5b019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b4f55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 0x7fd5b019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32924/0 (socket says 172.21.15.31:32924) 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b4f55700 1 -- 172.21.15.31:0/244721930 learned_addr learned my addr 172.21.15.31:0/244721930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b5756700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0068580 0x7fd5b019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b4f55700 1 -- 172.21.15.31:0/244721930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0068580 msgr2=0x7fd5b019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b4f55700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0068580 0x7fd5b019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b4f55700 1 -- 172.21.15.31:0/244721930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00095f0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.210+0000 7fd5b5756700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0068580 0x7fd5b019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.211+0000 7fd5b4f55700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 0x7fd5b019ee60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006010 tx=0x7fd5a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.211+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a001d070 con 0x7fd5b0106cd0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.211+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a0004fb0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.211+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b0198c20 con 0x7fd5b0106cd0 2024-04-22T23:49:15.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.211+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a0022ea0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.211+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b0199110 con 0x7fd5b0106cd0 2024-04-22T23:49:15.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.212+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b004f2e0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.213+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fd5a0022470 con 0x7fd5b0106cd0 2024-04-22T23:49:15.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.213+0000 7fd5a67fc700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd59c06e940 0x7fd59c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.213+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd5a008fde0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.213+0000 7fd5b5756700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd59c06e940 0x7fd59c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.214+0000 7fd5b5756700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd59c06e940 0x7fd59c070e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac007900 tx=0x7fd5ac008040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:15.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.218+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fd5a005ccd0 con 0x7fd5b0106cd0 2024-04-22T23:49:15.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.445+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5b0199440 con 0x7fd59c06e940 2024-04-22T23:49:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.447+0000 7fd5a67fc700 1 -- 172.21.15.31:0/244721930 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd5b0199440 con 0x7fd59c06e940 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd59c06e940 msgr2=0x7fd59c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd59c06e940 0x7fd59c070e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac007900 tx=0x7fd5ac008040 comp rx=0 tx=0).stop 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 msgr2=0x7fd5b019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 0x7fd5b019ee60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006010 tx=0x7fd5a000be20 comp rx=0 tx=0).stop 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 shutdown_connections 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5b0068580 0x7fd5b019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd59c06e940 0x7fd59c070e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 --2- 172.21.15.31:0/244721930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5b0106cd0 0x7fd5b019ee60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 >> 172.21.15.31:0/244721930 conn(0x7fd5b0075670 msgr2=0x7fd5b00feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 shutdown_connections 2024-04-22T23:49:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.449+0000 7fd5b79ba700 1 -- 172.21.15.31:0/244721930 wait complete. 2024-04-22T23:49:15.451 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:49:15.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.561+0000 7f59b76a9700 1 -- 172.21.15.31:0/2884505108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 msgr2=0x7f59b01021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.561+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2884505108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b01021e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f59a000b0a0 tx=0x7f59a000b3b0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.561+0000 7f59b76a9700 1 -- 172.21.15.31:0/2884505108 shutdown_connections 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.561+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2884505108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59b0102720 0x7f59b010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.561+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2884505108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b01021e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.561+0000 7f59b76a9700 1 -- 172.21.15.31:0/2884505108 >> 172.21.15.31:0/2884505108 conn(0x7f59b00fb650 msgr2=0x7f59b00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 -- 172.21.15.31:0/2884505108 shutdown_connections 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 -- 172.21.15.31:0/2884505108 wait complete. 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 Processor -- start 2024-04-22T23:49:15.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 -- start start 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b00ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59b0102720 0x7f59b00fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b01004e0 con 0x7f59b0101e00 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.562+0000 7f59b76a9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0100620 con 0x7f59b0102720 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b00ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b00ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32950/0 (socket says 172.21.15.31:32950) 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 -- 172.21.15.31:0/2504341791 learned_addr learned my addr 172.21.15.31:0/2504341791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b4c44700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59b0102720 0x7f59b00fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 -- 172.21.15.31:0/2504341791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59b0102720 msgr2=0x7f59b00fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59b0102720 0x7f59b00fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 -- 172.21.15.31:0/2504341791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a0009d00 con 0x7f59b0101e00 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b5445700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b00ffa60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000f80 tx=0x7f59a000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59a000e040 con 0x7f59b0101e00 2024-04-22T23:49:15.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b01008a0 con 0x7f59b0101e00 2024-04-22T23:49:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59a0003de0 con 0x7f59b0101e00 2024-04-22T23:49:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b01a2dc0 con 0x7f59b0101e00 2024-04-22T23:49:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.563+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59a001d730 con 0x7f59b0101e00 2024-04-22T23:49:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.564+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b0108310 con 0x7f59b0101e00 2024-04-22T23:49:15.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.565+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f59a0004040 con 0x7f59b0101e00 2024-04-22T23:49:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.565+0000 7f59ae7fc700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f599806e940 0x7f5998070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.565+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f59a00900e0 con 0x7f59b0101e00 2024-04-22T23:49:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.565+0000 7f59b4c44700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f599806e940 0x7f5998070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.566+0000 7f59b4c44700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f599806e940 0x7f5998070e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005950 tx=0x7f59a40058e0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.570+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f59a0059370 con 0x7f59b0101e00 2024-04-22T23:49:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.800+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59b00611d0 con 0x7f599806e940 2024-04-22T23:49:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.801+0000 7f59ae7fc700 1 -- 172.21.15.31:0/2504341791 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f59b00611d0 con 0x7f599806e940 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f599806e940 msgr2=0x7f5998070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f599806e940 0x7f5998070e00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005950 tx=0x7f59a40058e0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 msgr2=0x7f59b00ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b00ffa60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000f80 tx=0x7f59a000cdd0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 shutdown_connections 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f599806e940 0x7f5998070e00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b0101e00 0x7f59b00ffa60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 --2- 172.21.15.31:0/2504341791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59b0102720 0x7f59b00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 >> 172.21.15.31:0/2504341791 conn(0x7f59b00fb650 msgr2=0x7f59b0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.803+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 shutdown_connections 2024-04-22T23:49:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.804+0000 7f59b76a9700 1 -- 172.21.15.31:0/2504341791 wait complete. 2024-04-22T23:49:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 -- 172.21.15.31:0/3828812915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374106cd0 msgr2=0x7fd3741070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 --2- 172.21.15.31:0/3828812915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374106cd0 0x7fd3741070b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd368009910 tx=0x7fd368009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 -- 172.21.15.31:0/3828812915 shutdown_connections 2024-04-22T23:49:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 --2- 172.21.15.31:0/3828812915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374068580 0x7fd3740689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 --2- 172.21.15.31:0/3828812915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374106cd0 0x7fd3741070b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 -- 172.21.15.31:0/3828812915 >> 172.21.15.31:0/3828812915 conn(0x7fd374075670 msgr2=0x7fd374075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 -- 172.21.15.31:0/3828812915 shutdown_connections 2024-04-22T23:49:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.913+0000 7fd37aad0700 1 -- 172.21.15.31:0/3828812915 wait complete. 2024-04-22T23:49:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37aad0700 1 Processor -- start 2024-04-22T23:49:15.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37aad0700 1 -- start start 2024-04-22T23:49:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37aad0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 0x7fd37419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37aad0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374106cd0 0x7fd37419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37aad0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37419d230 con 0x7fd374068580 2024-04-22T23:49:15.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37aad0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741966e0 con 0x7fd374106cd0 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37886c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 0x7fd37419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37886c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 0x7fd37419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32970/0 (socket says 172.21.15.31:32970) 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37886c700 1 -- 172.21.15.31:0/2043160674 learned_addr learned my addr 172.21.15.31:0/2043160674 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd373fff700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374106cd0 0x7fd37419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37886c700 1 -- 172.21.15.31:0/2043160674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374106cd0 msgr2=0x7fd37419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37886c700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374106cd0 0x7fd37419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd37886c700 1 -- 172.21.15.31:0/2043160674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680095f0 con 0x7fd374068580 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.914+0000 7fd373fff700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374106cd0 0x7fd37419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.915+0000 7fd37886c700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 0x7fd37419c660 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd368006010 tx=0x7fd36800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.915+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36801d070 con 0x7fd374068580 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.915+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3680050d0 con 0x7fd374068580 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.915+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd374196960 con 0x7fd374068580 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.915+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd368003ea0 con 0x7fd374068580 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.915+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd374196e50 con 0x7fd374068580 2024-04-22T23:49:15.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.916+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3741091f0 con 0x7fd374068580 2024-04-22T23:49:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.917+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fd368003680 con 0x7fd374068580 2024-04-22T23:49:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.917+0000 7fd371ffb700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd36406e940 0x7fd364070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.917+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd36808fe50 con 0x7fd374068580 2024-04-22T23:49:15.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.917+0000 7fd373fff700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd36406e940 0x7fd364070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:15.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.918+0000 7fd373fff700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd36406e940 0x7fd364070e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd374197b50 tx=0x7fd360009260 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:15.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:15.922+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fd36805cc10 con 0x7fd374068580 2024-04-22T23:49:16.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.152+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd374197070 con 0x7fd36406e940 2024-04-22T23:49:16.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.163+0000 7fd371ffb700 1 -- 172.21.15.31:0/2043160674 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd374197070 con 0x7fd36406e940 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (88s) 57s ago 12m 16.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (12m) 57s ago 12m 7906k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (6m) 2m ago 11m 7692k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (12m) 57s ago 12m 6665k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (11m) 2m ago 11m 6665k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (63s) 57s ago 11m 52.9M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (6m) 57s ago 6m 14.7M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (6m) 57s ago 6m 92.8M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (6m) 2m ago 6m 240M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (6m) 2m ago 6m 18.1M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:49:16.156 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (3m) 57s ago 13m 510M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (2m) 2m ago 11m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (13m) 57s ago 13m 57.5M 2048M 18.2.1 5be31c24972a acf3e9104186 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (11m) 2m ago 11m 43.2M 2048M 18.2.1 5be31c24972a 0f950ab28ebe 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (2m) 57s ago 12m 8669k - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (2m) 2m ago 11m 5440k - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (10m) 57s ago 10m 197M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (9m) 57s ago 9m 205M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (9m) 57s ago 9m 174M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (9m) 2m ago 9m 213M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (8m) 2m ago 8m 186M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (8m) 2m ago 8m 175M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:49:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (109s) 57s ago 11m 55.7M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd36406e940 msgr2=0x7fd364070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd36406e940 0x7fd364070e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd374197b50 tx=0x7fd360009260 comp rx=0 tx=0).stop 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 msgr2=0x7fd37419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 0x7fd37419c660 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd368006010 tx=0x7fd36800f740 comp rx=0 tx=0).stop 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 shutdown_connections 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fd36406e940 0x7fd364070e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd374068580 0x7fd37419c660 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 --2- 172.21.15.31:0/2043160674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd374106cd0 0x7fd37419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.165+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 >> 172.21.15.31:0/2043160674 conn(0x7fd374075670 msgr2=0x7fd3740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:16.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.166+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 shutdown_connections 2024-04-22T23:49:16.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.166+0000 7fd37aad0700 1 -- 172.21.15.31:0/2043160674 wait complete. 2024-04-22T23:49:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 -- 172.21.15.31:0/195712025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 msgr2=0x7fdd68100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/195712025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68100080 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009910 tx=0x7fdd5c009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 -- 172.21.15.31:0/195712025 shutdown_connections 2024-04-22T23:49:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/195712025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68100080 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/195712025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd68105bd0 0x7fdd68105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 -- 172.21.15.31:0/195712025 >> 172.21.15.31:0/195712025 conn(0x7fdd68075670 msgr2=0x7fdd68075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:16.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 -- 172.21.15.31:0/195712025 shutdown_connections 2024-04-22T23:49:16.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.274+0000 7fdd6dd24700 1 -- 172.21.15.31:0/195712025 wait complete. 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.275+0000 7fdd6dd24700 1 Processor -- start 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.275+0000 7fdd6dd24700 1 -- start start 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd6dd24700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd6dd24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd68105bd0 0x7fdd68197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd6dd24700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68198320 con 0x7fdd680ffc20 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd6dd24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68198460 con 0x7fdd68105bd0 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32984/0 (socket says 172.21.15.31:32984) 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 -- 172.21.15.31:0/2205791678 learned_addr learned my addr 172.21.15.31:0/2205791678 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd66ffd700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd68105bd0 0x7fdd68197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 -- 172.21.15.31:0/2205791678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd68105bd0 msgr2=0x7fdd68197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd68105bd0 0x7fdd68197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 -- 172.21.15.31:0/2205791678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0095f0 con 0x7fdd680ffc20 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.276+0000 7fdd677fe700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68197700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c8a0 tx=0x7fdd5800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.277+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd580078e0 con 0x7fdd680ffc20 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.277+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd5800f450 con 0x7fdd680ffc20 2024-04-22T23:49:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.277+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd681a07d0 con 0x7fdd680ffc20 2024-04-22T23:49:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.277+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd5800e5e0 con 0x7fdd680ffc20 2024-04-22T23:49:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.277+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd681a0cf0 con 0x7fdd680ffc20 2024-04-22T23:49:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.278+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd681080d0 con 0x7fdd680ffc20 2024-04-22T23:49:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.278+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fdd5800f5c0 con 0x7fdd680ffc20 2024-04-22T23:49:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.279+0000 7fdd64ff9700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fdd5006e940 0x7fdd50070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.279+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdd5808e870 con 0x7fdd680ffc20 2024-04-22T23:49:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.279+0000 7fdd66ffd700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fdd5006e940 0x7fdd50070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.280+0000 7fdd66ffd700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fdd5006e940 0x7fdd50070e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd68069210 tx=0x7fdd5c01a040 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:16.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.283+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fdd58057c20 con 0x7fdd680ffc20 2024-04-22T23:49:16.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:16 smithi031 ceph-mon[26721]: pgmap v101: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:16.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:16 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:16.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:16 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:16.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:16 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:16.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:16 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:16.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:16 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:49:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:16 smithi044 ceph-mon[34463]: pgmap v101: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:16 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:16 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:16 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:16 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:16 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:49:16.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.605+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdd68107ab0 con 0x7fdd680ffc20 2024-04-22T23:49:16.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.605+0000 7fdd64ff9700 1 -- 172.21.15.31:0/2205791678 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fdd5805b2e0 con 0x7fdd680ffc20 2024-04-22T23:49:16.597 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:49:16.597 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:49:16.597 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-22T23:49:16.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:49:16.598 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fdd5006e940 msgr2=0x7fdd50070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fdd5006e940 0x7fdd50070e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd68069210 tx=0x7fdd5c01a040 comp rx=0 tx=0).stop 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 msgr2=0x7fdd68197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68197700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c8a0 tx=0x7fdd5800cbb0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 shutdown_connections 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fdd5006e940 0x7fdd50070e00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd680ffc20 0x7fdd68197700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 --2- 172.21.15.31:0/2205791678 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd68105bd0 0x7fdd68197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 >> 172.21.15.31:0/2205791678 conn(0x7fdd68075670 msgr2=0x7fdd680fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 shutdown_connections 2024-04-22T23:49:16.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.608+0000 7fdd6dd24700 1 -- 172.21.15.31:0/2205791678 wait complete. 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 -- 172.21.15.31:0/2940658714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764102700 msgr2=0x7f076410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 --2- 172.21.15.31:0/2940658714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764102700 0x7f076410abf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0758009910 tx=0x7f0758009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 -- 172.21.15.31:0/2940658714 shutdown_connections 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 --2- 172.21.15.31:0/2940658714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764102700 0x7f076410abf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 --2- 172.21.15.31:0/2940658714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764101de0 0x7f07641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 -- 172.21.15.31:0/2940658714 >> 172.21.15.31:0/2940658714 conn(0x7f07640fb610 msgr2=0x7f07640fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 -- 172.21.15.31:0/2940658714 shutdown_connections 2024-04-22T23:49:16.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.718+0000 7f0768b89700 1 -- 172.21.15.31:0/2940658714 wait complete. 2024-04-22T23:49:16.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f0768b89700 1 Processor -- start 2024-04-22T23:49:16.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f0768b89700 1 -- start start 2024-04-22T23:49:16.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f0768b89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 0x7f076419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:16.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f0768b89700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764102700 0x7f076419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:16.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f0768b89700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076419d220 con 0x7f0764101de0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f0768b89700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764196720 con 0x7f0764102700 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f076259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 0x7f076419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f076259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 0x7f076419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32996/0 (socket says 172.21.15.31:32996) 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.719+0000 7f076259c700 1 -- 172.21.15.31:0/3320850299 learned_addr learned my addr 172.21.15.31:0/3320850299 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f0761d9b700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764102700 0x7f076419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f076259c700 1 -- 172.21.15.31:0/3320850299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764102700 msgr2=0x7f076419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f076259c700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764102700 0x7f076419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f076259c700 1 -- 172.21.15.31:0/3320850299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580095f0 con 0x7f0764101de0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f0761d9b700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764102700 0x7f076419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f076259c700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 0x7f076419c650 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f075400d8d0 tx=0x7f075400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0754009940 con 0x7f0764101de0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0764196a00 con 0x7f0764101de0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0754010460 con 0x7f0764101de0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f075400f5f0 con 0x7f0764101de0 2024-04-22T23:49:16.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.720+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0764196f50 con 0x7f0764101de0 2024-04-22T23:49:16.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.721+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07641082f0 con 0x7f0764101de0 2024-04-22T23:49:16.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.722+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f07540105d0 con 0x7f0764101de0 2024-04-22T23:49:16.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.722+0000 7f07537fe700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f074c06e940 0x7f074c070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:16.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.722+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f075408e810 con 0x7f0764101de0 2024-04-22T23:49:16.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.722+0000 7f0761d9b700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f074c06e940 0x7f074c070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:16.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.723+0000 7f0761d9b700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f074c06e940 0x7f074c070e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f075800b5c0 tx=0x7f07580053a0 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.727+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f075405b650 con 0x7f0764101de0 2024-04-22T23:49:16.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.986+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0764197c10 con 0x7f0764101de0 2024-04-22T23:49:16.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.986+0000 7f07537fe700 1 -- 172.21.15.31:0/3320850299 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1849 (secure 0 0 0) 0x7f075405b1e0 con 0x7f0764101de0 2024-04-22T23:49:16.978 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:49:16.978 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:49:16.978 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:16.978 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:49:16.979 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:49:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:16.981 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f074c06e940 msgr2=0x7f074c070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f074c06e940 0x7f074c070e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f075800b5c0 tx=0x7f07580053a0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 msgr2=0x7f076419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 0x7f076419c650 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f075400d8d0 tx=0x7f075400dc90 comp rx=0 tx=0).stop 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 shutdown_connections 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f074c06e940 0x7f074c070e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0764101de0 0x7f076419c650 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 --2- 172.21.15.31:0/3320850299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0764102700 0x7f076419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:16.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.988+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 >> 172.21.15.31:0/3320850299 conn(0x7f07640fb610 msgr2=0x7f0764105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:16.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.989+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 shutdown_connections 2024-04-22T23:49:16.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:16.989+0000 7f0768b89700 1 -- 172.21.15.31:0/3320850299 wait complete. 2024-04-22T23:49:16.983 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.099+0000 7fc7e151a700 1 -- 172.21.15.31:0/2075859115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 msgr2=0x7fc7dc10d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.099+0000 7fc7e151a700 1 --2- 172.21.15.31:0/2075859115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc10d000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009910 tx=0x7fc7cc009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.099+0000 7fc7e151a700 1 -- 172.21.15.31:0/2075859115 shutdown_connections 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.099+0000 7fc7e151a700 1 --2- 172.21.15.31:0/2075859115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc10d000 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.099+0000 7fc7e151a700 1 --2- 172.21.15.31:0/2075859115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7dc0ff990 0x7fc7dc0ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.099+0000 7fc7e151a700 1 -- 172.21.15.31:0/2075859115 >> 172.21.15.31:0/2075859115 conn(0x7fc7dc0fb650 msgr2=0x7fc7dc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.100+0000 7fc7e151a700 1 -- 172.21.15.31:0/2075859115 shutdown_connections 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.100+0000 7fc7e151a700 1 -- 172.21.15.31:0/2075859115 wait complete. 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.100+0000 7fc7e151a700 1 Processor -- start 2024-04-22T23:49:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.100+0000 7fc7e151a700 1 -- start start 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7e151a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7dc0ff990 0x7fc7dc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7e151a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7e151a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc19f430 con 0x7fc7dc100340 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7e151a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc1988e0 con 0x7fc7dc0ff990 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33022/0 (socket says 172.21.15.31:33022) 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 -- 172.21.15.31:0/234962604 learned_addr learned my addr 172.21.15.31:0/234962604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7daffd700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7dc0ff990 0x7fc7dc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 -- 172.21.15.31:0/234962604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7dc0ff990 msgr2=0x7fc7dc19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7dc0ff990 0x7fc7dc19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 -- 172.21.15.31:0/234962604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0095f0 con 0x7fc7dc100340 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7da7fc700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc19eda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0092e0 tx=0x7fc7cc0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7cc01d070 con 0x7fc7dc100340 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7cc005dc0 con 0x7fc7dc100340 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.101+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc198b60 con 0x7fc7dc100340 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.102+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7cc021410 con 0x7fc7dc100340 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.102+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc1990b0 con 0x7fc7dc100340 2024-04-22T23:49:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.103+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc10a770 con 0x7fc7dc100340 2024-04-22T23:49:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.103+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7fc7cc00f460 con 0x7fc7dc100340 2024-04-22T23:49:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.104+0000 7fc7c3fff700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7c406e940 0x7fc7c4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.104+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc7cc08fb50 con 0x7fc7dc100340 2024-04-22T23:49:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.104+0000 7fc7daffd700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7c406e940 0x7fc7c4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.104+0000 7fc7daffd700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7c406e940 0x7fc7c4070e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005950 tx=0x7fc7d000b410 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:17.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.108+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7fc7cc05c8f0 con 0x7fc7dc100340 2024-04-22T23:49:17.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.344+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 --> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7dc199e40 con 0x7fc7c406e940 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: Upgrade: Target is version 19.0.0-3252-g43be0201 (squid) 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c'] 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: Upgrade: Setting container_image for all mgr 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:17.337 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2205791678' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:17.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:17 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/3320850299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.346+0000 7fc7c3fff700 1 -- 172.21.15.31:0/234962604 <== mgr.14652 v2:172.21.15.31:6800/418479598 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc7dc199e40 con 0x7fc7c406e940 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:49:17.338 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-22T23:49:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:49:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/23 daemons upgraded", 2024-04-22T23:49:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-04-22T23:49:17.339 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:49:17.339 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:49:17.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7c406e940 msgr2=0x7fc7c4070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7c406e940 0x7fc7c4070e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005950 tx=0x7fc7d000b410 comp rx=0 tx=0).stop 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 msgr2=0x7fc7dc19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc19eda0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0092e0 tx=0x7fc7cc0055f0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 shutdown_connections 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7dc0ff990 0x7fc7dc19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7fc7c406e940 0x7fc7c4070e00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 --2- 172.21.15.31:0/234962604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7dc100340 0x7fc7dc19eda0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 >> 172.21.15.31:0/234962604 conn(0x7fc7dc0fb650 msgr2=0x7fc7dc0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 shutdown_connections 2024-04-22T23:49:17.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.348+0000 7fc7e151a700 1 -- 172.21.15.31:0/234962604 wait complete. 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- 172.21.15.31:0/1268501907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 msgr2=0x7f45980ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 --2- 172.21.15.31:0/1268501907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f45980ff7c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4584009910 tx=0x7f4584009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- 172.21.15.31:0/1268501907 shutdown_connections 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 --2- 172.21.15.31:0/1268501907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f45980ff7c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 --2- 172.21.15.31:0/1268501907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4598101bb0 0x7f4598101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- 172.21.15.31:0/1268501907 >> 172.21.15.31:0/1268501907 conn(0x7f4598074e30 msgr2=0x7f4598075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- 172.21.15.31:0/1268501907 shutdown_connections 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- 172.21.15.31:0/1268501907 wait complete. 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 Processor -- start 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- start start 2024-04-22T23:49:17.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f459819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4598101bb0 0x7f459819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598198960 con 0x7f45980ff340 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.457+0000 7f459f151700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598198ad0 con 0x7f4598101bb0 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f459819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f459819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33048/0 (socket says 172.21.15.31:33048) 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 -- 172.21.15.31:0/2884717321 learned_addr learned my addr 172.21.15.31:0/2884717321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 -- 172.21.15.31:0/2884717321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4598101bb0 msgr2=0x7f459819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4598101bb0 0x7f459819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 -- 172.21.15.31:0/2884717321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45840095f0 con 0x7f45980ff340 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459ceed700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f459819e970 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b4b0 tx=0x7f458c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458c011800 con 0x7f45980ff340 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f458c011e20 con 0x7f45980ff340 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4598198db0 con 0x7f45980ff340 2024-04-22T23:49:17.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.458+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458c00f2e0 con 0x7f45980ff340 2024-04-22T23:49:17.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.459+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4598199300 con 0x7f45980ff340 2024-04-22T23:49:17.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.459+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459810b230 con 0x7f45980ff340 2024-04-22T23:49:17.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.460+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92598+0+0 (secure 0 0 0) 0x7f458c010380 con 0x7f45980ff340 2024-04-22T23:49:17.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.460+0000 7f4595ffb700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f458806e940 0x7f4588070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:17.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.460+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f458c08e490 con 0x7f45980ff340 2024-04-22T23:49:17.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.461+0000 7f4597fff700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f458806e940 0x7f4588070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:17.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.461+0000 7f4597fff700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f458806e940 0x7f4588070e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f458400b5c0 tx=0x7f4584005960 comp rx=0 tx=0).ready entity=mgr.14652 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:17.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.465+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187909 (secure 0 0 0) 0x7f458c057720 con 0x7f45980ff340 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: Upgrade: Target is version 19.0.0-3252-g43be0201 (squid) 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c'] 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: Upgrade: Setting container_image for all mgr 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2205791678' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:17 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/3320850299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:49:17.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.771+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4598199f30 con 0x7f45980ff340 2024-04-22T23:49:17.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.771+0000 7f4595ffb700 1 -- 172.21.15.31:0/2884717321 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f458c05ade0 con 0x7f45980ff340 2024-04-22T23:49:17.763 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:17.763 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f458806e940 msgr2=0x7f4588070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f458806e940 0x7f4588070e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f458400b5c0 tx=0x7f4584005960 comp rx=0 tx=0).stop 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 msgr2=0x7f459819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f459819e970 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b4b0 tx=0x7f458c00b870 comp rx=0 tx=0).stop 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 shutdown_connections 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:6800/418479598,v1:172.21.15.31:6801/418479598] conn(0x7f458806e940 0x7f4588070e00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45980ff340 0x7f459819e970 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 --2- 172.21.15.31:0/2884717321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4598101bb0 0x7f459819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:17.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.773+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 >> 172.21.15.31:0/2884717321 conn(0x7f4598074e30 msgr2=0x7f45980fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:17.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.774+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 shutdown_connections 2024-04-22T23:49:17.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:17.774+0000 7f459f151700 1 -- 172.21.15.31:0/2884717321 wait complete. 2024-04-22T23:49:18.386 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:18 smithi031 ceph-mon[26721]: pgmap v102: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:18.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:18.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:49:18.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:49:18.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:18 smithi031 ceph-mon[26721]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:18.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:18 smithi031 ceph-mon[26721]: from='client.? 172.21.15.31:0/2884717321' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:49:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:18 smithi044 ceph-mon[34463]: pgmap v102: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:49:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:49:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:18 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:18.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:18 smithi044 ceph-mon[34463]: from='client.? 172.21.15.31:0/2884717321' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:49:19.397 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 ceph-mon[26721]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:19.398 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 ceph-mon[26721]: Upgrade: Updating mon.smithi031 2024-04-22T23:49:19.398 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 ceph-mon[26721]: Deploying daemon mon.smithi031 on smithi031 2024-04-22T23:49:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:19 smithi044 ceph-mon[34463]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:19 smithi044 ceph-mon[34463]: Upgrade: Updating mon.smithi031 2024-04-22T23:49:19.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:19 smithi044 ceph-mon[34463]: Deploying daemon mon.smithi031 on smithi031 2024-04-22T23:49:19.654 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:49:19.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[26717]: 2024-04-22T23:49:19.661+0000 7f963d59a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:49:19.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[26717]: 2024-04-22T23:49:19.661+0000 7f963d59a700 -1 mon.smithi031@0(leader) e2 *** Got Signal Terminated *** 2024-04-22T23:49:19.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:19 smithi031 podman[111113]: 2024-04-22 23:49:19.897049834 +0000 UTC m=+0.340682638 container died acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-22T23:49:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 podman[111113]: 2024-04-22 23:49:20.269890292 +0000 UTC m=+0.713523083 container cleanup acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T23:49:20.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 bash[111113]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031 2024-04-22T23:49:20.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 podman[111139]: 2024-04-22 23:49:20.4542702 +0000 UTC m=+0.550692768 container remove acf3e910418643152717f8b035e293b6ccdc00b2a6838b5d1104d81c24f79c9a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:49:21.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-04-22T23:49:21.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:49:21.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service: Consumed 14.995s CPU time. 2024-04-22T23:49:21.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:20 smithi031 systemd[1]: Starting Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:49:21.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 podman[111270]: 2024-04-22 23:49:21.155551028 +0000 UTC m=+0.113112597 container create 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:49:21.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 podman[111270]: 2024-04-22 23:49:21.061480161 +0000 UTC m=+0.019041708 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:49:21.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 podman[111270]: 2024-04-22 23:49:21.367037058 +0000 UTC m=+0.324598595 container init 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:49:21.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 podman[111270]: 2024-04-22 23:49:21.370314884 +0000 UTC m=+0.327876411 container start 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-22T23:49:21.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev), process ceph-mon, pid 2 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: pidfile_write: ignore empty --pid-file 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: load: jerasure load: lrc 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: RocksDB version: 7.9.2 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Git sha 0 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Compile date 2024-04-22 21:33:31 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: DB SUMMARY 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: DB Session ID: LJZECHCOJX0MDWWW2RLO 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi031/store.db dir, Total Num: 1, files: 000032.sst 2024-04-22T23:49:21.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi031/store.db: 000030.log size: 3424850 ; 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.env: 0x5561f6203cc0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.info_log: 0x5561f6cd3aa0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.statistics: (nil) 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.use_fsync: 0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:49:21.404 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.db_log_dir: 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.wal_dir: 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:49:21.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.write_buffer_manager: 0x5561f6c670e0 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:49:21.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:49:21.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:49:21.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:49:21.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:49:21.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:49:21.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.unordered_write: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.row_cache: None 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.wal_filter: None 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.wal_compression: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:49:21.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:49:21.409 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_open_files: -1 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Compression algorithms supported: 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:49:21.410 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kZSTD supported: 0 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000015 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.merge_operator: 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_filter: None 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:49:21.411 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5561f6cd3c80) 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_type: 0 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_index_type: 0 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_shortening: 1 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: checksum: 4 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: no_block_cache: 0 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache: 0x5561f6c991f0 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:49:21.412 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_options: 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: capacity : 536870912 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_shard_bits : 4 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: strict_capacity_limit : 0 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_compressed: (nil) 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: persistent_cache: (nil) 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size: 4096 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size_deviation: 10 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_restart_interval: 16 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_block_restart_interval: 1 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_block_size: 4096 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: partition_filters: 0 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: use_delta_encoding: 1 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: filter_policy: bloomfilter 2024-04-22T23:49:21.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: whole_key_filtering: 1 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: verify_compression: 0 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: format_version: 5 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_index_compression: 1 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_align: 0 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_auto_readahead_size: 262144 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: prepopulate_block_cache: 0 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: initial_auto_readahead_size: 8192 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression: NoCompression 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:49:21.414 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.num_levels: 7 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:49:21.415 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:49:21.416 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T23:49:21.417 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:49:21.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.ttl: 2592000 2024-04-22T23:49:21.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12133, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-22T23:49:21.420 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6880fd68-273d-40a1-b7b7-c9596e00fc4f 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829761400548, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829761411002, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2986554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12134, "largest_seqno": 15052, "table_properties": {"data_size": 2975337, "index_size": 6842, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3333, "raw_key_size": 29087, "raw_average_key_size": 22, "raw_value_size": 2949677, "raw_average_value_size": 2232, "num_data_blocks": 310, "num_entries": 1321, "num_filter_entries": 1321, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713829761, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6880fd68-273d-40a1-b7b7-c9596e00fc4f", "db_session_id": "LJZECHCOJX0MDWWW2RLO", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829761411129, "job": 1, "event": "recovery_finished"} 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-22T23:49:21.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:49:21.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi031/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:49:21.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5561f6dae000 2024-04-22T23:49:21.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: DB pointer 0x5561f6d98000 2024-04-22T23:49:21.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T23:49:21.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T23:49:21.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** DB Stats ** 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 1/0 2.85 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 625.0 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L6 1/0 5.98 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-22T23:49:21.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 8.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 625.0 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 625.0 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 625.0 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:49:21.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 186.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 186.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache BinnedLRUCache@0x5561f6c991f0#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: starting mon.smithi031 rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi031 fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???) e2 preinit fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).mds e18 new map 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).mds e18 print_map 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: e18 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:21.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: legacy client fscid: 1 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Filesystem 'cephfs' (1) 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: fs_name cephfs 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: epoch 18 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: tableserver 0 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: root 0 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: session_timeout 60 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: session_autoclose 300 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_file_size 1099511627776 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_xattr_size 65536 2024-04-22T23:49:21.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: required_client_features {} 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: last_failure 0 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: last_failure_osd_epoch 46 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_mds 1 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: in 0 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: up {0=24293} 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: failed 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: damaged 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: stopped 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_pools [3] 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_pool 2 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: inline_data enabled 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: balancer 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: bal_rank_mask -1 2024-04-22T23:49:21.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: standby_count_wanted 1 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: qdb_cluster leader: 0 members: 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Standby daemons: 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 ceph-mon[111289]: mon.smithi031@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-04-22T23:49:21.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 bash[111270]: 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 2024-04-22T23:49:21.701 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:21 smithi031 systemd[1]: Started Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: pgmap v104: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: mon.smithi031 calling monitor election 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: mon.smithi031 is new leader, mons smithi031,smithi044 in quorum (ranks 0,1) 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: monmap epoch 2 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:23.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: last_changed 2024-04-22T23:38:13.792510+0000 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: created 2024-04-22T23:35:31.547688+0000 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: min_mon_release 18 (reef) 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: election_strategy: 1 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: mgrmap e31: smithi031.fazdcv(active, since 3m), standbys: smithi044.iglcti 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: from='mgr.14652 ' entity='' 2024-04-22T23:49:23.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:22 smithi031 ceph-mon[111289]: mgrmap e32: smithi031.fazdcv(active, since 3m), standbys: smithi044.iglcti 2024-04-22T23:49:23.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:49:23.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: from='mgr.14652 172.21.15.31:0/2856480475' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: pgmap v104: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: mon.smithi031 calling monitor election 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: mon.smithi031 is new leader, mons smithi031,smithi044 in quorum (ranks 0,1) 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: monmap epoch 2 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: last_changed 2024-04-22T23:38:13.792510+0000 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: created 2024-04-22T23:35:31.547688+0000 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: min_mon_release 18 (reef) 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: election_strategy: 1 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: mgrmap e31: smithi031.fazdcv(active, since 3m), standbys: smithi044.iglcti 2024-04-22T23:49:23.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:23.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:23.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:23.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: from='mgr.14652 ' entity='' 2024-04-22T23:49:23.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:22 smithi044 ceph-mon[34463]: mgrmap e32: smithi031.fazdcv(active, since 3m), standbys: smithi044.iglcti 2024-04-22T23:49:30.137 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:30 smithi031 ceph-mon[111289]: Standby manager daemon smithi044.iglcti restarted 2024-04-22T23:49:30.137 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:30 smithi031 ceph-mon[111289]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:49:30.137 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:30 smithi031 ceph-mon[111289]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:49:30.137 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:30 smithi031 ceph-mon[111289]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:49:30.137 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:30 smithi031 ceph-mon[111289]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:49:30.137 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:30 smithi031 ceph-mon[111289]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:49:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:30 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti restarted 2024-04-22T23:49:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:30 smithi044 ceph-mon[34463]: Standby manager daemon smithi044.iglcti started 2024-04-22T23:49:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:30 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/crt"}]: dispatch 2024-04-22T23:49:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:30 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:49:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:30 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi044.iglcti/key"}]: dispatch 2024-04-22T23:49:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:30 smithi044 ceph-mon[34463]: from='mgr.? 172.21.15.44:0/1184349900' entity='mgr.smithi044.iglcti' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: mgrmap e33: smithi031.fazdcv(active, since 3m), standbys: smithi044.iglcti 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: Active manager daemon smithi031.fazdcv restarted 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: mgrmap e34: smithi031.fazdcv(active, starting, since 0.00509939s), standbys: smithi044.iglcti 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:49:31.439 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:49:31.440 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:49:31.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:31 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:49:31.442 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: mgrmap e33: smithi031.fazdcv(active, since 3m), standbys: smithi044.iglcti 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: Active manager daemon smithi031.fazdcv restarted 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: Activating manager daemon smithi031.fazdcv 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: mgrmap e34: smithi031.fazdcv(active, starting, since 0.00509939s), standbys: smithi044.iglcti 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.fazdcv", "id": "smithi031.fazdcv"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr metadata", "who": "smithi044.iglcti", "id": "smithi044.iglcti"}]: dispatch 2024-04-22T23:49:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: Manager daemon smithi031.fazdcv is now available 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:49:31.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.fazdcv/trash_purge_schedule"}]: dispatch 2024-04-22T23:49:32.389 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:32 smithi031 ceph-mon[111289]: mgrmap e35: smithi031.fazdcv(active, since 1.01138s), standbys: smithi044.iglcti 2024-04-22T23:49:32.390 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:32 smithi031 ceph-mon[111289]: pgmap v3: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:32 smithi044 ceph-mon[34463]: mgrmap e35: smithi031.fazdcv(active, since 1.01138s), standbys: smithi044.iglcti 2024-04-22T23:49:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:32 smithi044 ceph-mon[34463]: pgmap v3: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:33 smithi031 ceph-mon[111289]: pgmap v4: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:33 smithi031 ceph-mon[111289]: mgrmap e36: smithi031.fazdcv(active, since 2s), standbys: smithi044.iglcti 2024-04-22T23:49:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:33 smithi044 ceph-mon[34463]: pgmap v4: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:33 smithi044 ceph-mon[34463]: mgrmap e36: smithi031.fazdcv(active, since 2s), standbys: smithi044.iglcti 2024-04-22T23:49:34.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:34 smithi031 ceph-mon[111289]: [22/Apr/2024:23:49:33] ENGINE Bus STARTING 2024-04-22T23:49:34.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:34 smithi031 ceph-mon[111289]: [22/Apr/2024:23:49:33] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:49:34.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:34 smithi031 ceph-mon[111289]: [22/Apr/2024:23:49:33] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:49:34.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:34 smithi031 ceph-mon[111289]: [22/Apr/2024:23:49:33] ENGINE Bus STARTED 2024-04-22T23:49:34.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:34 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:34.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:34 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:34.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:34 smithi044 ceph-mon[34463]: [22/Apr/2024:23:49:33] ENGINE Bus STARTING 2024-04-22T23:49:34.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:34 smithi044 ceph-mon[34463]: [22/Apr/2024:23:49:33] ENGINE Serving on https://172.21.15.31:7150 2024-04-22T23:49:34.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:34 smithi044 ceph-mon[34463]: [22/Apr/2024:23:49:33] ENGINE Serving on http://172.21.15.31:8765 2024-04-22T23:49:34.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:34 smithi044 ceph-mon[34463]: [22/Apr/2024:23:49:33] ENGINE Bus STARTED 2024-04-22T23:49:34.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:34 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:34.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:34 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:35.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:35 smithi031 ceph-mon[111289]: pgmap v5: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:35.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:35 smithi031 ceph-mon[111289]: mgrmap e37: smithi031.fazdcv(active, since 4s), standbys: smithi044.iglcti 2024-04-22T23:49:35.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:35.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:35 smithi044 ceph-mon[34463]: pgmap v5: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:35 smithi044 ceph-mon[34463]: mgrmap e37: smithi031.fazdcv(active, since 4s), standbys: smithi044.iglcti 2024-04-22T23:49:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:35 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:35 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:37.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:36 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:37.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:36 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:36 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:37.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:36 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:37 smithi031 ceph-mon[111289]: pgmap v6: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:37.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:37 smithi044 ceph-mon[34463]: pgmap v6: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:49:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:37 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:37 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:40.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:39 smithi031 ceph-mon[111289]: pgmap v7: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-22T23:49:40.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:39 smithi044 ceph-mon[34463]: pgmap v7: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: pgmap v8: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: Detected new or changed devices on smithi031 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:49:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:49:41.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:41 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: pgmap v8: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: Detected new or changed devices on smithi031 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-22T23:49:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-22T23:49:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:49:41.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:41 smithi031 ceph-mon[111289]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:42.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:42.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:49:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:49:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:49:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: Updating smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:49:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: Updating smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:49:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:42.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:42.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:42.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:42 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:49:43.829 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:43 smithi044 ceph-mon[34463]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:49:43.829 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:43 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:43.829 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:43 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:49:43.829 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:43 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:49:43.829 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:43 smithi044 ceph-mon[34463]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:43 smithi031 ceph-mon[111289]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:49:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:49:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:49:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:44.844 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:44 smithi044 ceph-mon[34463]: Upgrade: Updating mon.smithi044 2024-04-22T23:49:44.844 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:44 smithi044 ceph-mon[34463]: Deploying daemon mon.smithi044 on smithi044 2024-04-22T23:49:44.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:44 smithi031 ceph-mon[111289]: Upgrade: Updating mon.smithi044 2024-04-22T23:49:44.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:44 smithi031 ceph-mon[111289]: Deploying daemon mon.smithi044 on smithi044 2024-04-22T23:49:45.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:45 smithi044 systemd[1]: Stopping Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:49:45.590 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:45 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044[34454]: 2024-04-22T23:49:45.281+0000 7f52fcde6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi044 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:49:45.591 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:45 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044[34454]: 2024-04-22T23:49:45.281+0000 7f52fcde6700 -1 mon.smithi044@1(peon) e2 *** Got Signal Terminated *** 2024-04-22T23:49:45.591 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:45 smithi044 podman[81982]: 2024-04-22 23:49:45.303010085 +0000 UTC m=+0.101276985 container died 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-22T23:49:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:45 smithi044 podman[81982]: 2024-04-22 23:49:45.783201436 +0000 UTC m=+0.581468342 container cleanup 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1) 2024-04-22T23:49:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:45 smithi044 bash[81982]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044 2024-04-22T23:49:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 podman[81996]: 2024-04-22 23:49:46.118068505 +0000 UTC m=+0.808746085 container remove 0f950ab28ebe9c4c7b1ee9bea922e0812263fdcf17ccc795516d4def34308495 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:49:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi044.service: Deactivated successfully. 2024-04-22T23:49:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 systemd[1]: Stopped Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:49:46.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi044.service: Consumed 8.407s CPU time. 2024-04-22T23:49:46.773 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 systemd[1]: Starting Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:49:46.773 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 podman[82127]: 2024-04-22 23:49:46.685553787 +0000 UTC m=+0.110161582 container create 7af6a8053287c6ebe12b0a0ef7308dab1216dc6bfa54637ad7531a007dc2bbfc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-22T23:49:46.773 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 podman[82127]: 2024-04-22 23:49:46.593697681 +0000 UTC m=+0.018305430 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 podman[82127]: 2024-04-22 23:49:46.904608559 +0000 UTC m=+0.329216309 container init 7af6a8053287c6ebe12b0a0ef7308dab1216dc6bfa54637ad7531a007dc2bbfc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 podman[82127]: 2024-04-22 23:49:46.907962302 +0000 UTC m=+0.332570052 container start 7af6a8053287c6ebe12b0a0ef7308dab1216dc6bfa54637ad7531a007dc2bbfc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev), process ceph-mon, pid 2 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: pidfile_write: ignore empty --pid-file 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: load: jerasure load: lrc 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: RocksDB version: 7.9.2 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Git sha 0 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Compile date 2024-04-22 21:33:31 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: DB SUMMARY 2024-04-22T23:49:47.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: DB Session ID: FKNGQX3GMMVS1F3Q5OAB 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi044/store.db dir, Total Num: 1, files: 000027.sst 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi044/store.db: 000025.log size: 8099559 ; 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.env: 0x5594a61f0cc0 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.info_log: 0x5594a6dcfaa0 2024-04-22T23:49:47.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:49:47.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.statistics: (nil) 2024-04-22T23:49:47.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.use_fsync: 0 2024-04-22T23:49:47.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:49:47.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.db_log_dir: 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.wal_dir: 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:49:47.214 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.write_buffer_manager: 0x5594a6d630e0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:49:47.215 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.unordered_write: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.row_cache: None 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.wal_filter: None 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.wal_compression: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:49:47.216 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:49:47.217 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_open_files: -1 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Compression algorithms supported: 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:49:47.218 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kZSTD supported: 0 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000010 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.merge_operator: 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_filter: None 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:49:47.219 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5594a6dcfc80) 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_type: 0 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_index_type: 0 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_shortening: 1 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: checksum: 4 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: no_block_cache: 0 2024-04-22T23:49:47.220 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache: 0x5594a6d951f0 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_options: 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: capacity : 536870912 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_shard_bits : 4 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: strict_capacity_limit : 0 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_cache_compressed: (nil) 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: persistent_cache: (nil) 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size: 4096 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_size_deviation: 10 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_restart_interval: 16 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: index_block_restart_interval: 1 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: metadata_block_size: 4096 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: partition_filters: 0 2024-04-22T23:49:47.221 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: use_delta_encoding: 1 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: filter_policy: bloomfilter 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: whole_key_filtering: 1 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: verify_compression: 0 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: format_version: 5 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_index_compression: 1 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: block_align: 0 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_auto_readahead_size: 262144 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: prepopulate_block_cache: 0 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: initial_auto_readahead_size: 8192 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:49:47.222 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression: NoCompression 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.num_levels: 7 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:49:47.223 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:49:47.224 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:49:47.225 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:49:47.225 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:49:47.225 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:49:47.225 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:49:47.225 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:49:47.226 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T23:49:47.227 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.ttl: 2592000 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T23:49:47.228 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi044/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12191, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 39138f31-451a-4300-beb3-f3ad6c38bf76 2024-04-22T23:49:47.229 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829786937897, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829786958173, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4843970, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12196, "largest_seqno": 15913, "table_properties": {"data_size": 4830741, "index_size": 8342, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3845, "raw_key_size": 33983, "raw_average_key_size": 22, "raw_value_size": 4800659, "raw_average_value_size": 3137, "num_data_blocks": 376, "num_entries": 1530, "num_filter_entries": 1530, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713829786, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "39138f31-451a-4300-beb3-f3ad6c38bf76", "db_session_id": "FKNGQX3GMMVS1F3Q5OAB", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829786958287, "job": 1, "event": "recovery_finished"} 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi044/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5594a6eaa000 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: DB pointer 0x5594a6e94000 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** DB Stats ** 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:49:47.230 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: L0 1/0 4.62 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 548.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: L6 1/0 5.98 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Sum 2/0 10.60 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 548.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 548.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-22T23:49:47.231 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** Compaction Stats [default] ** 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 548.7 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Cumulative compaction: 0.00 GB write, 200.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:49:47.232 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Interval compaction: 0.00 GB write, 200.77 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache BinnedLRUCache@0x5594a6d951f0#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: starting mon.smithi044 rank 1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi044 fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???) e2 preinit fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).mds e18 new map 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).mds e18 print_map 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: e18 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:49:47.233 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: legacy client fscid: 1 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Filesystem 'cephfs' (1) 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: fs_name cephfs 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: epoch 18 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: tableserver 0 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: root 0 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: session_timeout 60 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: session_autoclose 300 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_file_size 1099511627776 2024-04-22T23:49:47.234 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_xattr_size 65536 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: required_client_features {} 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: last_failure 0 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: last_failure_osd_epoch 46 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: max_mds 1 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: in 0 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: up {0=24293} 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: failed 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: damaged 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: stopped 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: data_pools [3] 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: metadata_pool 2 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: inline_data enabled 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: balancer 2024-04-22T23:49:47.235 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: bal_rank_mask -1 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: standby_count_wanted 1 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: qdb_cluster leader: 0 members: 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: Standby daemons: 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout: [mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:49:47.236 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 ceph-mon[82146]: mon.smithi044@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-22T23:49:47.237 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 bash[82127]: 7af6a8053287c6ebe12b0a0ef7308dab1216dc6bfa54637ad7531a007dc2bbfc 2024-04-22T23:49:47.237 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:46 smithi044 systemd[1]: Started Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:49:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.897+0000 7f85ab24f700 1 -- 172.21.15.31:0/4123701230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 msgr2=0x7f85a410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.897+0000 7f85ab24f700 1 --2- 172.21.15.31:0/4123701230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a410ac10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8598009910 tx=0x7f8598009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- 172.21.15.31:0/4123701230 shutdown_connections 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 --2- 172.21.15.31:0/4123701230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a410ac10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 --2- 172.21.15.31:0/4123701230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- 172.21.15.31:0/4123701230 >> 172.21.15.31:0/4123701230 conn(0x7f85a40fb650 msgr2=0x7f85a40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- 172.21.15.31:0/4123701230 shutdown_connections 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- 172.21.15.31:0/4123701230 wait complete. 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 Processor -- start 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- start start 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a419d250 con 0x7f85a4102720 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.898+0000 7f85ab24f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4196700 con 0x7f85a4101e00 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51026/0 (socket says 172.21.15.31:51026) 2024-04-22T23:49:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 -- 172.21.15.31:0/946008776 learned_addr learned my addr 172.21.15.31:0/946008776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a8feb700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a8feb700 1 -- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 msgr2=0x7f85a419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a8feb700 1 -- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 msgr2=0x7f85a419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a8feb700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a8feb700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a419c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 -- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 msgr2=0x7f85a419c680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a419c680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 -- 172.21.15.31:0/946008776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85980095f0 con 0x7f85a4102720 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a3fff700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a419cbc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f859800bd70 tx=0x7f859800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f859801d070 con 0x7f85a4102720 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a4196980 con 0x7f85a4102720 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.899+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8598005dc0 con 0x7f85a4102720 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.900+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8598017400 con 0x7f85a4102720 2024-04-22T23:49:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.900+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a4196e70 con 0x7f85a4102720 2024-04-22T23:49:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.901+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a4108310 con 0x7f85a4102720 2024-04-22T23:49:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.901+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f859800f7b0 con 0x7f85a4102720 2024-04-22T23:49:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.902+0000 7f85a1ffb700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f858c06ee00 0x7f858c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.902+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8598090560 con 0x7f85a4102720 2024-04-22T23:49:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.902+0000 7f85a8feb700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f858c06ee00 0x7f858c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.903+0000 7f85a8feb700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f858c06ee00 0x7f858c0712c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8594005950 tx=0x7f85940058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:47.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:47.907+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f859805cc80 con 0x7f85a4102720 2024-04-22T23:49:48.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.142+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85a4197090 con 0x7f858c06ee00 2024-04-22T23:49:48.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.143+0000 7f85a1ffb700 1 -- 172.21.15.31:0/946008776 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f85a4197090 con 0x7f858c06ee00 2024-04-22T23:49:48.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f858c06ee00 msgr2=0x7f858c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f858c06ee00 0x7f858c0712c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8594005950 tx=0x7f85940058e0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 msgr2=0x7f85a419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a419cbc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f859800bd70 tx=0x7f859800be50 comp rx=0 tx=0).stop 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 shutdown_connections 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f858c06ee00 0x7f858c0712c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85a4101e00 0x7f85a419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 --2- 172.21.15.31:0/946008776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a4102720 0x7f85a419cbc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.145+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 >> 172.21.15.31:0/946008776 conn(0x7f85a40fb650 msgr2=0x7f85a4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.146+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 shutdown_connections 2024-04-22T23:49:48.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.146+0000 7f85ab24f700 1 -- 172.21.15.31:0/946008776 wait complete. 2024-04-22T23:49:48.147 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:49:48.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.252+0000 7f5c84b5b700 1 -- 172.21.15.31:0/1647247852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80100ea0 msgr2=0x7f5c80101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.252+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/1647247852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80100ea0 0x7f5c80101280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009910 tx=0x7f5c70009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:48.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 -- 172.21.15.31:0/1647247852 shutdown_connections 2024-04-22T23:49:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/1647247852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80101850 0x7f5c801058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/1647247852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80100ea0 0x7f5c80101280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 -- 172.21.15.31:0/1647247852 >> 172.21.15.31:0/1647247852 conn(0x7f5c800fc750 msgr2=0x7f5c800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 -- 172.21.15.31:0/1647247852 shutdown_connections 2024-04-22T23:49:48.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 -- 172.21.15.31:0/1647247852 wait complete. 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 Processor -- start 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.253+0000 7f5c84b5b700 1 -- start start 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c84b5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 0x7f5c80073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c84b5b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 0x7f5c800735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c84b5b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80073b10 con 0x7f5c80101850 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c84b5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80073c50 con 0x7f5c80100ea0 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 0x7f5c800735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 0x7f5c800735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51044/0 (socket says 172.21.15.31:51044) 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 -- 172.21.15.31:0/2459058861 learned_addr learned my addr 172.21.15.31:0/2459058861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:48.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7e59c700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 0x7f5c80073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7e59c700 1 -- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 msgr2=0x7f5c80073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7e59c700 1 -- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 msgr2=0x7f5c80073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7e59c700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 0x7f5c80073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7e59c700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 0x7f5c80073090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 -- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 msgr2=0x7f5c80073090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 0x7f5c80073090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 -- 172.21.15.31:0/2459058861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700095f0 con 0x7f5c80101850 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c7dd9b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 0x7f5c800735d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400d900 tx=0x7f5c7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.254+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c74004a00 con 0x7f5c80101850 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.255+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801a2af0 con 0x7f5c80101850 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.255+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c740054e0 con 0x7f5c80101850 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.255+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c74009d00 con 0x7f5c80101850 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.255+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c801a2f00 con 0x7f5c80101850 2024-04-22T23:49:48.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.256+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c80109240 con 0x7f5c80101850 2024-04-22T23:49:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.256+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f5c74005020 con 0x7f5c80101850 2024-04-22T23:49:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.257+0000 7f5c6f7fe700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5c6806edf0 0x7f5c680712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.257+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5c7408f020 con 0x7f5c80101850 2024-04-22T23:49:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.257+0000 7f5c7e59c700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5c6806edf0 0x7f5c680712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.257+0000 7f5c7e59c700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5c6806edf0 0x7f5c680712b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b5c0 tx=0x7f5c7001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.262+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5c74057c90 con 0x7f5c80101850 2024-04-22T23:49:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.494+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c800746d0 con 0x7f5c6806edf0 2024-04-22T23:49:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.496+0000 7f5c6f7fe700 1 -- 172.21.15.31:0/2459058861 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f5c800746d0 con 0x7f5c6806edf0 2024-04-22T23:49:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5c6806edf0 msgr2=0x7f5c680712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5c6806edf0 0x7f5c680712b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000b5c0 tx=0x7f5c7001a040 comp rx=0 tx=0).stop 2024-04-22T23:49:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 msgr2=0x7f5c800735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 0x7f5c800735d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400d900 tx=0x7f5c7400dcc0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 shutdown_connections 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5c6806edf0 0x7f5c680712b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c80100ea0 0x7f5c80073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 --2- 172.21.15.31:0/2459058861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c80101850 0x7f5c800735d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 >> 172.21.15.31:0/2459058861 conn(0x7f5c800fc750 msgr2=0x7f5c800fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 shutdown_connections 2024-04-22T23:49:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.498+0000 7f5c84b5b700 1 -- 172.21.15.31:0/2459058861 wait complete. 2024-04-22T23:49:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 -- 172.21.15.31:0/2042189919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4101e00 msgr2=0x7fc0b41021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/2042189919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4101e00 0x7fc0b41021e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b0a0 tx=0x7fc0a400b3b0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 -- 172.21.15.31:0/2042189919 shutdown_connections 2024-04-22T23:49:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/2042189919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4102720 0x7fc0b410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/2042189919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4101e00 0x7fc0b41021e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 -- 172.21.15.31:0/2042189919 >> 172.21.15.31:0/2042189919 conn(0x7fc0b40fb650 msgr2=0x7fc0b40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 -- 172.21.15.31:0/2042189919 shutdown_connections 2024-04-22T23:49:48.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.609+0000 7fc0b8f39700 1 -- 172.21.15.31:0/2042189919 wait complete. 2024-04-22T23:49:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b8f39700 1 Processor -- start 2024-04-22T23:49:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b8f39700 1 -- start start 2024-04-22T23:49:48.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b8f39700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 0x7fc0b419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b8f39700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 0x7fc0b419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b8f39700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b419d260 con 0x7fc0b4102720 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b8f39700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4196710 con 0x7fc0b4101e00 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 0x7fc0b419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 0x7fc0b419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51060/0 (socket says 172.21.15.31:51060) 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b1d9b700 1 -- 172.21.15.31:0/993485730 learned_addr learned my addr 172.21.15.31:0/993485730 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.610+0000 7fc0b259c700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 0x7fc0b419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b259c700 1 -- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 msgr2=0x7fc0b419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b259c700 1 -- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 msgr2=0x7fc0b419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b259c700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 0x7fc0b419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b259c700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 0x7fc0b419c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b1d9b700 1 -- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 msgr2=0x7fc0b419c690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b1d9b700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 0x7fc0b419c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b1d9b700 1 -- 172.21.15.31:0/993485730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a4009d00 con 0x7fc0b4102720 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b1d9b700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 0x7fc0b419cbd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800d8d0 tx=0x7fc0a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0a80041f0 con 0x7fc0b4102720 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0a8004350 con 0x7fc0b4102720 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0a8003d60 con 0x7fc0b4102720 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b41969f0 con 0x7fc0b4102720 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.611+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b4196f40 con 0x7fc0b4102720 2024-04-22T23:49:48.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.612+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b4108310 con 0x7fc0b4102720 2024-04-22T23:49:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.613+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fc0a8009730 con 0x7fc0b4102720 2024-04-22T23:49:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.613+0000 7fc0a37fe700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc09c06ee00 0x7fc09c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.613+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc0a808ece0 con 0x7fc0b4102720 2024-04-22T23:49:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.613+0000 7fc0b259c700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc09c06ee00 0x7fc09c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.614+0000 7fc0b259c700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc09c06ee00 0x7fc09c0712c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b830 tx=0x7fc0a40061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:48.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.618+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc0a805b400 con 0x7fc0b4102720 2024-04-22T23:49:48.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.836+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0b4197160 con 0x7fc09c06ee00 2024-04-22T23:49:48.827 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:48 smithi031 ceph-mon[111289]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-22T23:49:48.828 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:48 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:49:48.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.847+0000 7fc0a37fe700 1 -- 172.21.15.31:0/993485730 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc0b4197160 con 0x7fc09c06ee00 2024-04-22T23:49:48.838 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (2m) 12s ago 12m 16.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (12m) 12s ago 12m 8006k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (7m) 15s ago 11m 7876k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (12m) 12s ago 12m 6665k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (11m) 15s ago 11m 6669k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (96s) 12s ago 12m 76.0M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (7m) 12s ago 7m 15.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (7m) 12s ago 7m 93.0M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (7m) 15s ago 7m 148M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (7m) 15s ago 7m 19.1M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (4m) 12s ago 14m 474M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (3m) 15s ago 11m 448M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (27s) 12s ago 14m 28.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 starting - - - 2048M 2024-04-22T23:49:48.839 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (3m) 12s ago 12m 14.9M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (2m) 15s ago 11m 15.7M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (10m) 12s ago 10m 198M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (10m) 12s ago 10m 206M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (9m) 12s ago 9m 175M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (9m) 15s ago 9m 236M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (9m) 15s ago 9m 209M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (8m) 15s ago 8m 196M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (2m) 12s ago 12m 63.1M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:49:48.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc09c06ee00 msgr2=0x7fc09c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc09c06ee00 0x7fc09c0712c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b830 tx=0x7fc0a40061f0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 msgr2=0x7fc0b419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 0x7fc0b419cbd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800d8d0 tx=0x7fc0a800dc90 comp rx=0 tx=0).stop 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 shutdown_connections 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc09c06ee00 0x7fc09c0712c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b4101e00 0x7fc0b419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 --2- 172.21.15.31:0/993485730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0b4102720 0x7fc0b419cbd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 >> 172.21.15.31:0/993485730 conn(0x7fc0b40fb650 msgr2=0x7fc0b4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 shutdown_connections 2024-04-22T23:49:48.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.849+0000 7fc0b8f39700 1 -- 172.21.15.31:0/993485730 wait complete. 2024-04-22T23:49:48.875 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:48 smithi044 ceph-mon[82146]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-22T23:49:48.875 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:48 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.954+0000 7f07ad734700 1 -- 172.21.15.31:0/3252744739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8101850 msgr2=0x7f07a81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.954+0000 7f07ad734700 1 --2- 172.21.15.31:0/3252744739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8101850 0x7f07a81058a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f079c009910 tx=0x7f079c009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.954+0000 7f07ad734700 1 -- 172.21.15.31:0/3252744739 shutdown_connections 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.954+0000 7f07ad734700 1 --2- 172.21.15.31:0/3252744739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8101850 0x7f07a81058a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.954+0000 7f07ad734700 1 --2- 172.21.15.31:0/3252744739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8100ea0 0x7f07a8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.954+0000 7f07ad734700 1 -- 172.21.15.31:0/3252744739 >> 172.21.15.31:0/3252744739 conn(0x7f07a80fc750 msgr2=0x7f07a80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.955+0000 7f07ad734700 1 -- 172.21.15.31:0/3252744739 shutdown_connections 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.955+0000 7f07ad734700 1 -- 172.21.15.31:0/3252744739 wait complete. 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.955+0000 7f07ad734700 1 Processor -- start 2024-04-22T23:49:48.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.955+0000 7f07ad734700 1 -- start start 2024-04-22T23:49:48.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07ad734700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 0x7f07a819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07ad734700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 0x7f07a819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07ad734700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a819d2d0 con 0x7f07a8100ea0 2024-04-22T23:49:48.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07ad734700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a8196780 con 0x7f07a8101850 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 0x7f07a819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 0x7f07a819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51080/0 (socket says 172.21.15.31:51080) 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 -- 172.21.15.31:0/421367279 learned_addr learned my addr 172.21.15.31:0/421367279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a67fc700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 0x7f07a819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a67fc700 1 -- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 msgr2=0x7f07a819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a67fc700 1 -- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 msgr2=0x7f07a819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a67fc700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 0x7f07a819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a67fc700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 0x7f07a819cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 -- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 msgr2=0x7f07a819cc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 0x7f07a819cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 -- 172.21.15.31:0/421367279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0095f0 con 0x7f07a8100ea0 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07a6ffd700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 0x7f07a819c700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f079800d8d0 tx=0x7f079800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0798009940 con 0x7f07a8100ea0 2024-04-22T23:49:48.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a8196a60 con 0x7f07a8100ea0 2024-04-22T23:49:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0798010460 con 0x7f07a8100ea0 2024-04-22T23:49:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f079800f5f0 con 0x7f07a8100ea0 2024-04-22T23:49:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.956+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8196fb0 con 0x7f07a8100ea0 2024-04-22T23:49:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.957+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a81092a0 con 0x7f07a8100ea0 2024-04-22T23:49:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.958+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f07980105d0 con 0x7f07a8100ea0 2024-04-22T23:49:48.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.958+0000 7f078ffff700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f079006ee00 0x7f07900712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.958+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f079808f130 con 0x7f07a8100ea0 2024-04-22T23:49:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.958+0000 7f07a67fc700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f079006ee00 0x7f07900712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:48.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.959+0000 7f07a67fc700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f079006ee00 0x7f07900712c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f07a8197bf0 tx=0x7f079c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:48.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:48.963+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f079805b7b0 con 0x7f07a8100ea0 2024-04-22T23:49:49.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.266+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f07a804f2e0 con 0x7f07a8100ea0 2024-04-22T23:49:49.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.267+0000 7f078ffff700 1 -- 172.21.15.31:0/421367279 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f079805b340 con 0x7f07a8100ea0 2024-04-22T23:49:49.258 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:49:49.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:49:49.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:49:49.260 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f079006ee00 msgr2=0x7f07900712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f079006ee00 0x7f07900712c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f07a8197bf0 tx=0x7f079c01a040 comp rx=0 tx=0).stop 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 msgr2=0x7f07a819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 0x7f07a819c700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f079800d8d0 tx=0x7f079800dc90 comp rx=0 tx=0).stop 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 shutdown_connections 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f079006ee00 0x7f07900712c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a8100ea0 0x7f07a819c700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 --2- 172.21.15.31:0/421367279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07a8101850 0x7f07a819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.269+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 >> 172.21.15.31:0/421367279 conn(0x7f07a80fc750 msgr2=0x7f07a80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:49.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.270+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 shutdown_connections 2024-04-22T23:49:49.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.270+0000 7f07ad734700 1 -- 172.21.15.31:0/421367279 wait complete. 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.376+0000 7fde0649d700 1 -- 172.21.15.31:0/2919125486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00101850 msgr2=0x7fde001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.376+0000 7fde0649d700 1 --2- 172.21.15.31:0/2919125486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00101850 0x7fde001058a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009910 tx=0x7fddf4009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.376+0000 7fde0649d700 1 -- 172.21.15.31:0/2919125486 shutdown_connections 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.376+0000 7fde0649d700 1 --2- 172.21.15.31:0/2919125486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00101850 0x7fde001058a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.376+0000 7fde0649d700 1 --2- 172.21.15.31:0/2919125486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00100ea0 0x7fde00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.376+0000 7fde0649d700 1 -- 172.21.15.31:0/2919125486 >> 172.21.15.31:0/2919125486 conn(0x7fde000fc750 msgr2=0x7fde000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.377+0000 7fde0649d700 1 -- 172.21.15.31:0/2919125486 shutdown_connections 2024-04-22T23:49:49.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.377+0000 7fde0649d700 1 -- 172.21.15.31:0/2919125486 wait complete. 2024-04-22T23:49:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.377+0000 7fde0649d700 1 Processor -- start 2024-04-22T23:49:49.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.377+0000 7fde0649d700 1 -- start start 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fde0649d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 0x7fde0019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fde0649d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 0x7fde0019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fde0649d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019f430 con 0x7fde00100ea0 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fde0649d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001988e0 con 0x7fde00101850 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 0x7fde0019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 0x7fde0019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51100/0 (socket says 172.21.15.31:51100) 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 -- 172.21.15.31:0/1762460735 learned_addr learned my addr 172.21.15.31:0/1762460735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddff7fe700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 0x7fde0019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddff7fe700 1 -- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 msgr2=0x7fde0019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddff7fe700 1 -- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 msgr2=0x7fde0019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddff7fe700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 0x7fde0019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 -- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 msgr2=0x7fde0019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddff7fe700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 0x7fde0019eda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 0x7fde0019eda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 -- 172.21.15.31:0/1762460735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40095f0 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfffff700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 0x7fde0019e860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d8d0 tx=0x7fddf000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf0009940 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf0010460 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde00198bc0 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf000f5f0 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.378+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde00199110 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.379+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00105020 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.380+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fddf00105d0 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.380+0000 7fddfd7fa700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdde806ee00 0x7fdde80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.380+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fddf008f0a0 con 0x7fde00100ea0 2024-04-22T23:49:49.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.380+0000 7fddff7fe700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdde806ee00 0x7fdde80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:49.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.381+0000 7fddff7fe700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdde806ee00 0x7fdde80712c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fde000fcee0 tx=0x7fddf401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:49.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.385+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fddf005b7c0 con 0x7fde00100ea0 2024-04-22T23:49:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.642+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fde00199330 con 0x7fde00100ea0 2024-04-22T23:49:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.643+0000 7fddfd7fa700 1 -- 172.21.15.31:0/1762460735 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7fddf005b350 con 0x7fde00100ea0 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:49:49.635 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:49:49.636 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:49:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:49:49.638 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:49.638 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdde806ee00 msgr2=0x7fdde80712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdde806ee00 0x7fdde80712c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fde000fcee0 tx=0x7fddf401a040 comp rx=0 tx=0).stop 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 msgr2=0x7fde0019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 0x7fde0019e860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d8d0 tx=0x7fddf000dc90 comp rx=0 tx=0).stop 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 shutdown_connections 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdde806ee00 0x7fdde80712c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde00100ea0 0x7fde0019e860 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 --2- 172.21.15.31:0/1762460735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde00101850 0x7fde0019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 >> 172.21.15.31:0/1762460735 conn(0x7fde000fc750 msgr2=0x7fde000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 shutdown_connections 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.645+0000 7fde0649d700 1 -- 172.21.15.31:0/1762460735 wait complete. 2024-04-22T23:49:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:49:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 -- 172.21.15.31:0/3993460532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 msgr2=0x7fa97c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 --2- 172.21.15.31:0/3993460532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c1070b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00b0a0 tx=0x7fa96c00b3b0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 -- 172.21.15.31:0/3993460532 shutdown_connections 2024-04-22T23:49:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 --2- 172.21.15.31:0/3993460532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 --2- 172.21.15.31:0/3993460532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c1070b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 -- 172.21.15.31:0/3993460532 >> 172.21.15.31:0/3993460532 conn(0x7fa97c075670 msgr2=0x7fa97c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 -- 172.21.15.31:0/3993460532 shutdown_connections 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.753+0000 7fa981736700 1 -- 172.21.15.31:0/3993460532 wait complete. 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa981736700 1 Processor -- start 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa981736700 1 -- start start 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa981736700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa981736700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa981736700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c19d230 con 0x7fa97c106cd0 2024-04-22T23:49:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa981736700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c1966e0 con 0x7fa97c068580 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51120/0 (socket says 172.21.15.31:51120) 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97a7fc700 1 -- 172.21.15.31:0/3625366516 learned_addr learned my addr 172.21.15.31:0/3625366516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97affd700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97affd700 1 -- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 msgr2=0x7fa97c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97affd700 1 -- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 msgr2=0x7fa97c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97affd700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97affd700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c19c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97a7fc700 1 -- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 msgr2=0x7fa97c19c660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97a7fc700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c19c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.754+0000 7fa97a7fc700 1 -- 172.21.15.31:0/3625366516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c009d00 con 0x7fa97c106cd0 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.755+0000 7fa97a7fc700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c19cba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa97000d8d0 tx=0x7fa97000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.755+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa970009940 con 0x7fa97c106cd0 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.755+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa970010460 con 0x7fa97c106cd0 2024-04-22T23:49:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.755+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c1969c0 con 0x7fa97c106cd0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.755+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa97000f5f0 con 0x7fa97c106cd0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.755+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c196f10 con 0x7fa97c106cd0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.756+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97c1091d0 con 0x7fa97c106cd0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.756+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fa9700105d0 con 0x7fa97c106cd0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.757+0000 7fa963fff700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa96406ee00 0x7fa9640712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.757+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa97008ef90 con 0x7fa97c106cd0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.757+0000 7fa97affd700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa96406ee00 0x7fa9640712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.757+0000 7fa97affd700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa96406ee00 0x7fa9640712c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00b830 tx=0x7fa96c0061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.762+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa97005b730 con 0x7fa97c106cd0 2024-04-22T23:49:49.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: pgmap v12: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:49:49.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:49.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:49.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:49:49.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: mgrmap e37: smithi031.fazdcv(active, since 18s), standbys: smithi044.iglcti 2024-04-22T23:49:49.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: mon.smithi031 calling monitor election 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: mon.smithi044 calling monitor election 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: mon.smithi031 is new leader, mons smithi031,smithi044 in quorum (ranks 0,1) 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: monmap epoch 3 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: last_changed 2024-04-22T23:49:48.567777+0000 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: created 2024-04-22T23:35:31.547688+0000 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: min_mon_release 19 (squid) 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: election_strategy: 1 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-22T23:49:49.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: mgrmap e37: smithi031.fazdcv(active, since 18s), standbys: smithi044.iglcti 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:49.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:49 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/421367279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:49.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: pgmap v12: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: mgrmap e37: smithi031.fazdcv(active, since 18s), standbys: smithi044.iglcti 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mon metadata", "id": "smithi044"}]: dispatch 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: mon.smithi031 calling monitor election 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: mon.smithi044 calling monitor election 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: mon.smithi031 is new leader, mons smithi031,smithi044 in quorum (ranks 0,1) 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: monmap epoch 3 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: fsid efa92c84-0100-11ef-bc93-c7b262605968 2024-04-22T23:49:49.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: last_changed 2024-04-22T23:49:48.567777+0000 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: created 2024-04-22T23:35:31.547688+0000 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: min_mon_release 19 (squid) 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: election_strategy: 1 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: 1: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.smithi044 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: mgrmap e37: smithi031.fazdcv(active, since 18s), standbys: smithi044.iglcti 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:49.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:49.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:49.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:49:49.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:49.963 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:49 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/421367279' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:49:49.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.994+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa97c197180 con 0x7fa96406ee00 2024-04-22T23:49:49.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.995+0000 7fa963fff700 1 -- 172.21.15.31:0/3625366516 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa97c197180 con 0x7fa96406ee00 2024-04-22T23:49:49.986 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:49:49.986 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "3/23 daemons upgraded", 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:49:49.987 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:49:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa96406ee00 msgr2=0x7fa9640712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa96406ee00 0x7fa9640712c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00b830 tx=0x7fa96c0061f0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 msgr2=0x7fa97c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c19cba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa97000d8d0 tx=0x7fa97000dc90 comp rx=0 tx=0).stop 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 shutdown_connections 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa96406ee00 0x7fa9640712c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa97c068580 0x7fa97c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 --2- 172.21.15.31:0/3625366516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa97c106cd0 0x7fa97c19cba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 >> 172.21.15.31:0/3625366516 conn(0x7fa97c075670 msgr2=0x7fa97c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 shutdown_connections 2024-04-22T23:49:49.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:49.997+0000 7fa981736700 1 -- 172.21.15.31:0/3625366516 wait complete. 2024-04-22T23:49:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 -- 172.21.15.31:0/300177627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101850 msgr2=0x7f2f501058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 --2- 172.21.15.31:0/300177627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101850 0x7f2f501058a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009910 tx=0x7f2f44009c20 comp rx=0 tx=0).stop 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 -- 172.21.15.31:0/300177627 shutdown_connections 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 --2- 172.21.15.31:0/300177627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101850 0x7f2f501058a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 --2- 172.21.15.31:0/300177627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50100ea0 0x7f2f50101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 -- 172.21.15.31:0/300177627 >> 172.21.15.31:0/300177627 conn(0x7f2f500fc750 msgr2=0x7f2f500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 -- 172.21.15.31:0/300177627 shutdown_connections 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 -- 172.21.15.31:0/300177627 wait complete. 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 Processor -- start 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.105+0000 7f2f56152700 1 -- start start 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f56152700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 0x7f2f5019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f56152700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 0x7f2f5019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:50.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f56152700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5019d240 con 0x7f2f50100ea0 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f56152700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50196780 con 0x7f2f50101850 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 0x7f2f5019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 0x7f2f5019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51128/0 (socket says 172.21.15.31:51128) 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 -- 172.21.15.31:0/2215008207 learned_addr learned my addr 172.21.15.31:0/2215008207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4effd700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 0x7f2f5019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4effd700 1 -- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 msgr2=0x7f2f5019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4effd700 1 -- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 msgr2=0x7f2f5019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4effd700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 0x7f2f5019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4effd700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 0x7f2f5019cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 -- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 msgr2=0x7f2f5019cc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 0x7f2f5019cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 -- 172.21.15.31:0/2215008207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440095f0 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.106+0000 7f2f4f7fe700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 0x7f2f5019c700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c930 tx=0x7f2f4000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.107+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f40007ad0 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.107+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f50196a60 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.107+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f40007c30 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.107+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f40018610 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.107+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f50196fb0 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.108+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f501092a0 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.108+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f2f40018830 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.108+0000 7f2f4cff9700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2f3806ee00 0x7f2f380712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.109+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2f4008f220 con 0x7f2f50100ea0 2024-04-22T23:49:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.109+0000 7f2f4effd700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2f3806ee00 0x7f2f380712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:49:50.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.109+0000 7f2f4effd700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2f3806ee00 0x7f2f380712c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2f50197bf0 tx=0x7f2f4401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:49:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.113+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2f4005bb10 con 0x7f2f50100ea0 2024-04-22T23:49:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.411+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2f50197d50 con 0x7f2f50100ea0 2024-04-22T23:49:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.411+0000 7f2f4cff9700 1 -- 172.21.15.31:0/2215008207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2f4005b6a0 con 0x7f2f50100ea0 2024-04-22T23:49:50.403 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:50.403 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:49:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.413+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2f3806ee00 msgr2=0x7f2f380712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.413+0000 7f2f56152700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2f3806ee00 0x7f2f380712c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2f50197bf0 tx=0x7f2f4401a040 comp rx=0 tx=0).stop 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.413+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 msgr2=0x7f2f5019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.413+0000 7f2f56152700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 0x7f2f5019c700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c930 tx=0x7f2f4000ccf0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 shutdown_connections 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2f3806ee00 0x7f2f380712c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50100ea0 0x7f2f5019c700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 --2- 172.21.15.31:0/2215008207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f50101850 0x7f2f5019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 >> 172.21.15.31:0/2215008207 conn(0x7f2f500fc750 msgr2=0x7f2f500fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 shutdown_connections 2024-04-22T23:49:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:49:50.414+0000 7f2f56152700 1 -- 172.21.15.31:0/2215008207 wait complete. 2024-04-22T23:49:50.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:50 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1762460735' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:49:50.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:50 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2215008207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:49:50.913 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:50 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1762460735' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:49:50.913 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:50 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2215008207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:49:51.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:51 smithi031 ceph-mon[111289]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:51.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:51 smithi031 ceph-mon[111289]: pgmap v13: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:51.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:51 smithi044 ceph-mon[82146]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:49:51.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:51 smithi044 ceph-mon[82146]: pgmap v13: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:52.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:52 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:52.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:52 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:52.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:52 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:52.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:52 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:53.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:53 smithi031 ceph-mon[111289]: pgmap v14: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:53.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:53 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:53.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:53 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:53 smithi044 ceph-mon[82146]: pgmap v14: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:53 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:53 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:55.917 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:55 smithi044 ceph-mon[82146]: pgmap v15: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:56.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:55 smithi031 ceph-mon[111289]: pgmap v15: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: pgmap v16: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: Detected new or changed devices on smithi044 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:57.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:57.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: Reconfiguring mon.smithi031 (monmap changed)... 2024-04-22T23:49:57.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:49:57.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:49:57.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:57.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:57 smithi031 ceph-mon[111289]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-22T23:49:57.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: pgmap v16: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:49:57.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: Detected new or changed devices on smithi044 2024-04-22T23:49:57.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:57.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:57.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:57.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:49:57.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:57.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: Reconfiguring mon.smithi031 (monmap changed)... 2024-04-22T23:49:57.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:49:57.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:49:57.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:57.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:57 smithi044 ceph-mon[82146]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-22T23:49:59.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: Reconfiguring mgr.smithi031.fazdcv (monmap changed)... 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: Reconfiguring daemon mgr.smithi031.fazdcv on smithi031 2024-04-22T23:49:59.291 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:49:58 smithi031 ceph-mon[111289]: pgmap v17: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:49:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:49:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: Reconfiguring mgr.smithi031.fazdcv (monmap changed)... 2024-04-22T23:49:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.fazdcv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:49:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:49:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:49:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: Reconfiguring daemon mgr.smithi031.fazdcv on smithi031 2024-04-22T23:49:59.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:49:58 smithi044 ceph-mon[82146]: pgmap v17: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:00.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:00.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:00.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-04-22T23:50:00.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:00.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:00.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:50:00.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: Unable to update caps for client.ceph-exporter.smithi031 2024-04-22T23:50:00.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi031"}]: dispatch 2024-04-22T23:50:00.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:00.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:50:00.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:50:00.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: Unable to update caps for client.ceph-exporter.smithi031 2024-04-22T23:50:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi031"}]: dispatch 2024-04-22T23:50:00.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:00.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:50:00.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:50:00.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:01.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:01 smithi031 ceph-mon[111289]: pgmap v18: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:01.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:01.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:01.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:01.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:01 smithi044 ceph-mon[82146]: pgmap v18: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:02.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:02 smithi031 ceph-mon[111289]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-22T23:50:02.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:02 smithi031 ceph-mon[111289]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-22T23:50:02.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:02 smithi044 ceph-mon[82146]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-22T23:50:02.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:02 smithi044 ceph-mon[82146]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-22T23:50:04.102 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: pgmap v19: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:04.102 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:04.102 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:04.102 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: Reconfiguring osd.0 (monmap changed)... 2024-04-22T23:50:04.102 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:50:04.102 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:04.103 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:03 smithi031 ceph-mon[111289]: Reconfiguring daemon osd.0 on smithi031 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: pgmap v19: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: Reconfiguring osd.0 (monmap changed)... 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:04.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:03 smithi044 ceph-mon[82146]: Reconfiguring daemon osd.0 on smithi031 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: pgmap v20: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: Reconfiguring osd.1 (monmap changed)... 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:05.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:05 smithi031 ceph-mon[111289]: Reconfiguring daemon osd.1 on smithi031 2024-04-22T23:50:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: pgmap v20: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: Reconfiguring osd.1 (monmap changed)... 2024-04-22T23:50:05.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:50:05.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:05.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:05 smithi044 ceph-mon[82146]: Reconfiguring daemon osd.1 on smithi031 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: pgmap v21: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: Reconfiguring osd.2 (monmap changed)... 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:07.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:07 smithi031 ceph-mon[111289]: Reconfiguring daemon osd.2 on smithi031 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: pgmap v21: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: Reconfiguring osd.2 (monmap changed)... 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:07 smithi044 ceph-mon[82146]: Reconfiguring daemon osd.2 on smithi031 2024-04-22T23:50:08.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:08 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:08.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:08 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:08.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:08 smithi031 ceph-mon[111289]: Reconfiguring mds.cephfs.smithi031.ezuwxk (monmap changed)... 2024-04-22T23:50:08.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:08 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:08.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:08 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:08.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:08 smithi031 ceph-mon[111289]: Reconfiguring daemon mds.cephfs.smithi031.ezuwxk on smithi031 2024-04-22T23:50:09.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:08 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:09.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:08 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:09.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:08 smithi044 ceph-mon[82146]: Reconfiguring mds.cephfs.smithi031.ezuwxk (monmap changed)... 2024-04-22T23:50:09.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:08 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:09.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:08 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:09.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:08 smithi044 ceph-mon[82146]: Reconfiguring daemon mds.cephfs.smithi031.ezuwxk on smithi031 2024-04-22T23:50:10.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:09 smithi031 ceph-mon[111289]: pgmap v22: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:10.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:10.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:10.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:10.072 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:09 smithi044 ceph-mon[82146]: pgmap v22: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:11.042 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:10 smithi031 ceph-mon[111289]: Reconfiguring mds.cephfs.smithi031.usiyoa (monmap changed)... 2024-04-22T23:50:11.042 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:10 smithi031 ceph-mon[111289]: Reconfiguring daemon mds.cephfs.smithi031.usiyoa on smithi031 2024-04-22T23:50:11.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:10 smithi044 ceph-mon[82146]: Reconfiguring mds.cephfs.smithi031.usiyoa (monmap changed)... 2024-04-22T23:50:11.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:10 smithi044 ceph-mon[82146]: Reconfiguring daemon mds.cephfs.smithi031.usiyoa on smithi031 2024-04-22T23:50:12.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: pgmap v23: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:12.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:12.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:12.026 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:12.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:12.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:50:12.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi044"}]: dispatch 2024-04-22T23:50:12.027 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: pgmap v23: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-22T23:50:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi044"}]: dispatch 2024-04-22T23:50:12.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:13.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:12 smithi031 ceph-mon[111289]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-04-22T23:50:13.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:12 smithi031 ceph-mon[111289]: Unable to update caps for client.ceph-exporter.smithi044 2024-04-22T23:50:13.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:12 smithi031 ceph-mon[111289]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:50:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:12 smithi044 ceph-mon[82146]: Reconfiguring ceph-exporter.smithi044 (monmap changed)... 2024-04-22T23:50:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:12 smithi044 ceph-mon[82146]: Unable to update caps for client.ceph-exporter.smithi044 2024-04-22T23:50:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:12 smithi044 ceph-mon[82146]: Reconfiguring daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:50:14.138 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:13 smithi044 ceph-mon[82146]: pgmap v24: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:14.138 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:14.138 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:14.138 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:14.138 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:14.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:13 smithi031 ceph-mon[111289]: pgmap v24: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:15.132 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: Reconfiguring crash.smithi044 (monmap changed)... 2024-04-22T23:50:15.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: Reconfiguring daemon crash.smithi044 on smithi044 2024-04-22T23:50:15.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:15.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:15.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:50:15.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:50:15.133 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:14 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:15.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: Reconfiguring crash.smithi044 (monmap changed)... 2024-04-22T23:50:15.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: Reconfiguring daemon crash.smithi044 on smithi044 2024-04-22T23:50:15.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:15.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:15.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi044.iglcti", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:50:15.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:50:15.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:14 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:16.141 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:15 smithi044 ceph-mon[82146]: pgmap v25: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:16.142 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:15 smithi044 ceph-mon[82146]: Reconfiguring mgr.smithi044.iglcti (monmap changed)... 2024-04-22T23:50:16.142 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:15 smithi044 ceph-mon[82146]: Reconfiguring daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:50:16.142 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:16.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:15 smithi031 ceph-mon[111289]: pgmap v25: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:16.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:15 smithi031 ceph-mon[111289]: Reconfiguring mgr.smithi044.iglcti (monmap changed)... 2024-04-22T23:50:16.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:15 smithi031 ceph-mon[111289]: Reconfiguring daemon mgr.smithi044.iglcti on smithi044 2024-04-22T23:50:16.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: pgmap v26: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: Reconfiguring mon.smithi044 (monmap changed)... 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:17 smithi031 ceph-mon[111289]: Reconfiguring daemon mon.smithi044 on smithi044 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: pgmap v26: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: Reconfiguring mon.smithi044 (monmap changed)... 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:17 smithi044 ceph-mon[82146]: Reconfiguring daemon mon.smithi044 on smithi044 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: Reconfiguring osd.3 (monmap changed)... 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: Reconfiguring daemon osd.3 on smithi044 2024-04-22T23:50:19.186 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:18 smithi044 ceph-mon[82146]: pgmap v27: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:19.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: Reconfiguring osd.3 (monmap changed)... 2024-04-22T23:50:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:50:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: Reconfiguring daemon osd.3 on smithi044 2024-04-22T23:50:19.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:18 smithi031 ceph-mon[111289]: pgmap v27: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:20.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.536+0000 7f107b59c700 1 -- 172.21.15.31:0/898287090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 msgr2=0x7f1074068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:20.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.536+0000 7f107b59c700 1 --2- 172.21.15.31:0/898287090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f1074068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1070009910 tx=0x7f1070009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.536+0000 7f107b59c700 1 -- 172.21.15.31:0/898287090 shutdown_connections 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.536+0000 7f107b59c700 1 --2- 172.21.15.31:0/898287090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1074068ea0 0x7f107410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.536+0000 7f107b59c700 1 --2- 172.21.15.31:0/898287090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f1074068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.536+0000 7f107b59c700 1 -- 172.21.15.31:0/898287090 >> 172.21.15.31:0/898287090 conn(0x7f1074075670 msgr2=0x7f1074075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 -- 172.21.15.31:0/898287090 shutdown_connections 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 -- 172.21.15.31:0/898287090 wait complete. 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 Processor -- start 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 -- start start 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f107419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1074068ea0 0x7f107419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107419f430 con 0x7f1074068580 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.537+0000 7f107b59c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10741988e0 con 0x7f1074068ea0 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f107419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:20.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f107419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55204/0 (socket says 172.21.15.31:55204) 2024-04-22T23:50:20.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 -- 172.21.15.31:0/3284483634 learned_addr learned my addr 172.21.15.31:0/3284483634 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:20.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1078b37700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1074068ea0 0x7f107419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:20.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 -- 172.21.15.31:0/3284483634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1074068ea0 msgr2=0x7f107419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:20.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1074068ea0 0x7f107419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 -- 172.21.15.31:0/3284483634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700095f0 con 0x7f1074068580 2024-04-22T23:50:20.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f1079338700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f107419e860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1070005f50 tx=0x7f107000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f107001d070 con 0x7f1074068580 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f107000b610 con 0x7f1074068580 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1074198b60 con 0x7f1074068580 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1070022470 con 0x7f1074068580 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.538+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1074199050 con 0x7f1074068580 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.539+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107410aa00 con 0x7f1074068580 2024-04-22T23:50:20.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.540+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f1070003780 con 0x7f1074068580 2024-04-22T23:50:20.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.540+0000 7f106a7fc700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f106006ee00 0x7f10600712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:20.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.541+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1070090330 con 0x7f1074068580 2024-04-22T23:50:20.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.541+0000 7f1078b37700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f106006ee00 0x7f10600712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:20.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.541+0000 7f1078b37700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f106006ee00 0x7f10600712c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1074101420 tx=0x7f1064009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:20.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.546+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f107005cad0 con 0x7f1074068580 2024-04-22T23:50:20.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.776+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1074199e60 con 0x7f106006ee00 2024-04-22T23:50:20.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.777+0000 7f106a7fc700 1 -- 172.21.15.31:0/3284483634 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f1074199e60 con 0x7f106006ee00 2024-04-22T23:50:20.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f106006ee00 msgr2=0x7f10600712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:20.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f106006ee00 0x7f10600712c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1074101420 tx=0x7f1064009260 comp rx=0 tx=0).stop 2024-04-22T23:50:20.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 msgr2=0x7f107419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:20.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f107419e860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1070005f50 tx=0x7f107000f690 comp rx=0 tx=0).stop 2024-04-22T23:50:20.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 shutdown_connections 2024-04-22T23:50:20.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f106006ee00 0x7f10600712c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1074068580 0x7f107419e860 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 --2- 172.21.15.31:0/3284483634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1074068ea0 0x7f107419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 >> 172.21.15.31:0/3284483634 conn(0x7f1074075670 msgr2=0x7f10740fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:20.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 shutdown_connections 2024-04-22T23:50:20.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.780+0000 7f107b59c700 1 -- 172.21.15.31:0/3284483634 wait complete. 2024-04-22T23:50:20.782 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:50:20.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:20 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:20.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:20 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:20.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:20 smithi031 ceph-mon[111289]: Reconfiguring osd.4 (monmap changed)... 2024-04-22T23:50:20.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:20 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:50:20.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:20 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:20.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:20 smithi031 ceph-mon[111289]: Reconfiguring daemon osd.4 on smithi044 2024-04-22T23:50:20.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 -- 172.21.15.31:0/283314649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 msgr2=0x7f215410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:20.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 --2- 172.21.15.31:0/283314649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215410acd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2148009940 tx=0x7f2148009c50 comp rx=0 tx=0).stop 2024-04-22T23:50:20.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 -- 172.21.15.31:0/283314649 shutdown_connections 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 --2- 172.21.15.31:0/283314649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215410acd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 --2- 172.21.15.31:0/283314649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154101ec0 0x7f21541022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 -- 172.21.15.31:0/283314649 >> 172.21.15.31:0/283314649 conn(0x7f21540fb650 msgr2=0x7f21540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 -- 172.21.15.31:0/283314649 shutdown_connections 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.887+0000 7f2159d4b700 1 -- 172.21.15.31:0/283314649 wait complete. 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.888+0000 7f2159d4b700 1 Processor -- start 2024-04-22T23:50:20.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.888+0000 7f2159d4b700 1 -- start start 2024-04-22T23:50:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.888+0000 7f2159d4b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154101ec0 0x7f215419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:20.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.888+0000 7f2159d4b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.888+0000 7f2159d4b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419d230 con 0x7f21541027e0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.888+0000 7f2159d4b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21541966e0 con 0x7f2154101ec0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55476/0 (socket says 172.21.15.31:55476) 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 -- 172.21.15.31:0/3303809149 learned_addr learned my addr 172.21.15.31:0/3303809149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f21537fe700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154101ec0 0x7f215419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 -- 172.21.15.31:0/3303809149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154101ec0 msgr2=0x7f215419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154101ec0 0x7f215419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 -- 172.21.15.31:0/3303809149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480095f0 con 0x7f21541027e0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2152ffd700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215419cba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21480055f0 tx=0x7f2148005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f214801d070 con 0x7f21541027e0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2148005920 con 0x7f21541027e0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2154196960 con 0x7f21541027e0 2024-04-22T23:50:20.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2148021410 con 0x7f21541027e0 2024-04-22T23:50:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.889+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154196e50 con 0x7f21541027e0 2024-04-22T23:50:20.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.890+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2154108360 con 0x7f21541027e0 2024-04-22T23:50:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.891+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f214800f6f0 con 0x7f21541027e0 2024-04-22T23:50:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.891+0000 7f2150ff9700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f213c06ee00 0x7f213c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.891+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2148090250 con 0x7f21541027e0 2024-04-22T23:50:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.891+0000 7f21537fe700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f213c06ee00 0x7f213c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:20.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.892+0000 7f21537fe700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f213c06ee00 0x7f213c0712c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2144005950 tx=0x7f214400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:20.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:20.896+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f214805ca20 con 0x7f21541027e0 2024-04-22T23:50:20.907 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:20 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:20.907 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:20 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:20.907 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:20 smithi044 ceph-mon[82146]: Reconfiguring osd.4 (monmap changed)... 2024-04-22T23:50:20.907 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:20 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:50:20.907 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:20 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:20.908 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:20 smithi044 ceph-mon[82146]: Reconfiguring daemon osd.4 on smithi044 2024-04-22T23:50:21.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.127+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2154197130 con 0x7f213c06ee00 2024-04-22T23:50:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.128+0000 7f2150ff9700 1 -- 172.21.15.31:0/3303809149 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2154197130 con 0x7f213c06ee00 2024-04-22T23:50:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f213c06ee00 msgr2=0x7f213c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f213c06ee00 0x7f213c0712c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2144005950 tx=0x7f214400a300 comp rx=0 tx=0).stop 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 msgr2=0x7f215419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215419cba0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21480055f0 tx=0x7f2148005620 comp rx=0 tx=0).stop 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 shutdown_connections 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f213c06ee00 0x7f213c0712c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2154101ec0 0x7f215419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 --2- 172.21.15.31:0/3303809149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21541027e0 0x7f215419cba0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 >> 172.21.15.31:0/3303809149 conn(0x7f21540fb650 msgr2=0x7f21541054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 shutdown_connections 2024-04-22T23:50:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.130+0000 7f2159d4b700 1 -- 172.21.15.31:0/3303809149 wait complete. 2024-04-22T23:50:21.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.237+0000 7f9a241f1700 1 -- 172.21.15.31:0/2234592309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 msgr2=0x7f9a1c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.237+0000 7f9a241f1700 1 --2- 172.21.15.31:0/2234592309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c101280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009910 tx=0x7f9a18009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:21.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.237+0000 7f9a241f1700 1 -- 172.21.15.31:0/2234592309 shutdown_connections 2024-04-22T23:50:21.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.237+0000 7f9a241f1700 1 --2- 172.21.15.31:0/2234592309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c101850 0x7f9a1c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.237+0000 7f9a241f1700 1 --2- 172.21.15.31:0/2234592309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c101280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.237+0000 7f9a241f1700 1 -- 172.21.15.31:0/2234592309 >> 172.21.15.31:0/2234592309 conn(0x7f9a1c0fc750 msgr2=0x7f9a1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.238+0000 7f9a241f1700 1 -- 172.21.15.31:0/2234592309 shutdown_connections 2024-04-22T23:50:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.238+0000 7f9a241f1700 1 -- 172.21.15.31:0/2234592309 wait complete. 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.238+0000 7f9a241f1700 1 Processor -- start 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.238+0000 7f9a241f1700 1 -- start start 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a241f1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a241f1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c101850 0x7f9a1c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a241f1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c073b10 con 0x7f9a1c100ea0 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a241f1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c073c50 con 0x7f9a1c101850 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55488/0 (socket says 172.21.15.31:55488) 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 -- 172.21.15.31:0/750915515 learned_addr learned my addr 172.21.15.31:0/750915515 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a2178c700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c101850 0x7f9a1c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 -- 172.21.15.31:0/750915515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c101850 msgr2=0x7f9a1c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c101850 0x7f9a1c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 -- 172.21.15.31:0/750915515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180095f0 con 0x7f9a1c100ea0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a21f8d700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c073090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005f50 tx=0x7f9a1800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.239+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a1801d070 con 0x7f9a1c100ea0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.240+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c073ed0 con 0x7f9a1c100ea0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.240+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a1800b610 con 0x7f9a1c100ea0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.240+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a18022470 con 0x7f9a1c100ea0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.240+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c1a2e20 con 0x7f9a1c100ea0 2024-04-22T23:50:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.240+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c109240 con 0x7f9a1c100ea0 2024-04-22T23:50:21.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.241+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f9a18003780 con 0x7f9a1c100ea0 2024-04-22T23:50:21.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.241+0000 7f9a12ffd700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9a0806ee00 0x7f9a080712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:21.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.242+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9a18090440 con 0x7f9a1c100ea0 2024-04-22T23:50:21.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.242+0000 7f9a2178c700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9a0806ee00 0x7f9a080712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:21.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.242+0000 7f9a2178c700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9a0806ee00 0x7f9a080712c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c074880 tx=0x7f9a0c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:21.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.246+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9a18059030 con 0x7f9a1c100ea0 2024-04-22T23:50:21.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.469+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a1c0611d0 con 0x7f9a0806ee00 2024-04-22T23:50:21.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.480+0000 7f9a12ffd700 1 -- 172.21.15.31:0/750915515 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9a1c0611d0 con 0x7f9a0806ee00 2024-04-22T23:50:21.471 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:50:21.471 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (2m) 45s ago 13m 16.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:50:21.471 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (13m) 45s ago 13m 8006k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:50:21.471 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (7m) 29s ago 12m 7906k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (13m) 45s ago 13m 6665k - 18.2.1 5be31c24972a c16998a64c5e 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (12m) 29s ago 12m 6669k - 18.2.1 5be31c24972a 5a21f89aeca9 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (2m) 45s ago 12m 76.0M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (7m) 45s ago 7m 15.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (7m) 45s ago 7m 93.0M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (7m) 29s ago 7m 148M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (7m) 29s ago 7m 19.7M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (4m) 45s ago 14m 474M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (3m) 29s ago 12m 449M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (60s) 45s ago 14m 28.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (34s) 29s ago 12m 25.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (3m) 45s ago 13m 14.9M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (3m) 29s ago 12m 15.4M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (11m) 45s ago 11m 198M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:50:21.472 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (10m) 45s ago 10m 206M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (10m) 45s ago 10m 175M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (10m) 29s ago 10m 237M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (9m) 29s ago 9m 210M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (9m) 29s ago 9m 196M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (2m) 45s ago 12m 63.1M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9a0806ee00 msgr2=0x7f9a080712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9a0806ee00 0x7f9a080712c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c074880 tx=0x7f9a0c009260 comp rx=0 tx=0).stop 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 msgr2=0x7f9a1c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c073090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9a18005f50 tx=0x7f9a1800f690 comp rx=0 tx=0).stop 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 shutdown_connections 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9a0806ee00 0x7f9a080712c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a1c100ea0 0x7f9a1c073090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 --2- 172.21.15.31:0/750915515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c101850 0x7f9a1c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 >> 172.21.15.31:0/750915515 conn(0x7f9a1c0fc750 msgr2=0x7f9a1c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 shutdown_connections 2024-04-22T23:50:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.482+0000 7f9a241f1700 1 -- 172.21.15.31:0/750915515 wait complete. 2024-04-22T23:50:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 -- 172.21.15.31:0/755543880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 msgr2=0x7f8454101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 --2- 172.21.15.31:0/755543880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454101280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8444009910 tx=0x7f8444009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 -- 172.21.15.31:0/755543880 shutdown_connections 2024-04-22T23:50:21.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 --2- 172.21.15.31:0/755543880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8454101850 0x7f84541058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 --2- 172.21.15.31:0/755543880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454101280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 -- 172.21.15.31:0/755543880 >> 172.21.15.31:0/755543880 conn(0x7f84540fc750 msgr2=0x7f84540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 -- 172.21.15.31:0/755543880 shutdown_connections 2024-04-22T23:50:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.589+0000 7f84588a5700 1 -- 172.21.15.31:0/755543880 wait complete. 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f84588a5700 1 Processor -- start 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f84588a5700 1 -- start start 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f84588a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f84588a5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8454101850 0x7f84540735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f84588a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454073b10 con 0x7f8454100ea0 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f84588a5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454073c50 con 0x7f8454101850 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f845259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f845259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55494/0 (socket says 172.21.15.31:55494) 2024-04-22T23:50:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.590+0000 7f845259c700 1 -- 172.21.15.31:0/2794758870 learned_addr learned my addr 172.21.15.31:0/2794758870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f8451d9b700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8454101850 0x7f84540735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f845259c700 1 -- 172.21.15.31:0/2794758870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8454101850 msgr2=0x7f84540735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f845259c700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8454101850 0x7f84540735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f845259c700 1 -- 172.21.15.31:0/2794758870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440095f0 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f845259c700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454073090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8444006010 tx=0x7f844400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f844401d070 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8454073ed0 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8444005dc0 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8444017400 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.591+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84541a2e20 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.592+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8454109240 con 0x7f8454100ea0 2024-04-22T23:50:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.593+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f8444003a60 con 0x7f8454100ea0 2024-04-22T23:50:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.593+0000 7f84437fe700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f843c06ee00 0x7f843c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.593+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8444090520 con 0x7f8454100ea0 2024-04-22T23:50:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.593+0000 7f8451d9b700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f843c06ee00 0x7f843c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.594+0000 7f8451d9b700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f843c06ee00 0x7f843c0712c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8454074880 tx=0x7f84480073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:21.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.598+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f8444059110 con 0x7f8454100ea0 2024-04-22T23:50:21.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.898+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8454074680 con 0x7f8454100ea0 2024-04-22T23:50:21.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:21 smithi031 ceph-mon[111289]: pgmap v28: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:21.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:21 smithi031 ceph-mon[111289]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:21.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:21.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:21.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:50:21.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:21.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.899+0000 7f84437fe700 1 -- 172.21.15.31:0/2794758870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f844405c7d0 con 0x7f8454100ea0 2024-04-22T23:50:21.891 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:50:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-04-22T23:50:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:50:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:50:21.893 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f843c06ee00 msgr2=0x7f843c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f843c06ee00 0x7f843c0712c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8454074880 tx=0x7f84480073d0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 msgr2=0x7f8454073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454073090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8444006010 tx=0x7f844400bd70 comp rx=0 tx=0).stop 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 shutdown_connections 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f843c06ee00 0x7f843c0712c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8454100ea0 0x7f8454073090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 --2- 172.21.15.31:0/2794758870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8454101850 0x7f84540735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 >> 172.21.15.31:0/2794758870 conn(0x7f84540fc750 msgr2=0x7f84540fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 shutdown_connections 2024-04-22T23:50:21.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:21.901+0000 7f84588a5700 1 -- 172.21.15.31:0/2794758870 wait complete. 2024-04-22T23:50:21.901 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:21 smithi044 ceph-mon[82146]: pgmap v28: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:21.901 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:21 smithi044 ceph-mon[82146]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:21.901 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:21.901 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:21.901 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:50:21.901 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.008+0000 7f7148a40700 1 -- 172.21.15.31:0/3924867475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 msgr2=0x7f7144101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.008+0000 7f7148a40700 1 --2- 172.21.15.31:0/3924867475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f7144101280 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7138009910 tx=0x7f7138009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.008+0000 7f7148a40700 1 -- 172.21.15.31:0/3924867475 shutdown_connections 2024-04-22T23:50:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.008+0000 7f7148a40700 1 --2- 172.21.15.31:0/3924867475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7144101850 0x7f71441058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.009+0000 7f7148a40700 1 --2- 172.21.15.31:0/3924867475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f7144101280 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.009+0000 7f7148a40700 1 -- 172.21.15.31:0/3924867475 >> 172.21.15.31:0/3924867475 conn(0x7f71440fc750 msgr2=0x7f71440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.009+0000 7f7148a40700 1 -- 172.21.15.31:0/3924867475 shutdown_connections 2024-04-22T23:50:22.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.009+0000 7f7148a40700 1 -- 172.21.15.31:0/3924867475 wait complete. 2024-04-22T23:50:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.009+0000 7f7148a40700 1 Processor -- start 2024-04-22T23:50:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f7148a40700 1 -- start start 2024-04-22T23:50:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f7148a40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f714419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f7148a40700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7144101850 0x7f714419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f7148a40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714419d2c0 con 0x7f7144100ea0 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f7148a40700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7144196770 con 0x7f7144101850 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f714419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f714419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55508/0 (socket says 172.21.15.31:55508) 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 -- 172.21.15.31:0/1473392168 learned_addr learned my addr 172.21.15.31:0/1473392168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f7141d9b700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7144101850 0x7f714419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 -- 172.21.15.31:0/1473392168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7144101850 msgr2=0x7f714419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7144101850 0x7f714419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 -- 172.21.15.31:0/1473392168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380095f0 con 0x7f7144100ea0 2024-04-22T23:50:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.010+0000 7f714259c700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f714419c6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7138006010 tx=0x7f713800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.011+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f713801d070 con 0x7f7144100ea0 2024-04-22T23:50:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.011+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f713800f460 con 0x7f7144100ea0 2024-04-22T23:50:22.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.011+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71441969f0 con 0x7f7144100ea0 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.011+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7138017630 con 0x7f7144100ea0 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.011+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7144196ee0 con 0x7f7144100ea0 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.012+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7144109240 con 0x7f7144100ea0 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.012+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f713800f5d0 con 0x7f7144100ea0 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.012+0000 7f71337fe700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f712c06ee00 0x7f712c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.013+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7138090530 con 0x7f7144100ea0 2024-04-22T23:50:22.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.013+0000 7f7141d9b700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f712c06ee00 0x7f712c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.013+0000 7f7141d9b700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f712c06ee00 0x7f712c0712c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7134005fd0 tx=0x7f7134005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:22.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.017+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f713805cc50 con 0x7f7144100ea0 2024-04-22T23:50:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.280+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f714404f2e0 con 0x7f7144100ea0 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.280+0000 7f71337fe700 1 -- 172.21.15.31:0/1473392168 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f713805c7e0 con 0x7f7144100ea0 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:50:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:50:22.273 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:50:22.275 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.282+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f712c06ee00 msgr2=0x7f712c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f712c06ee00 0x7f712c0712c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7134005fd0 tx=0x7f7134005f60 comp rx=0 tx=0).stop 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 msgr2=0x7f714419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f714419c6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7138006010 tx=0x7f713800be20 comp rx=0 tx=0).stop 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 shutdown_connections 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f712c06ee00 0x7f712c0712c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7144100ea0 0x7f714419c6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 --2- 172.21.15.31:0/1473392168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7144101850 0x7f714419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 >> 172.21.15.31:0/1473392168 conn(0x7f71440fc750 msgr2=0x7f71440fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 shutdown_connections 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.283+0000 7f7148a40700 1 -- 172.21.15.31:0/1473392168 wait complete. 2024-04-22T23:50:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.391+0000 7f23d3d2e700 1 -- 172.21.15.31:0/1133899134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 msgr2=0x7f23cc0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.391+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/1133899134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc0ff980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009910 tx=0x7f23c8009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.391+0000 7f23d3d2e700 1 -- 172.21.15.31:0/1133899134 shutdown_connections 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.391+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/1133899134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23cc0fff50 0x7f23cc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.391+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/1133899134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc0ff980 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.391+0000 7f23d3d2e700 1 -- 172.21.15.31:0/1133899134 >> 172.21.15.31:0/1133899134 conn(0x7f23cc074e30 msgr2=0x7f23cc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 -- 172.21.15.31:0/1133899134 shutdown_connections 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 -- 172.21.15.31:0/1133899134 wait complete. 2024-04-22T23:50:22.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 Processor -- start 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 -- start start 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23cc0fff50 0x7f23cc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc19d2d0 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d3d2e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc196780 con 0x7f23cc0fff50 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d1aca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.392+0000 7f23d1aca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55516/0 (socket says 172.21.15.31:55516) 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d1aca700 1 -- 172.21.15.31:0/3564543613 learned_addr learned my addr 172.21.15.31:0/3564543613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d12c9700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23cc0fff50 0x7f23cc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d1aca700 1 -- 172.21.15.31:0/3564543613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23cc0fff50 msgr2=0x7f23cc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d1aca700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23cc0fff50 0x7f23cc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d1aca700 1 -- 172.21.15.31:0/3564543613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80095f0 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d1aca700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc19c700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c801d070 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23c800f460 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c8017630 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc196a00 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.393+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc196ef0 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.394+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc109270 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.395+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f23c800f5d0 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.395+0000 7f23c2ffd700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ee00 0x7f23b80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.395+0000 7f23d12c9700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ee00 0x7f23b80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.395+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f23c80904b0 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.396+0000 7f23d12c9700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ee00 0x7f23b80712c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005fd0 tx=0x7f23bc005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:22.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.400+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f23c805cc50 con 0x7f23cc0ff5a0 2024-04-22T23:50:22.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.634+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23cc197d50 con 0x7f23b806ee00 2024-04-22T23:50:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.636+0000 7f23c2ffd700 1 -- 172.21.15.31:0/3564543613 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f23cc197d50 con 0x7f23b806ee00 2024-04-22T23:50:22.627 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "4/23 daemons upgraded", 2024-04-22T23:50:22.628 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-04-22T23:50:22.629 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:50:22.629 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:50:22.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ee00 msgr2=0x7f23b80712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ee00 0x7f23b80712c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005fd0 tx=0x7f23bc005f60 comp rx=0 tx=0).stop 2024-04-22T23:50:22.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 msgr2=0x7f23cc19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc19c700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c800be20 comp rx=0 tx=0).stop 2024-04-22T23:50:22.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 shutdown_connections 2024-04-22T23:50:22.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ee00 0x7f23b80712c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23cc0ff5a0 0x7f23cc19c700 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 --2- 172.21.15.31:0/3564543613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23cc0fff50 0x7f23cc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 >> 172.21.15.31:0/3564543613 conn(0x7f23cc074e30 msgr2=0x7f23cc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:22.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 shutdown_connections 2024-04-22T23:50:22.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.638+0000 7f23d3d2e700 1 -- 172.21.15.31:0/3564543613 wait complete. 2024-04-22T23:50:22.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.745+0000 7fe2ab407700 1 -- 172.21.15.31:0/1380490876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4101850 msgr2=0x7fe2a41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.745+0000 7fe2ab407700 1 --2- 172.21.15.31:0/1380490876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4101850 0x7fe2a41058a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe294009910 tx=0x7fe294009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:22.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.745+0000 7fe2ab407700 1 -- 172.21.15.31:0/1380490876 shutdown_connections 2024-04-22T23:50:22.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.745+0000 7fe2ab407700 1 --2- 172.21.15.31:0/1380490876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4101850 0x7fe2a41058a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.745+0000 7fe2ab407700 1 --2- 172.21.15.31:0/1380490876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2a4100ea0 0x7fe2a4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.745+0000 7fe2ab407700 1 -- 172.21.15.31:0/1380490876 >> 172.21.15.31:0/1380490876 conn(0x7fe2a40fc750 msgr2=0x7fe2a40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:22.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.746+0000 7fe2ab407700 1 -- 172.21.15.31:0/1380490876 shutdown_connections 2024-04-22T23:50:22.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.746+0000 7fe2ab407700 1 -- 172.21.15.31:0/1380490876 wait complete. 2024-04-22T23:50:22.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.746+0000 7fe2ab407700 1 Processor -- start 2024-04-22T23:50:22.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.746+0000 7fe2ab407700 1 -- start start 2024-04-22T23:50:22.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2ab407700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 0x7fe2a419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2ab407700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2a4101850 0x7fe2a419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2ab407700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a419d2d0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2ab407700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a4196780 con 0x7fe2a4101850 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 0x7fe2a419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 0x7fe2a419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55536/0 (socket says 172.21.15.31:55536) 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 -- 172.21.15.31:0/522630107 learned_addr learned my addr 172.21.15.31:0/522630107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 -- 172.21.15.31:0/522630107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2a4101850 msgr2=0x7fe2a419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2a4101850 0x7fe2a419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 -- 172.21.15.31:0/522630107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2940095f0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2a91a3700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 0x7fe2a419c700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c8a0 tx=0x7fe2a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a000cea0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2a0004830 con 0x7fe2a4100ea0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a4196a60 con 0x7fe2a4100ea0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a00055b0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.747+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4196fb0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.748+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a41092a0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.749+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fe2a00049a0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.749+0000 7fe29a7fc700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe29006ee00 0x7fe2900712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:22.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.750+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe2a008eca0 con 0x7fe2a4100ea0 2024-04-22T23:50:22.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.750+0000 7fe2a89a2700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe29006ee00 0x7fe2900712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:22.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.750+0000 7fe2a89a2700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe29006ee00 0x7fe2900712c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b5c0 tx=0x7fe294005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:22.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:22.754+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe2a0093070 con 0x7fe2a4100ea0 2024-04-22T23:50:22.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:22 smithi031 ceph-mon[111289]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:22.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:22 smithi031 ceph-mon[111289]: Reconfiguring osd.5 (monmap changed)... 2024-04-22T23:50:22.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:22 smithi031 ceph-mon[111289]: Reconfiguring daemon osd.5 on smithi044 2024-04-22T23:50:22.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:22 smithi031 ceph-mon[111289]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:22.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:22 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2794758870' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:22.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:22 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1473392168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:50:22.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:22 smithi044 ceph-mon[82146]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:22.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:22 smithi044 ceph-mon[82146]: Reconfiguring osd.5 (monmap changed)... 2024-04-22T23:50:22.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:22 smithi044 ceph-mon[82146]: Reconfiguring daemon osd.5 on smithi044 2024-04-22T23:50:22.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:22 smithi044 ceph-mon[82146]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:22.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:22 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2794758870' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:22.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:22 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1473392168' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:50:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.063+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe2a404f2e0 con 0x7fe2a4100ea0 2024-04-22T23:50:23.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.063+0000 7fe29a7fc700 1 -- 172.21.15.31:0/522630107 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe2a001d090 con 0x7fe2a4100ea0 2024-04-22T23:50:23.055 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:50:23.055 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:50:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe29006ee00 msgr2=0x7fe2900712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe29006ee00 0x7fe2900712c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe29400b5c0 tx=0x7fe294005960 comp rx=0 tx=0).stop 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 msgr2=0x7fe2a419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 0x7fe2a419c700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000c8a0 tx=0x7fe2a000cc60 comp rx=0 tx=0).stop 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 shutdown_connections 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe29006ee00 0x7fe2900712c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2a4100ea0 0x7fe2a419c700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 --2- 172.21.15.31:0/522630107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2a4101850 0x7fe2a419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:23.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 >> 172.21.15.31:0/522630107 conn(0x7fe2a40fc750 msgr2=0x7fe2a40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:23.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 shutdown_connections 2024-04-22T23:50:23.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:23.065+0000 7fe2ab407700 1 -- 172.21.15.31:0/522630107 wait complete. 2024-04-22T23:50:23.772 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: pgmap v29: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:23.772 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:23.772 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/522630107' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:50:23.772 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:23.772 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:23.772 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:23.773 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: pgmap v29: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/522630107' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:23.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:24.838 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:24 smithi044 ceph-mon[82146]: Reconfiguring mds.cephfs.smithi044.ibkztg (monmap changed)... 2024-04-22T23:50:24.838 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:24 smithi044 ceph-mon[82146]: Reconfiguring daemon mds.cephfs.smithi044.ibkztg on smithi044 2024-04-22T23:50:24.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:24 smithi031 ceph-mon[111289]: Reconfiguring mds.cephfs.smithi044.ibkztg (monmap changed)... 2024-04-22T23:50:24.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:24 smithi031 ceph-mon[111289]: Reconfiguring daemon mds.cephfs.smithi044.ibkztg on smithi044 2024-04-22T23:50:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:25 smithi031 ceph-mon[111289]: pgmap v30: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:26.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:25 smithi044 ceph-mon[82146]: pgmap v30: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:26.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:50:26.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: Reconfiguring mds.cephfs.smithi044.onzoro (monmap changed)... 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: Reconfiguring daemon mds.cephfs.smithi044.onzoro on smithi044 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:27.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]: dispatch 2024-04-22T23:50:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]': finished 2024-04-22T23:50:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]: dispatch 2024-04-22T23:50:27.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]': finished 2024-04-22T23:50:27.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: Reconfiguring mds.cephfs.smithi044.onzoro (monmap changed)... 2024-04-22T23:50:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: Reconfiguring daemon mds.cephfs.smithi044.onzoro on smithi044 2024-04-22T23:50:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:27.212 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:27.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:27.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]: dispatch 2024-04-22T23:50:27.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]': finished 2024-04-22T23:50:27.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]: dispatch 2024-04-22T23:50:27.213 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi044"}]': finished 2024-04-22T23:50:28.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:27 smithi031 ceph-mon[111289]: pgmap v31: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:28.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:27 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all mon 2024-04-22T23:50:28.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:27 smithi044 ceph-mon[82146]: pgmap v31: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:28.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:27 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all mon 2024-04-22T23:50:29.690 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:29 smithi031 ceph-mon[111289]: pgmap v32: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:29.690 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:29 smithi031 ceph-mon[111289]: Upgrade: Updating crash.smithi031 (1/2) 2024-04-22T23:50:29.690 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:29.690 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:29.690 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:29.690 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:29 smithi031 ceph-mon[111289]: Deploying daemon crash.smithi031 on smithi031 2024-04-22T23:50:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:29 smithi044 ceph-mon[82146]: pgmap v32: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:29 smithi044 ceph-mon[82146]: Upgrade: Updating crash.smithi031 (1/2) 2024-04-22T23:50:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:29.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:29 smithi044 ceph-mon[82146]: Deploying daemon crash.smithi031 on smithi031 2024-04-22T23:50:30.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:30.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:31.423 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:31 smithi031 ceph-mon[111289]: pgmap v33: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:31.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:31 smithi044 ceph-mon[82146]: pgmap v33: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:33 smithi031 ceph-mon[111289]: pgmap v34: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:33 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:33 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:33 smithi044 ceph-mon[82146]: pgmap v34: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:33 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:33 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:34.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:34 smithi044 ceph-mon[82146]: Upgrade: Updating crash.smithi044 (2/2) 2024-04-22T23:50:34.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:34 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:34.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:34 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:34.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:34 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:34.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:34 smithi044 ceph-mon[82146]: Deploying daemon crash.smithi044 on smithi044 2024-04-22T23:50:35.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:34 smithi031 ceph-mon[111289]: Upgrade: Updating crash.smithi044 (2/2) 2024-04-22T23:50:35.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:34 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:35.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:34 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi044", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:50:35.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:34 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:35.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:34 smithi031 ceph-mon[111289]: Deploying daemon crash.smithi044 on smithi044 2024-04-22T23:50:35.951 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:35 smithi044 ceph-mon[82146]: pgmap v35: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:36.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:35 smithi031 ceph-mon[111289]: pgmap v35: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:38.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:37 smithi031 ceph-mon[111289]: pgmap v36: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:38.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:38.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:38.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:37 smithi044 ceph-mon[82146]: pgmap v36: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:37 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:37 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:38.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:37 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:39.990 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:39 smithi044 ceph-mon[82146]: pgmap v37: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:40.117 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:39 smithi031 ceph-mon[111289]: pgmap v37: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:41.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:41 smithi031 ceph-mon[111289]: pgmap v38: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:41.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:41.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:41 smithi044 ceph-mon[82146]: pgmap v38: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:41.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:41.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:42 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:42 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:42 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:42.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:42 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:43.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:43 smithi044 ceph-mon[82146]: pgmap v39: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:44.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:43 smithi031 ceph-mon[111289]: pgmap v39: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:45.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:44 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:45.098 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:44 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:45.119 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:44 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:45.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:44 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:45 smithi031 ceph-mon[111289]: pgmap v40: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:46.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:46.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:45 smithi044 ceph-mon[82146]: pgmap v40: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:50:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:46.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: pgmap v41: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]: dispatch 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]': finished 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]: dispatch 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]': finished 2024-04-22T23:50:48.171 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:50:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: pgmap v41: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:50:48.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]: dispatch 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]': finished 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]: dispatch 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi044"}]': finished 2024-04-22T23:50:48.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:50:49.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:48 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all crash 2024-04-22T23:50:49.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:48 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:50:49.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:48 smithi031 ceph-mon[111289]: Upgrade: osd.0 is safe to restart 2024-04-22T23:50:49.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:48 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all crash 2024-04-22T23:50:49.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:48 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:50:49.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:48 smithi044 ceph-mon[82146]: Upgrade: osd.0 is safe to restart 2024-04-22T23:50:50.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:49 smithi031 ceph-mon[111289]: pgmap v42: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:50.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:50.126 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:50:50.126 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:50.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:49 smithi044 ceph-mon[82146]: pgmap v42: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:50.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:50.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:50:50.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:50:50.987 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:50 smithi031 ceph-mon[111289]: Upgrade: Updating osd.0 2024-04-22T23:50:50.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:50 smithi031 ceph-mon[111289]: Deploying daemon osd.0 on smithi031 2024-04-22T23:50:51.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:50 smithi044 ceph-mon[82146]: Upgrade: Updating osd.0 2024-04-22T23:50:51.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:50 smithi044 ceph-mon[82146]: Deploying daemon osd.0 on smithi031 2024-04-22T23:50:52.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:51 smithi031 ceph-mon[111289]: pgmap v43: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:52.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:51 smithi044 ceph-mon[82146]: pgmap v43: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:50:52.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:52 smithi031 systemd[1]: Stopping Ceph osd.0 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:50:52.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:52 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[45209]: 2024-04-22T23:50:52.497+0000 7fec4ed98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:50:52.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:52 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[45209]: 2024-04-22T23:50:52.497+0000 7fec4ed98700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-22T23:50:52.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:52 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[45209]: 2024-04-22T23:50:52.497+0000 7fec4ed98700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:50:53.111 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:52 smithi044 ceph-mon[82146]: osd.0 marked itself down and dead 2024-04-22T23:50:53.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:52 smithi031 ceph-mon[111289]: osd.0 marked itself down and dead 2024-04-22T23:50:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 -- 172.21.15.31:0/3683671251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 msgr2=0x7f2ff41070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/3683671251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff41070b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400b0a0 tx=0x7f2fe400b3b0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 -- 172.21.15.31:0/3683671251 shutdown_connections 2024-04-22T23:50:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/3683671251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff4068580 0x7f2ff40689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/3683671251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff41070b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 -- 172.21.15.31:0/3683671251 >> 172.21.15.31:0/3683671251 conn(0x7f2ff4075670 msgr2=0x7f2ff4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 -- 172.21.15.31:0/3683671251 shutdown_connections 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 -- 172.21.15.31:0/3683671251 wait complete. 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.190+0000 7f2ff9a66700 1 Processor -- start 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff9a66700 1 -- start start 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff9a66700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff4068580 0x7f2ff419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff9a66700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff9a66700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419d260 con 0x7f2ff4106cd0 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff9a66700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4196710 con 0x7f2ff4068580 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53764/0 (socket says 172.21.15.31:53764) 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 -- 172.21.15.31:0/918250132 learned_addr learned my addr 172.21.15.31:0/918250132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff37fe700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff4068580 0x7f2ff419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 -- 172.21.15.31:0/918250132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff4068580 msgr2=0x7f2ff419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff4068580 0x7f2ff419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 -- 172.21.15.31:0/918250132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4009d00 con 0x7f2ff4106cd0 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.191+0000 7f2ff2ffd700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff419cbd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00d900 tx=0x7f2fdc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.192+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fdc0041f0 con 0x7f2ff4106cd0 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.192+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fdc004cf0 con 0x7f2ff4106cd0 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.192+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff41969f0 con 0x7f2ff4106cd0 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.192+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fdc00b750 con 0x7f2ff4106cd0 2024-04-22T23:50:53.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.192+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff4196f40 con 0x7f2ff4106cd0 2024-04-22T23:50:53.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.193+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff41091d0 con 0x7f2ff4106cd0 2024-04-22T23:50:53.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.193+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f2fdc004750 con 0x7f2ff4106cd0 2024-04-22T23:50:53.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.194+0000 7f2ff0ff9700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2fe006edf0 0x7f2fe00712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.194+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2fdc08ee70 con 0x7f2ff4106cd0 2024-04-22T23:50:53.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.194+0000 7f2ff37fe700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2fe006edf0 0x7f2fe00712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.194+0000 7f2ff37fe700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2fe006edf0 0x7f2fe00712b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4000f80 tx=0x7f2fe40061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:53.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.199+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2fdc05b610 con 0x7f2ff4106cd0 2024-04-22T23:50:53.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.446+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ff41971b0 con 0x7f2fe006edf0 2024-04-22T23:50:53.438 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:53 smithi031 podman[122252]: 2024-04-22 23:50:53.325263402 +0000 UTC m=+0.926042283 container died 247f1998112779a3e24249d65725651d210ff25adf20007a938abf4f0866065a (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-22T23:50:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.448+0000 7f2ff0ff9700 1 -- 172.21.15.31:0/918250132 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2ff41971b0 con 0x7f2fe006edf0 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2fe006edf0 msgr2=0x7f2fe00712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2fe006edf0 0x7f2fe00712b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4000f80 tx=0x7f2fe40061f0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 msgr2=0x7f2ff419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff419cbd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc00d900 tx=0x7f2fdc00dc10 comp rx=0 tx=0).stop 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 shutdown_connections 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2fe006edf0 0x7f2fe00712b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ff4068580 0x7f2ff419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 --2- 172.21.15.31:0/918250132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff4106cd0 0x7f2ff419cbd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 >> 172.21.15.31:0/918250132 conn(0x7f2ff4075670 msgr2=0x7f2ff40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:53.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 shutdown_connections 2024-04-22T23:50:53.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.450+0000 7f2ff9a66700 1 -- 172.21.15.31:0/918250132 wait complete. 2024-04-22T23:50:53.453 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:50:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 -- 172.21.15.31:0/422928261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74100ea0 msgr2=0x7f2b74101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/422928261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74100ea0 0x7f2b74101280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009910 tx=0x7f2b68009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:53.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 -- 172.21.15.31:0/422928261 shutdown_connections 2024-04-22T23:50:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/422928261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b74101850 0x7f2b741058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/422928261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74100ea0 0x7f2b74101280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 -- 172.21.15.31:0/422928261 >> 172.21.15.31:0/422928261 conn(0x7f2b740fc750 msgr2=0x7f2b740feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 -- 172.21.15.31:0/422928261 shutdown_connections 2024-04-22T23:50:53.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 -- 172.21.15.31:0/422928261 wait complete. 2024-04-22T23:50:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 Processor -- start 2024-04-22T23:50:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.561+0000 7f2b7ab78700 1 -- start start 2024-04-22T23:50:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b7ab78700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b74100ea0 0x7f2b74073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b7ab78700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 0x7f2b740735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b7ab78700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b74073b10 con 0x7f2b74101850 2024-04-22T23:50:53.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b7ab78700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b74073c50 con 0x7f2b74100ea0 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 0x7f2b740735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 0x7f2b740735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53780/0 (socket says 172.21.15.31:53780) 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 -- 172.21.15.31:0/4099182283 learned_addr learned my addr 172.21.15.31:0/4099182283 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b78914700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b74100ea0 0x7f2b74073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 -- 172.21.15.31:0/4099182283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b74100ea0 msgr2=0x7f2b74073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b74100ea0 0x7f2b74073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 -- 172.21.15.31:0/4099182283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b680095f0 con 0x7f2b74101850 2024-04-22T23:50:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b73fff700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 0x7f2b740735d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e970 tx=0x7f2b6000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6000c9d0 con 0x7f2b74101850 2024-04-22T23:50:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b741a2b10 con 0x7f2b74101850 2024-04-22T23:50:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b60004830 con 0x7f2b74101850 2024-04-22T23:50:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.562+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b60005660 con 0x7f2b74101850 2024-04-22T23:50:53.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.563+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b741a2f20 con 0x7f2b74101850 2024-04-22T23:50:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.564+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b74109240 con 0x7f2b74101850 2024-04-22T23:50:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.564+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f2b6000cb30 con 0x7f2b74101850 2024-04-22T23:50:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.565+0000 7f2b71ffb700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2b6406edf0 0x7f2b640712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.565+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2b60014070 con 0x7f2b74101850 2024-04-22T23:50:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.565+0000 7f2b78914700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2b6406edf0 0x7f2b640712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.565+0000 7f2b78914700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2b6406edf0 0x7f2b640712b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800b760 tx=0x7f2b68005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.570+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2b60057f80 con 0x7f2b74101850 2024-04-22T23:50:53.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.824+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b740611d0 con 0x7f2b6406edf0 2024-04-22T23:50:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.826+0000 7f2b71ffb700 1 -- 172.21.15.31:0/4099182283 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2b740611d0 con 0x7f2b6406edf0 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2b6406edf0 msgr2=0x7f2b640712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2b6406edf0 0x7f2b640712b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800b760 tx=0x7f2b68005a90 comp rx=0 tx=0).stop 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 msgr2=0x7f2b740735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 0x7f2b740735d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000e970 tx=0x7f2b6000ec80 comp rx=0 tx=0).stop 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 shutdown_connections 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2b6406edf0 0x7f2b640712b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b74100ea0 0x7f2b74073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 --2- 172.21.15.31:0/4099182283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b74101850 0x7f2b740735d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 >> 172.21.15.31:0/4099182283 conn(0x7f2b740fc750 msgr2=0x7f2b740fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 shutdown_connections 2024-04-22T23:50:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.828+0000 7f2b7ab78700 1 -- 172.21.15.31:0/4099182283 wait complete. 2024-04-22T23:50:53.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.943+0000 7f013de3b700 1 -- 172.21.15.31:0/380918919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 msgr2=0x7f01380ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.943+0000 7f013de3b700 1 --2- 172.21.15.31:0/380918919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f01380ffdf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0128009910 tx=0x7f0128009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.943+0000 7f013de3b700 1 -- 172.21.15.31:0/380918919 shutdown_connections 2024-04-22T23:50:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.943+0000 7f013de3b700 1 --2- 172.21.15.31:0/380918919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0138100330 0x7f013810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.943+0000 7f013de3b700 1 --2- 172.21.15.31:0/380918919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f01380ffdf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.943+0000 7f013de3b700 1 -- 172.21.15.31:0/380918919 >> 172.21.15.31:0/380918919 conn(0x7f01380fb650 msgr2=0x7f01380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.944+0000 7f013de3b700 1 -- 172.21.15.31:0/380918919 shutdown_connections 2024-04-22T23:50:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.944+0000 7f013de3b700 1 -- 172.21.15.31:0/380918919 wait complete. 2024-04-22T23:50:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.944+0000 7f013de3b700 1 Processor -- start 2024-04-22T23:50:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.944+0000 7f013de3b700 1 -- start start 2024-04-22T23:50:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.944+0000 7f013de3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f013819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.944+0000 7f013de3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0138100330 0x7f013819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f013de3b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013819f440 con 0x7f01380ffa10 2024-04-22T23:50:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f013de3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381988f0 con 0x7f0138100330 2024-04-22T23:50:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f013819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f013819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53784/0 (socket says 172.21.15.31:53784) 2024-04-22T23:50:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 -- 172.21.15.31:0/4268224325 learned_addr learned my addr 172.21.15.31:0/4268224325 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f0136ffd700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0138100330 0x7f013819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 -- 172.21.15.31:0/4268224325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0138100330 msgr2=0x7f013819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0138100330 0x7f013819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 -- 172.21.15.31:0/4268224325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01280095f0 con 0x7f01380ffa10 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f01377fe700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f013819e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f012800c010 tx=0x7f012800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012801d070 con 0x7f01380ffa10 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0138198b70 con 0x7f01380ffa10 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01280052a0 con 0x7f01380ffa10 2024-04-22T23:50:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0128022470 con 0x7f01380ffa10 2024-04-22T23:50:53.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.945+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0138199060 con 0x7f01380ffa10 2024-04-22T23:50:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.947+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013810a7f0 con 0x7f01380ffa10 2024-04-22T23:50:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.947+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f012800f770 con 0x7f01380ffa10 2024-04-22T23:50:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.947+0000 7f0134ff9700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f012006ee00 0x7f01200712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.948+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f01280904b0 con 0x7f01380ffa10 2024-04-22T23:50:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.948+0000 7f0136ffd700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f012006ee00 0x7f01200712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.948+0000 7f0136ffd700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f012006ee00 0x7f01200712c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01380fbe00 tx=0x7f012c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:53.953+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f012805cbd0 con 0x7f01380ffa10 2024-04-22T23:50:53.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:53 smithi031 ceph-mon[111289]: pgmap v44: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:53.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:53 smithi031 ceph-mon[111289]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:50:53.949 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:53 smithi031 ceph-mon[111289]: osdmap e50: 6 total, 5 up, 6 in 2024-04-22T23:50:53.949 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:53 smithi031 podman[122252]: 2024-04-22 23:50:53.706407777 +0000 UTC m=+1.307186657 container cleanup 247f1998112779a3e24249d65725651d210ff25adf20007a938abf4f0866065a (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-22T23:50:53.949 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:53 smithi031 bash[122252]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0 2024-04-22T23:50:54.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.200+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0138199280 con 0x7f012006ee00 2024-04-22T23:50:54.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.213+0000 7f0134ff9700 1 -- 172.21.15.31:0/4268224325 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0138199280 con 0x7f012006ee00 2024-04-22T23:50:54.205 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:53 smithi031 podman[122292]: 2024-04-22 23:50:53.958027129 +0000 UTC m=+0.627338700 container remove 247f1998112779a3e24249d65725651d210ff25adf20007a938abf4f0866065a (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222) 2024-04-22T23:50:54.205 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:50:54.205 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 11s ago 13m 25.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:50:54.205 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (13m) 11s ago 13m 8102k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:50:54.205 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (8m) 13s ago 12m 7964k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:50:54.205 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (22s) 11s ago 13m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (17s) 13s ago 12m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (2m) 11s ago 13m 75.9M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (8m) 11s ago 8m 16.2M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (8m) 11s ago 8m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (8m) 13s ago 8m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (8m) 13s ago 8m 20.0M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (5m) 11s ago 15m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (4m) 13s ago 12m 450M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (93s) 11s ago 15m 43.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (67s) 13s ago 12m 34.9M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (4m) 11s ago 13m 15.5M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (3m) 13s ago 12m 15.8M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (11m) 11s ago 11m 198M 1217M 18.2.1 5be31c24972a 247f19981127 2024-04-22T23:50:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (11m) 11s ago 11m 208M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (11m) 11s ago 11m 176M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (10m) 13s ago 10m 237M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (10m) 13s ago 10m 211M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (9m) 13s ago 9m 196M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 11s ago 13m 71.6M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f012006ee00 msgr2=0x7f01200712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f012006ee00 0x7f01200712c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01380fbe00 tx=0x7f012c009260 comp rx=0 tx=0).stop 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 msgr2=0x7f013819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f013819e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f012800c010 tx=0x7f012800b5b0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 shutdown_connections 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f012006ee00 0x7f01200712c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01380ffa10 0x7f013819e870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 --2- 172.21.15.31:0/4268224325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0138100330 0x7f013819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 >> 172.21.15.31:0/4268224325 conn(0x7f01380fb650 msgr2=0x7f01380fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 shutdown_connections 2024-04-22T23:50:54.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.216+0000 7f013de3b700 1 -- 172.21.15.31:0/4268224325 wait complete. 2024-04-22T23:50:54.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:53 smithi044 ceph-mon[82146]: pgmap v44: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:50:54.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:53 smithi044 ceph-mon[82146]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:50:54.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:53 smithi044 ceph-mon[82146]: osdmap e50: 6 total, 5 up, 6 in 2024-04-22T23:50:54.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.324+0000 7f39cc8c9700 1 -- 172.21.15.31:0/650656386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 msgr2=0x7f39c8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.324+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/650656386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c8100180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009870 tx=0x7f39b8009b80 comp rx=0 tx=0).stop 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 -- 172.21.15.31:0/650656386 shutdown_connections 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/650656386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c8100750 0x7f39c81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/650656386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c8100180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 -- 172.21.15.31:0/650656386 >> 172.21.15.31:0/650656386 conn(0x7f39c80fb650 msgr2=0x7f39c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 -- 172.21.15.31:0/650656386 shutdown_connections 2024-04-22T23:50:54.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 -- 172.21.15.31:0/650656386 wait complete. 2024-04-22T23:50:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 Processor -- start 2024-04-22T23:50:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.325+0000 7f39cc8c9700 1 -- start start 2024-04-22T23:50:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39cc8c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39cc8c9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c8100750 0x7f39c819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39cc8c9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c819f4d0 con 0x7f39c80ffda0 2024-04-22T23:50:54.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39cc8c9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8198980 con 0x7f39c8100750 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53800/0 (socket says 172.21.15.31:53800) 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 -- 172.21.15.31:0/165791321 learned_addr learned my addr 172.21.15.31:0/165791321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c5d9b700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c8100750 0x7f39c819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 -- 172.21.15.31:0/165791321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c8100750 msgr2=0x7f39c819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c8100750 0x7f39c819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 -- 172.21.15.31:0/165791321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b8009520 con 0x7f39c80ffda0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39c659c700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c819e900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009230 tx=0x7f39b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.326+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39b801c070 con 0x7f39c80ffda0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.327+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39b800fbf0 con 0x7f39c80ffda0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.327+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c8198c00 con 0x7f39c80ffda0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.327+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39b8022dc0 con 0x7f39c80ffda0 2024-04-22T23:50:54.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.327+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c81990f0 con 0x7f39c80ffda0 2024-04-22T23:50:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.328+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c804f2e0 con 0x7f39c80ffda0 2024-04-22T23:50:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.328+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f39b8020070 con 0x7f39c80ffda0 2024-04-22T23:50:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.329+0000 7f39b77fe700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f39b006ee00 0x7f39b00712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.329+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f39b80901c0 con 0x7f39c80ffda0 2024-04-22T23:50:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.329+0000 7f39c5d9b700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f39b006ee00 0x7f39b00712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:54.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.329+0000 7f39c5d9b700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f39b006ee00 0x7f39b00712c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f39c80fbf10 tx=0x7f39bc009310 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:54.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.334+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f39b805c9a0 con 0x7f39c80ffda0 2024-04-22T23:50:54.463 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:54 smithi031 podman[122434]: 2024-04-22 23:50:54.211229167 +0000 UTC m=+0.019324794 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:50:54.463 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:54 smithi031 podman[122434]: 2024-04-22 23:50:54.309341925 +0000 UTC m=+0.117437532 container create 8cd9d37547c3675bf8c1ecc378f49f08ffd7160160706e6599a4b9eb4bf3831c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-22T23:50:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.633+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f39c8199360 con 0x7f39c80ffda0 2024-04-22T23:50:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.633+0000 7f39b77fe700 1 -- 172.21.15.31:0/165791321 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7f39b805c530 con 0x7f39c80ffda0 2024-04-22T23:50:54.625 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:50:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:50:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:50:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:50:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:50:54.627 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:50:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f39b006ee00 msgr2=0x7f39b00712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f39b006ee00 0x7f39b00712c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f39c80fbf10 tx=0x7f39bc009310 comp rx=0 tx=0).stop 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 msgr2=0x7f39c819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c819e900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009230 tx=0x7f39b80047b0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 shutdown_connections 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f39b006ee00 0x7f39b00712c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c80ffda0 0x7f39c819e900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 --2- 172.21.15.31:0/165791321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39c8100750 0x7f39c819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 >> 172.21.15.31:0/165791321 conn(0x7f39c80fb650 msgr2=0x7f39c8069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 shutdown_connections 2024-04-22T23:50:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.636+0000 7f39cc8c9700 1 -- 172.21.15.31:0/165791321 wait complete. 2024-04-22T23:50:54.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 -- 172.21.15.31:0/2743321737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80ffc20 msgr2=0x7fa9f8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 --2- 172.21.15.31:0/2743321737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80ffc20 0x7fa9f8100080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009910 tx=0x7fa9e8009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:54.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 -- 172.21.15.31:0/2743321737 shutdown_connections 2024-04-22T23:50:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 --2- 172.21.15.31:0/2743321737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f80ffc20 0x7fa9f8100080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 --2- 172.21.15.31:0/2743321737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 -- 172.21.15.31:0/2743321737 >> 172.21.15.31:0/2743321737 conn(0x7fa9f8075670 msgr2=0x7fa9f8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:54.740 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:54 smithi031 podman[122434]: 2024-04-22 23:50:54.562168744 +0000 UTC m=+0.370264357 container init 8cd9d37547c3675bf8c1ecc378f49f08ffd7160160706e6599a4b9eb4bf3831c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T23:50:54.740 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:54 smithi031 podman[122434]: 2024-04-22 23:50:54.566687609 +0000 UTC m=+0.374783216 container start 8cd9d37547c3675bf8c1ecc378f49f08ffd7160160706e6599a4b9eb4bf3831c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:50:54.740 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:54 smithi031 podman[122434]: 2024-04-22 23:50:54.662098066 +0000 UTC m=+0.470193678 container attach 8cd9d37547c3675bf8c1ecc378f49f08ffd7160160706e6599a4b9eb4bf3831c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-22T23:50:54.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 -- 172.21.15.31:0/2743321737 shutdown_connections 2024-04-22T23:50:54.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.746+0000 7faa007a3700 1 -- 172.21.15.31:0/2743321737 wait complete. 2024-04-22T23:50:54.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7faa007a3700 1 Processor -- start 2024-04-22T23:50:54.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7faa007a3700 1 -- start start 2024-04-22T23:50:54.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7faa007a3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80ffc20 0x7fa9f8197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7faa007a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7faa007a3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81982f0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7faa007a3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8198460 con 0x7fa9f80ffc20 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7fa9fdd3e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7fa9fdd3e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53814/0 (socket says 172.21.15.31:53814) 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7fa9fdd3e700 1 -- 172.21.15.31:0/1414901095 learned_addr learned my addr 172.21.15.31:0/1414901095 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7fa9fe53f700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80ffc20 0x7fa9f8197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7fa9fdd3e700 1 -- 172.21.15.31:0/1414901095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80ffc20 msgr2=0x7fa9f8197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.747+0000 7fa9fdd3e700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80ffc20 0x7fa9f8197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7fa9fdd3e700 1 -- 172.21.15.31:0/1414901095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80095f0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7fa9fdd3e700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8197ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e801d070 con 0x7fa9f8105bd0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f81a07d0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9e800f460 con 0x7fa9f8105bd0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e8017630 con 0x7fa9f8105bd0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.748+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f81a0be0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.749+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f81080d0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.750+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7fa9e800f5d0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.750+0000 7fa9ef7fe700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa9e406ee00 0x7fa9e40712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.751+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa9e80905d0 con 0x7fa9f8105bd0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.751+0000 7fa9fe53f700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa9e406ee00 0x7fa9e40712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.751+0000 7fa9fe53f700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa9e406ee00 0x7fa9e40712c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400ac50 tx=0x7fa9f400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:54.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:54.759+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa9e80591c0 con 0x7fa9f8105bd0 2024-04-22T23:50:55.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.020+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa9f8107ab0 con 0x7fa9f8105bd0 2024-04-22T23:50:55.012 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:54 smithi031 podman[122434]: 2024-04-22 23:50:54.887528705 +0000 UTC m=+0.695624320 container died 8cd9d37547c3675bf8c1ecc378f49f08ffd7160160706e6599a4b9eb4bf3831c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:50:55.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:54 smithi031 ceph-mon[111289]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:55.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:54 smithi031 ceph-mon[111289]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:55.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:54 smithi031 ceph-mon[111289]: osdmap e51: 6 total, 5 up, 6 in 2024-04-22T23:50:55.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:54 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/165791321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:55.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.021+0000 7fa9ef7fe700 1 -- 172.21.15.31:0/1414901095 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7fa9e805c880 con 0x7fa9f8105bd0 2024-04-22T23:50:55.013 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:50:55.013 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:50:55.013 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:50:55.014 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:50:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa9e406ee00 msgr2=0x7fa9e40712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa9e406ee00 0x7fa9e40712c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400ac50 tx=0x7fa9f400a380 comp rx=0 tx=0).stop 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 msgr2=0x7fa9f8197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8197ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e800bd70 comp rx=0 tx=0).stop 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 shutdown_connections 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa9e406ee00 0x7fa9e40712c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9f80ffc20 0x7fa9f8197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 --2- 172.21.15.31:0/1414901095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f8105bd0 0x7fa9f8197ca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 >> 172.21.15.31:0/1414901095 conn(0x7fa9f8075670 msgr2=0x7fa9f80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 shutdown_connections 2024-04-22T23:50:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.023+0000 7faa007a3700 1 -- 172.21.15.31:0/1414901095 wait complete. 2024-04-22T23:50:55.018 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 -- 172.21.15.31:0/222465150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 msgr2=0x7f0564100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 --2- 172.21.15.31:0/222465150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f0564100ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0558009910 tx=0x7f0558009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 -- 172.21.15.31:0/222465150 shutdown_connections 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 --2- 172.21.15.31:0/222465150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f0564100ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 --2- 172.21.15.31:0/222465150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0564106a50 0x7f0564106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 -- 172.21.15.31:0/222465150 >> 172.21.15.31:0/222465150 conn(0x7f05640fc560 msgr2=0x7f05640fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 -- 172.21.15.31:0/222465150 shutdown_connections 2024-04-22T23:50:55.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.132+0000 7f056af9d700 1 -- 172.21.15.31:0/222465150 wait complete. 2024-04-22T23:50:55.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056af9d700 1 Processor -- start 2024-04-22T23:50:55.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056af9d700 1 -- start start 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056af9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f056419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056af9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0564106a50 0x7f056419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056af9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056419d0e0 con 0x7f0564100a50 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056af9d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0564196590 con 0x7f0564106a50 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f0569f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f056419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f0569f9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f056419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53844/0 (socket says 172.21.15.31:53844) 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f0569f9b700 1 -- 172.21.15.31:0/3360575238 learned_addr learned my addr 172.21.15.31:0/3360575238 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:55.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f056979a700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0564106a50 0x7f056419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f0569f9b700 1 -- 172.21.15.31:0/3360575238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0564106a50 msgr2=0x7f056419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f0569f9b700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0564106a50 0x7f056419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.133+0000 7f0569f9b700 1 -- 172.21.15.31:0/3360575238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05580095f0 con 0x7f0564100a50 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.134+0000 7f0569f9b700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f056419c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ba50 tx=0x7f055c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.134+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055c00ca00 con 0x7f0564100a50 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.134+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f055c00cb60 con 0x7f0564100a50 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.134+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0564196870 con 0x7f0564100a50 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.134+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055c0075a0 con 0x7f0564100a50 2024-04-22T23:50:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.134+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0564196dc0 con 0x7f0564100a50 2024-04-22T23:50:55.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.135+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0564109070 con 0x7f0564100a50 2024-04-22T23:50:55.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.135+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f055c00ccd0 con 0x7f0564100a50 2024-04-22T23:50:55.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.136+0000 7f0562ffd700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f054c06ee00 0x7f054c0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:55.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.136+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f055c006e40 con 0x7f0564100a50 2024-04-22T23:50:55.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.136+0000 7f056979a700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f054c06ee00 0x7f054c0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.136+0000 7f056979a700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f054c06ee00 0x7f054c0712c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05580059d0 tx=0x7f055801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:55.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.141+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f055c05afe0 con 0x7f0564100a50 2024-04-22T23:50:55.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:54 smithi044 ceph-mon[82146]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:55.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:54 smithi044 ceph-mon[82146]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:55.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:54 smithi044 ceph-mon[82146]: osdmap e51: 6 total, 5 up, 6 in 2024-04-22T23:50:55.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:54 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/165791321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:50:55.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.372+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0564197040 con 0x7f054c06ee00 2024-04-22T23:50:55.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.373+0000 7f0562ffd700 1 -- 172.21.15.31:0/3360575238 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0564197040 con 0x7f054c06ee00 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "crash" 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "6/23 daemons upgraded", 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:50:55.365 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f054c06ee00 msgr2=0x7f054c0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f054c06ee00 0x7f054c0712c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05580059d0 tx=0x7f055801a040 comp rx=0 tx=0).stop 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 msgr2=0x7f056419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f056419c510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ba50 tx=0x7f055c00be10 comp rx=0 tx=0).stop 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 shutdown_connections 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f054c06ee00 0x7f054c0712c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0564100a50 0x7f056419c510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 --2- 172.21.15.31:0/3360575238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0564106a50 0x7f056419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.375+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 >> 172.21.15.31:0/3360575238 conn(0x7f05640fc560 msgr2=0x7f05640fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.376+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 shutdown_connections 2024-04-22T23:50:55.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.376+0000 7f056af9d700 1 -- 172.21.15.31:0/3360575238 wait complete. 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 -- 172.21.15.31:0/403497434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 msgr2=0x7f012c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 --2- 172.21.15.31:0/403497434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c1021e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0120009910 tx=0x7f0120009c20 comp rx=0 tx=0).stop 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 -- 172.21.15.31:0/403497434 shutdown_connections 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 --2- 172.21.15.31:0/403497434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f012c102720 0x7f012c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 --2- 172.21.15.31:0/403497434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c1021e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 -- 172.21.15.31:0/403497434 >> 172.21.15.31:0/403497434 conn(0x7f012c0fb650 msgr2=0x7f012c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 -- 172.21.15.31:0/403497434 shutdown_connections 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 -- 172.21.15.31:0/403497434 wait complete. 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.484+0000 7f01318ad700 1 Processor -- start 2024-04-22T23:50:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f01318ad700 1 -- start start 2024-04-22T23:50:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f01318ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f01318ad700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f012c102720 0x7f012c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f01318ad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c19d260 con 0x7f012c101e00 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f01318ad700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c196710 con 0x7f012c102720 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53864/0 (socket says 172.21.15.31:53864) 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 -- 172.21.15.31:0/142562710 learned_addr learned my addr 172.21.15.31:0/142562710 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012a7fc700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f012c102720 0x7f012c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 -- 172.21.15.31:0/142562710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f012c102720 msgr2=0x7f012c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f012c102720 0x7f012c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 -- 172.21.15.31:0/142562710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01200095f0 con 0x7f012c101e00 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.485+0000 7f012affd700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c19c690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f012000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.486+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012001d070 con 0x7f012c101e00 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.486+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c196990 con 0x7f012c101e00 2024-04-22T23:50:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.486+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0120005dc0 con 0x7f012c101e00 2024-04-22T23:50:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.486+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0120017400 con 0x7f012c101e00 2024-04-22T23:50:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.486+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c196e80 con 0x7f012c101e00 2024-04-22T23:50:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.487+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012c108310 con 0x7f012c101e00 2024-04-22T23:50:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.487+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 92847+0+0 (secure 0 0 0) 0x7f0120003a60 con 0x7f012c101e00 2024-04-22T23:50:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.488+0000 7f01308ab700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f011406ee00 0x7f01140712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:50:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.488+0000 7f012a7fc700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f011406ee00 0x7f01140712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:50:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.488+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0120090510 con 0x7f012c101e00 2024-04-22T23:50:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.488+0000 7f012a7fc700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f011406ee00 0x7f01140712c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f012c197b80 tx=0x7f011c007400 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:50:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.493+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f012005cc30 con 0x7f012c101e00 2024-04-22T23:50:55.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:55 smithi031 podman[122506]: 2024-04-22 23:50:55.583368341 +0000 UTC m=+0.687058016 container remove 8cd9d37547c3675bf8c1ecc378f49f08ffd7160160706e6599a4b9eb4bf3831c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD) 2024-04-22T23:50:55.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:55 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-22T23:50:55.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:55 smithi031 systemd[1]: Stopped Ceph osd.0 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:50:55.694 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:55 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.0.service: Consumed 53.226s CPU time. 2024-04-22T23:50:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.804+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f012c197d20 con 0x7f012c101e00 2024-04-22T23:50:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.804+0000 7f01308ab700 1 -- 172.21.15.31:0/142562710 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+290 (secure 0 0 0) 0x7f012c197d20 con 0x7f012c101e00 2024-04-22T23:50:55.797 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-04-22T23:50:55.797 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:50:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:50:55.797 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-22T23:50:55.797 INFO:teuthology.orchestra.run.smithi031.stdout: osd.0 (root=default,host=smithi031) is down 2024-04-22T23:50:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f011406ee00 msgr2=0x7f01140712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f011406ee00 0x7f01140712c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f012c197b80 tx=0x7f011c007400 comp rx=0 tx=0).stop 2024-04-22T23:50:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 msgr2=0x7f012c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:50:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c19c690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f012000bd70 comp rx=0 tx=0).stop 2024-04-22T23:50:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 shutdown_connections 2024-04-22T23:50:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f011406ee00 0x7f01140712c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012c101e00 0x7f012c19c690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 --2- 172.21.15.31:0/142562710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f012c102720 0x7f012c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:50:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 >> 172.21.15.31:0/142562710 conn(0x7f012c0fb650 msgr2=0x7f012c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:50:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 shutdown_connections 2024-04-22T23:50:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:50:55.807+0000 7f01318ad700 1 -- 172.21.15.31:0/142562710 wait complete. 2024-04-22T23:50:56.059 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:55 smithi031 systemd[1]: Starting Ceph osd.0 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:50:56.059 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 podman[122658]: 2024-04-22 23:50:55.944860458 +0000 UTC m=+0.018510128 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:50:56.059 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:55 smithi031 ceph-mon[111289]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-22T23:50:56.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:55 smithi031 ceph-mon[111289]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:56.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:55 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1414901095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:50:56.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:55 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/142562710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:50:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:55 smithi044 ceph-mon[82146]: pgmap v47: 65 pgs: 9 stale+active+clean, 56 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-22T23:50:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:55 smithi044 ceph-mon[82146]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:55 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1414901095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:50:56.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:55 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/142562710' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:50:56.395 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 podman[122658]: 2024-04-22 23:50:56.067980676 +0000 UTC m=+0.141630331 container create 900820be37d7662be4093657eb89cfde22aa63c486b2012802a4bb5315c58e94 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-22T23:50:56.395 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 podman[122658]: 2024-04-22 23:50:56.312978902 +0000 UTC m=+0.386628567 container init 900820be37d7662be4093657eb89cfde22aa63c486b2012802a4bb5315c58e94 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5) 2024-04-22T23:50:56.395 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 podman[122658]: 2024-04-22 23:50:56.316414341 +0000 UTC m=+0.390064006 container start 900820be37d7662be4093657eb89cfde22aa63c486b2012802a4bb5315c58e94 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-22T23:50:56.395 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 podman[122658]: 2024-04-22 23:50:56.404220655 +0000 UTC m=+0.477870311 container attach 900820be37d7662be4093657eb89cfde22aa63c486b2012802a4bb5315c58e94 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-22T23:50:57.116 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-mon[111289]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:57.116 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:50:57.116 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:50:57.116 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:50:57.116 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:50:57.116 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:50:57.116 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 bash[122658]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate[122673]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-22T23:50:57.117 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:56 smithi031 podman[122658]: 2024-04-22 23:50:56.837754657 +0000 UTC m=+0.911404307 container died 900820be37d7662be4093657eb89cfde22aa63c486b2012802a4bb5315c58e94 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T23:50:57.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:56 smithi044 ceph-mon[82146]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:50:57.693 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:57 smithi031 podman[122819]: 2024-04-22 23:50:57.525793803 +0000 UTC m=+0.681816214 container remove 900820be37d7662be4093657eb89cfde22aa63c486b2012802a4bb5315c58e94 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T23:50:58.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:57 smithi031 ceph-mon[111289]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-22T23:50:58.061 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:57 smithi031 ceph-mon[111289]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-04-22T23:50:58.061 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:57 smithi031 podman[122884]: 2024-04-22 23:50:57.743260018 +0000 UTC m=+0.108210457 container create 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-22T23:50:58.062 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:57 smithi031 podman[122884]: 2024-04-22 23:50:57.654110658 +0000 UTC m=+0.019061082 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:50:58.062 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:57 smithi031 podman[122884]: 2024-04-22 23:50:57.945921903 +0000 UTC m=+0.310872306 container init 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:50:58.062 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:57 smithi031 podman[122884]: 2024-04-22 23:50:57.949252825 +0000 UTC m=+0.314203231 container start 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-22T23:50:58.062 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:57 smithi031 ceph-osd[122903]: -- 172.21.15.31:0/3552515751 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3cec15680 con 0x55b3cebdf000 2024-04-22T23:50:58.062 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:58 smithi031 bash[122884]: 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f 2024-04-22T23:50:58.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:57 smithi044 ceph-mon[82146]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-22T23:50:58.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:57 smithi044 ceph-mon[82146]: Health check failed: Degraded data redundancy: 50/264 objects degraded (18.939%), 19 pgs degraded (PG_DEGRADED) 2024-04-22T23:50:58.443 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:58 smithi031 systemd[1]: Started Ceph osd.0 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:50:58.944 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:50:58 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-22T23:50:58.494+0000 7f14ea6e17c0 -1 Falling back to public interface 2024-04-22T23:50:59.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:59 smithi031 ceph-mon[111289]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-22T23:50:59.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:59.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:59.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:50:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:50:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:59 smithi044 ceph-mon[82146]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-22T23:50:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:50:59.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:50:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:01 smithi031 ceph-mon[111289]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-22T23:51:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:01 smithi044 ceph-mon[82146]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 50/264 objects degraded (18.939%) 2024-04-22T23:51:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:03.150 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:51:02 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-22T23:51:02.949+0000 7f14ea6e17c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-22T23:51:03.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:03 smithi031 ceph-mon[111289]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 331 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:03 smithi031 ceph-mon[111289]: mgrmap e38: smithi031.fazdcv(active, since 92s), standbys: smithi044.iglcti 2024-04-22T23:51:03.444 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:51:03 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-22T23:51:03.271+0000 7f14ea6e17c0 -1 osd.0 49 log_to_monitors true 2024-04-22T23:51:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:03 smithi044 ceph-mon[82146]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 331 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:03 smithi044 ceph-mon[82146]: mgrmap e38: smithi031.fazdcv(active, since 92s), standbys: smithi044.iglcti 2024-04-22T23:51:04.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:04 smithi044 ceph-mon[82146]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:51:04.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:04 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:04.435 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:04 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:04.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:04 smithi031 ceph-mon[111289]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:51:04.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:04 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:04.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:04 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:05.386 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-22T23:51:05.127+0000 7f14d7b31700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 299 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: osdmap e52: 6 total, 5 up, 6 in 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:05.387 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:05 smithi031 ceph-mon[111289]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 299 B/s wr, 0 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: osdmap e52: 6 total, 5 up, 6 in 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:05 smithi044 ceph-mon[82146]: from='osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089]' entity='osd.0' 2024-04-22T23:51:06.426 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:06 smithi031 ceph-mon[111289]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:51:06.426 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:06 smithi031 ceph-mon[111289]: osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089] boot 2024-04-22T23:51:06.426 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:06 smithi031 ceph-mon[111289]: osdmap e53: 6 total, 6 up, 6 in 2024-04-22T23:51:06.426 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:51:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:06 smithi044 ceph-mon[82146]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:51:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:06 smithi044 ceph-mon[82146]: osd.0 [v2:172.21.15.31:6802/3402089089,v1:172.21.15.31:6803/3402089089] boot 2024-04-22T23:51:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:06 smithi044 ceph-mon[82146]: osdmap e53: 6 total, 6 up, 6 in 2024-04-22T23:51:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:51:07.441 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:07 smithi031 ceph-mon[111289]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:07.442 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:07 smithi031 ceph-mon[111289]: osdmap e54: 6 total, 6 up, 6 in 2024-04-22T23:51:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:07 smithi044 ceph-mon[82146]: pgmap v54: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:07 smithi044 ceph-mon[82146]: osdmap e54: 6 total, 6 up, 6 in 2024-04-22T23:51:09.680 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:09.681 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:09 smithi044 ceph-mon[82146]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-22T23:51:09.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: pgmap v57: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 50/264 objects degraded (18.939%) 2024-04-22T23:51:09.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:09.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:09 smithi031 ceph-mon[111289]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-22T23:51:10.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:10 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 43/264 objects degraded (16.288%), 16 pgs degraded (PG_DEGRADED) 2024-04-22T23:51:10.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:10 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 43/264 objects degraded (16.288%), 16 pgs degraded (PG_DEGRADED) 2024-04-22T23:51:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:11 smithi031 ceph-mon[111289]: pgmap v58: 65 pgs: 16 active+undersized+degraded, 9 active+undersized, 40 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 43/264 objects degraded (16.288%); 46 B/s, 0 objects/s recovering 2024-04-22T23:51:11.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:11 smithi044 ceph-mon[82146]: pgmap v58: 65 pgs: 16 active+undersized+degraded, 9 active+undersized, 40 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 0 op/s; 43/264 objects degraded (16.288%); 46 B/s, 0 objects/s recovering 2024-04-22T23:51:12.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:12 smithi031 ceph-mon[111289]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/264 objects degraded (16.288%), 16 pgs degraded) 2024-04-22T23:51:12.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:12 smithi044 ceph-mon[82146]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 43/264 objects degraded (16.288%), 16 pgs degraded) 2024-04-22T23:51:13.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:13 smithi031 ceph-mon[111289]: pgmap v59: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 35 B/s, 0 objects/s recovering 2024-04-22T23:51:13.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:13 smithi044 ceph-mon[82146]: pgmap v59: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 35 B/s, 0 objects/s recovering 2024-04-22T23:51:15.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:15 smithi031 ceph-mon[111289]: pgmap v60: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-04-22T23:51:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:15.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:15.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:15 smithi044 ceph-mon[82146]: pgmap v60: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-04-22T23:51:15.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:15.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:17.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:17 smithi031 ceph-mon[111289]: pgmap v61: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 820 B/s rd, 1 op/s; 28 B/s, 0 objects/s recovering 2024-04-22T23:51:17.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:17 smithi044 ceph-mon[82146]: pgmap v61: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 820 B/s rd, 1 op/s; 28 B/s, 0 objects/s recovering 2024-04-22T23:51:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:19 smithi031 ceph-mon[111289]: pgmap v62: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 745 B/s rd, 0 op/s; 25 B/s, 0 objects/s recovering 2024-04-22T23:51:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:19 smithi044 ceph-mon[82146]: pgmap v62: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 745 B/s rd, 0 op/s; 25 B/s, 0 objects/s recovering 2024-04-22T23:51:21.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:21 smithi031 ceph-mon[111289]: pgmap v63: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-04-22T23:51:21.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:21.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:21 smithi044 ceph-mon[82146]: pgmap v63: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 23 B/s, 0 objects/s recovering 2024-04-22T23:51:21.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:23.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:23 smithi031 ceph-mon[111289]: pgmap v64: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:51:23.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:23 smithi044 ceph-mon[82146]: pgmap v64: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:51:24.357 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:24 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:24.357 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:24 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:24.357 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:24 smithi031 ceph-mon[111289]: Upgrade: osd.1 is safe to restart 2024-04-22T23:51:24.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:24 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:24 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:51:24.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:24 smithi044 ceph-mon[82146]: Upgrade: osd.1 is safe to restart 2024-04-22T23:51:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:25 smithi031 ceph-mon[111289]: pgmap v65: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:51:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:51:25.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:51:25.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:25 smithi044 ceph-mon[82146]: pgmap v65: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:51:25.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:25.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:51:25.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 -- 172.21.15.31:0/2087113919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 msgr2=0x7f0c8c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 --2- 172.21.15.31:0/2087113919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c101280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009910 tx=0x7f0c7c009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 -- 172.21.15.31:0/2087113919 shutdown_connections 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 --2- 172.21.15.31:0/2087113919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c8c101850 0x7f0c8c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 --2- 172.21.15.31:0/2087113919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c101280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 -- 172.21.15.31:0/2087113919 >> 172.21.15.31:0/2087113919 conn(0x7f0c8c0fc750 msgr2=0x7f0c8c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 -- 172.21.15.31:0/2087113919 shutdown_connections 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.920+0000 7f0c913eb700 1 -- 172.21.15.31:0/2087113919 wait complete. 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c913eb700 1 Processor -- start 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c913eb700 1 -- start start 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c913eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:25.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c913eb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c8c101850 0x7f0c8c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c913eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c19d2e0 con 0x7f0c8c100ea0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c913eb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c196790 con 0x7f0c8c101850 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34792/0 (socket says 172.21.15.31:34792) 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c8affd700 1 -- 172.21.15.31:0/1910516676 learned_addr learned my addr 172.21.15.31:0/1910516676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.921+0000 7f0c8a7fc700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c8c101850 0x7f0c8c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c8affd700 1 -- 172.21.15.31:0/1910516676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c8c101850 msgr2=0x7f0c8c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c8affd700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c8c101850 0x7f0c8c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c8affd700 1 -- 172.21.15.31:0/1910516676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0095f0 con 0x7f0c8c100ea0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c8affd700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c19c710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006010 tx=0x7f0c7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7c01d070 con 0x7f0c8c100ea0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c7c005dc0 con 0x7f0c8c100ea0 2024-04-22T23:51:25.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c196a10 con 0x7f0c8c100ea0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7c017400 con 0x7f0c8c100ea0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.922+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c196f00 con 0x7f0c8c100ea0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.923+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c109240 con 0x7f0c8c100ea0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.924+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0c7c003a60 con 0x7f0c8c100ea0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.924+0000 7f0c73fff700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f0c7406ecc0 0x7f0c74071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.924+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0c7c090290 con 0x7f0c8c100ea0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.924+0000 7f0c8a7fc700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f0c7406ecc0 0x7f0c74071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.925+0000 7f0c8a7fc700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f0c7406ecc0 0x7f0c74071180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c197c00 tx=0x7f0c8000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:25.929+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0c7c05cbd0 con 0x7f0c8c100ea0 2024-04-22T23:51:26.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.163+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c8c197cf0 con 0x7f0c7406ecc0 2024-04-22T23:51:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.164+0000 7f0c73fff700 1 -- 172.21.15.31:0/1910516676 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0c8c197cf0 con 0x7f0c7406ecc0 2024-04-22T23:51:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f0c7406ecc0 msgr2=0x7f0c74071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f0c7406ecc0 0x7f0c74071180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c197c00 tx=0x7f0c8000b410 comp rx=0 tx=0).stop 2024-04-22T23:51:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 msgr2=0x7f0c8c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c19c710 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006010 tx=0x7f0c7c00bd70 comp rx=0 tx=0).stop 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 shutdown_connections 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f0c7406ecc0 0x7f0c74071180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c8c100ea0 0x7f0c8c19c710 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 --2- 172.21.15.31:0/1910516676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c8c101850 0x7f0c8c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 >> 172.21.15.31:0/1910516676 conn(0x7f0c8c0fc750 msgr2=0x7f0c8c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 shutdown_connections 2024-04-22T23:51:26.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.167+0000 7f0c913eb700 1 -- 172.21.15.31:0/1910516676 wait complete. 2024-04-22T23:51:26.169 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:51:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/3594779494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 msgr2=0x7f7c881058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/3594779494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c881058a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009910 tx=0x7f7c70009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/3594779494 shutdown_connections 2024-04-22T23:51:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/3594779494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c881058a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/3594779494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c88100ea0 0x7f7c88101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/3594779494 >> 172.21.15.31:0/3594779494 conn(0x7f7c880fc750 msgr2=0x7f7c880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:26.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:26 smithi031 ceph-mon[111289]: Upgrade: Updating osd.1 2024-04-22T23:51:26.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:26 smithi031 ceph-mon[111289]: Deploying daemon osd.1 on smithi031 2024-04-22T23:51:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/3594779494 shutdown_connections 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.279+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/3594779494 wait complete. 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c8d7e2700 1 Processor -- start 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c8d7e2700 1 -- start start 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c8d7e2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c88100ea0 0x7f7c88073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c8d7e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c8d7e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c88073b10 con 0x7f7c88101850 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c8d7e2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c88073c50 con 0x7f7c88100ea0 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c880735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34810/0 (socket says 172.21.15.31:34810) 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c867fc700 1 -- 172.21.15.31:0/566626298 learned_addr learned my addr 172.21.15.31:0/566626298 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c86ffd700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c88100ea0 0x7f7c88073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c867fc700 1 -- 172.21.15.31:0/566626298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c88100ea0 msgr2=0x7f7c88073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c867fc700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c88100ea0 0x7f7c88073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.280+0000 7f7c867fc700 1 -- 172.21.15.31:0/566626298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c700095f0 con 0x7f7c88101850 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.281+0000 7f7c867fc700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c880735d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c700092e0 tx=0x7f7c7000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.281+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c7001d070 con 0x7f7c88101850 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.281+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c88073ed0 con 0x7f7c88101850 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.281+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c7000f460 con 0x7f7c88101850 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.281+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c700175d0 con 0x7f7c88101850 2024-04-22T23:51:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.281+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c881a2d70 con 0x7f7c88101850 2024-04-22T23:51:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.282+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c881092a0 con 0x7f7c88101850 2024-04-22T23:51:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.283+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7c7000f5d0 con 0x7f7c88101850 2024-04-22T23:51:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.283+0000 7f7c7ffff700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f7c7406ecc0 0x7f7c74071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.283+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7c70090290 con 0x7f7c88101850 2024-04-22T23:51:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.283+0000 7f7c86ffd700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f7c7406ecc0 0x7f7c74071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.284+0000 7f7c86ffd700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f7c7406ecc0 0x7f7c74071180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c78005950 tx=0x7f7c7800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:26.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.288+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7c700590a0 con 0x7f7c88101850 2024-04-22T23:51:26.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:26 smithi044 ceph-mon[82146]: Upgrade: Updating osd.1 2024-04-22T23:51:26.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:26 smithi044 ceph-mon[82146]: Deploying daemon osd.1 on smithi031 2024-04-22T23:51:26.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.520+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c880611d0 con 0x7f7c7406ecc0 2024-04-22T23:51:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.521+0000 7f7c7ffff700 1 -- 172.21.15.31:0/566626298 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7c880611d0 con 0x7f7c7406ecc0 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f7c7406ecc0 msgr2=0x7f7c74071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f7c7406ecc0 0x7f7c74071180 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7c78005950 tx=0x7f7c7800a300 comp rx=0 tx=0).stop 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 msgr2=0x7f7c880735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c880735d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7c700092e0 tx=0x7f7c7000bd70 comp rx=0 tx=0).stop 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 shutdown_connections 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f7c7406ecc0 0x7f7c74071180 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c88100ea0 0x7f7c88073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 --2- 172.21.15.31:0/566626298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c88101850 0x7f7c880735d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.523+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 >> 172.21.15.31:0/566626298 conn(0x7f7c880fc750 msgr2=0x7f7c880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.524+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 shutdown_connections 2024-04-22T23:51:26.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.524+0000 7f7c8d7e2700 1 -- 172.21.15.31:0/566626298 wait complete. 2024-04-22T23:51:26.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 -- 172.21.15.31:0/1079502156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740fff50 msgr2=0x7f9474105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 --2- 172.21.15.31:0/1079502156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740fff50 0x7f9474105840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9464009910 tx=0x7f9464009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 -- 172.21.15.31:0/1079502156 shutdown_connections 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 --2- 172.21.15.31:0/1079502156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740fff50 0x7f9474105840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 --2- 172.21.15.31:0/1079502156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94740ff5a0 0x7f94740ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 -- 172.21.15.31:0/1079502156 >> 172.21.15.31:0/1079502156 conn(0x7f9474074e30 msgr2=0x7f9474075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 -- 172.21.15.31:0/1079502156 shutdown_connections 2024-04-22T23:51:26.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.631+0000 7f947b47d700 1 -- 172.21.15.31:0/1079502156 wait complete. 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f947b47d700 1 Processor -- start 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f947b47d700 1 -- start start 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f947b47d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 0x7f9474100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f947b47d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94740fff50 0x7f9474100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f947b47d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474100fa0 con 0x7f94740ff5a0 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f947b47d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94741010e0 con 0x7f94740fff50 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9479219700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 0x7f9474100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:26.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9479219700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 0x7f9474100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34834/0 (socket says 172.21.15.31:34834) 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9479219700 1 -- 172.21.15.31:0/1752644120 learned_addr learned my addr 172.21.15.31:0/1752644120 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9478a18700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94740fff50 0x7f9474100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9479219700 1 -- 172.21.15.31:0/1752644120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94740fff50 msgr2=0x7f9474100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9479219700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94740fff50 0x7f9474100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.632+0000 7f9479219700 1 -- 172.21.15.31:0/1752644120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640095f0 con 0x7f94740ff5a0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.633+0000 7f9479219700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 0x7f9474100520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f947000e920 tx=0x7f947000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.633+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f947000c9d0 con 0x7f94740ff5a0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.633+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9470010430 con 0x7f94740ff5a0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.633+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94741a2a40 con 0x7f94740ff5a0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.633+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94741a2df0 con 0x7f94740ff5a0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.633+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9470018630 con 0x7f94740ff5a0 2024-04-22T23:51:26.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.634+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9474109200 con 0x7f94740ff5a0 2024-04-22T23:51:26.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.634+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f94700105a0 con 0x7f94740ff5a0 2024-04-22T23:51:26.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.635+0000 7f946a7fc700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f946006ecc0 0x7f9460071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:26.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.635+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9470014070 con 0x7f94740ff5a0 2024-04-22T23:51:26.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.635+0000 7f9478a18700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f946006ecc0 0x7f9460071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:26.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.635+0000 7f9478a18700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f946006ecc0 0x7f9460071180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94640059d0 tx=0x7f9464005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:26.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.640+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9470054150 con 0x7f94740ff5a0 2024-04-22T23:51:26.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.880+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9474101bb0 con 0x7f946006ecc0 2024-04-22T23:51:26.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.900+0000 7f946a7fc700 1 -- 172.21.15.31:0/1752644120 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9474101bb0 con 0x7f946006ecc0 2024-04-22T23:51:26.892 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:51:26.892 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 23s ago 14m 25.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:51:26.892 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (14m) 23s ago 14m 8128k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:51:26.892 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (8m) 46s ago 13m 7964k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (55s) 23s ago 14m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (49s) 46s ago 13m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 23s ago 14m 76.1M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (8m) 23s ago 8m 16.4M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (8m) 23s ago 8m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (8m) 46s ago 8m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (8m) 46s ago 8m 20.0M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (5m) 23s ago 15m 504M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (4m) 46s ago 13m 450M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (2m) 23s ago 15m 46.0M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (100s) 46s ago 13m 34.9M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:51:26.893 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (4m) 23s ago 14m 15.2M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (4m) 46s ago 13m 15.8M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (29s) 23s ago 12m 15.1M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (12m) 23s ago 12m 208M 1217M 18.2.1 5be31c24972a ac296b0010f9 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (11m) 23s ago 11m 176M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (11m) 46s ago 11m 237M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (10m) 46s ago 10m 211M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (10m) 46s ago 10m 196M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:51:26.894 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 23s ago 13m 71.6M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:51:26.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f946006ecc0 msgr2=0x7f9460071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f946006ecc0 0x7f9460071180 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94640059d0 tx=0x7f9464005940 comp rx=0 tx=0).stop 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 msgr2=0x7f9474100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 0x7f9474100520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f947000e920 tx=0x7f947000ec30 comp rx=0 tx=0).stop 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 shutdown_connections 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f946006ecc0 0x7f9460071180 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94740ff5a0 0x7f9474100520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 --2- 172.21.15.31:0/1752644120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94740fff50 0x7f9474100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:26.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 >> 172.21.15.31:0/1752644120 conn(0x7f9474074e30 msgr2=0x7f94740fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:26.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 shutdown_connections 2024-04-22T23:51:26.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:26.903+0000 7f947b47d700 1 -- 172.21.15.31:0/1752644120 wait complete. 2024-04-22T23:51:27.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 -- 172.21.15.31:0/853173703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 msgr2=0x7f9530100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 --2- 172.21.15.31:0/853173703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530100080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9518009910 tx=0x7f9518009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:27.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 -- 172.21.15.31:0/853173703 shutdown_connections 2024-04-22T23:51:27.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 --2- 172.21.15.31:0/853173703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530100080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 --2- 172.21.15.31:0/853173703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9530105bd0 0x7f9530105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 -- 172.21.15.31:0/853173703 >> 172.21.15.31:0/853173703 conn(0x7f9530075670 msgr2=0x7f9530075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:27.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 -- 172.21.15.31:0/853173703 shutdown_connections 2024-04-22T23:51:27.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 -- 172.21.15.31:0/853173703 wait complete. 2024-04-22T23:51:27.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.013+0000 7f953568d700 1 Processor -- start 2024-04-22T23:51:27.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f953568d700 1 -- start start 2024-04-22T23:51:27.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f953568d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f953568d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9530105bd0 0x7f9530197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f953568d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9530198380 con 0x7f95300ffc20 2024-04-22T23:51:27.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f953568d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301984f0 con 0x7f9530105bd0 2024-04-22T23:51:27.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34856/0 (socket says 172.21.15.31:34856) 2024-04-22T23:51:27.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 -- 172.21.15.31:0/2598482388 learned_addr learned my addr 172.21.15.31:0/2598482388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:27.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952e7fc700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9530105bd0 0x7f9530197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 -- 172.21.15.31:0/2598482388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9530105bd0 msgr2=0x7f9530197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9530105bd0 0x7f9530197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 -- 172.21.15.31:0/2598482388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180095f0 con 0x7f95300ffc20 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.014+0000 7f952effd700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530197760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f952800d8d0 tx=0x7f952800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.015+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95280098a0 con 0x7f95300ffc20 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.015+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9528010460 con 0x7f95300ffc20 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.015+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95301a07d0 con 0x7f95300ffc20 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.015+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952800f5f0 con 0x7f95300ffc20 2024-04-22T23:51:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.015+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95301a0d20 con 0x7f95300ffc20 2024-04-22T23:51:27.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.016+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95301080d0 con 0x7f95300ffc20 2024-04-22T23:51:27.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.016+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9528009a00 con 0x7f95300ffc20 2024-04-22T23:51:27.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.017+0000 7f9527fff700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f951c06ecc0 0x7f951c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.017+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f952808ecd0 con 0x7f95300ffc20 2024-04-22T23:51:27.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.017+0000 7f952e7fc700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f951c06ecc0 0x7f951c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.017+0000 7f952e7fc700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f951c06ecc0 0x7f951c071180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9530068e30 tx=0x7f951801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:27.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.022+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9528057ae0 con 0x7f95300ffc20 2024-04-22T23:51:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.328+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f953004f2e0 con 0x7f95300ffc20 2024-04-22T23:51:27.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.328+0000 7f9527fff700 1 -- 172.21.15.31:0/2598482388 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f952805b1a0 con 0x7f95300ffc20 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:51:27.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:51:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:51:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:51:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-04-22T23:51:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 5 2024-04-22T23:51:27.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:51:27.321 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f951c06ecc0 msgr2=0x7f951c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f951c06ecc0 0x7f951c071180 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9530068e30 tx=0x7f951801a040 comp rx=0 tx=0).stop 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 msgr2=0x7f9530197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530197760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f952800d8d0 tx=0x7f952800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 shutdown_connections 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f951c06ecc0 0x7f951c071180 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95300ffc20 0x7f9530197760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 --2- 172.21.15.31:0/2598482388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9530105bd0 0x7f9530197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.330+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 >> 172.21.15.31:0/2598482388 conn(0x7f9530075670 msgr2=0x7f95300fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.331+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 shutdown_connections 2024-04-22T23:51:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.331+0000 7f953568d700 1 -- 172.21.15.31:0/2598482388 wait complete. 2024-04-22T23:51:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1607323457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 msgr2=0x7f6d201070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1607323457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d201070b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009910 tx=0x7f6d10009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1607323457 shutdown_connections 2024-04-22T23:51:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1607323457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 0x7f6d200689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1607323457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d201070b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.432 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:27 smithi031 ceph-mon[111289]: pgmap v66: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:51:27.432 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:27 smithi031 ceph-mon[111289]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:27.432 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:27 smithi031 ceph-mon[111289]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:27.432 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:27 smithi031 ceph-mon[111289]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1607323457 >> 172.21.15.31:0/1607323457 conn(0x7f6d20075670 msgr2=0x7f6d20075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1607323457 shutdown_connections 2024-04-22T23:51:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.439+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1607323457 wait complete. 2024-04-22T23:51:27.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.440+0000 7f6d24ef3700 1 Processor -- start 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.440+0000 7f6d24ef3700 1 -- start start 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d24ef3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 0x7f6d2019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d24ef3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d2019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d24ef3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2019d230 con 0x7f6d20106cd0 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d24ef3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d201966e0 con 0x7f6d20068580 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d2019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d2019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34866/0 (socket says 172.21.15.31:34866) 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 -- 172.21.15.31:0/1325135163 learned_addr learned my addr 172.21.15.31:0/1325135163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1e59c700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 0x7f6d2019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 -- 172.21.15.31:0/1325135163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 msgr2=0x7f6d2019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 0x7f6d2019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 -- 172.21.15.31:0/1325135163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d100095f0 con 0x7f6d20106cd0 2024-04-22T23:51:27.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1e59c700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 0x7f6d2019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.441+0000 7f6d1dd9b700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d2019cba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d8d0 tx=0x7f6d0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.442+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d08009940 con 0x7f6d20106cd0 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.442+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d08010460 con 0x7f6d20106cd0 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.442+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d201969c0 con 0x7f6d20106cd0 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.442+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0800f5f0 con 0x7f6d20106cd0 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.442+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d20196e50 con 0x7f6d20106cd0 2024-04-22T23:51:27.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.443+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d201091d0 con 0x7f6d20106cd0 2024-04-22T23:51:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.444+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6d080105d0 con 0x7f6d20106cd0 2024-04-22T23:51:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.444+0000 7f6d177fe700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6d0c06ecc0 0x7f6d0c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.444+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6d0808ee10 con 0x7f6d20106cd0 2024-04-22T23:51:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.445+0000 7f6d1e59c700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6d0c06ecc0 0x7f6d0c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.445+0000 7f6d1e59c700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6d0c06ecc0 0x7f6d0c071180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005b40 tx=0x7f6d1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:27.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.449+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6d0805b750 con 0x7f6d20106cd0 2024-04-22T23:51:27.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:27 smithi044 ceph-mon[82146]: pgmap v66: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:51:27.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:27 smithi044 ceph-mon[82146]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:27.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:27 smithi044 ceph-mon[82146]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:27.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:27 smithi044 ceph-mon[82146]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.719+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d2004f2e0 con 0x7f6d20106cd0 2024-04-22T23:51:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.719+0000 7f6d177fe700 1 -- 172.21.15.31:0/1325135163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f6d0805b2e0 con 0x7f6d20106cd0 2024-04-22T23:51:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:51:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:51:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:51:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:51:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:51:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:51:27.714 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.721+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6d0c06ecc0 msgr2=0x7f6d0c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.721+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6d0c06ecc0 0x7f6d0c071180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005b40 tx=0x7f6d1001a040 comp rx=0 tx=0).stop 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.721+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 msgr2=0x7f6d2019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.721+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d2019cba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800d8d0 tx=0x7f6d0800dc90 comp rx=0 tx=0).stop 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 shutdown_connections 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6d0c06ecc0 0x7f6d0c071180 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d20068580 0x7f6d2019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 --2- 172.21.15.31:0/1325135163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d20106cd0 0x7f6d2019cba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 >> 172.21.15.31:0/1325135163 conn(0x7f6d20075670 msgr2=0x7f6d200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 shutdown_connections 2024-04-22T23:51:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.722+0000 7f6d24ef3700 1 -- 172.21.15.31:0/1325135163 wait complete. 2024-04-22T23:51:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:51:27.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 -- 172.21.15.31:0/2072161501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000ffda0 msgr2=0x7f8100100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2072161501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000ffda0 0x7f8100100180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009910 tx=0x7f80fc009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:27.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 -- 172.21.15.31:0/2072161501 shutdown_connections 2024-04-22T23:51:27.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2072161501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8100100750 0x7f81001047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2072161501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81000ffda0 0x7f8100100180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 -- 172.21.15.31:0/2072161501 >> 172.21.15.31:0/2072161501 conn(0x7f81000fb650 msgr2=0x7f81000fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:27.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 -- 172.21.15.31:0/2072161501 shutdown_connections 2024-04-22T23:51:27.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 -- 172.21.15.31:0/2072161501 wait complete. 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.833+0000 7f8107ab2700 1 Processor -- start 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f8107ab2700 1 -- start start 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f8107ab2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ffda0 0x7f81001976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f8107ab2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 0x7f8100197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f8107ab2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8100198230 con 0x7f8100100750 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f8107ab2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81001983a0 con 0x7f81000ffda0 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 0x7f8100197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 0x7f8100197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34892/0 (socket says 172.21.15.31:34892) 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 -- 172.21.15.31:0/2248835095 learned_addr learned my addr 172.21.15.31:0/2248835095 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810584e700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ffda0 0x7f81001976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 -- 172.21.15.31:0/2248835095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ffda0 msgr2=0x7f81001976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:27.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ffda0 0x7f81001976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 -- 172.21.15.31:0/2248835095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0095f0 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.834+0000 7f810504d700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 0x7f8100197c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80f000d900 tx=0x7f80f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.835+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f0009900 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.835+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81001a0740 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.835+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80f0010460 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.835+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80f000f5f0 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.835+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81001a0c30 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.836+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8100108140 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.836+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f80f00105d0 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.837+0000 7f80f6ffd700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80ec06ed10 0x7f80ec0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.837+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f80f008ecf0 con 0x7f8100100750 2024-04-22T23:51:27.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.837+0000 7f810584e700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80ec06ed10 0x7f80ec0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:27.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.837+0000 7f810584e700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80ec06ed10 0x7f80ec0711d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b5c0 tx=0x7f80fc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:27.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:27.841+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f80f0057a60 con 0x7f8100100750 2024-04-22T23:51:28.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.076+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f810002cce0 con 0x7f80ec06ed10 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.078+0000 7f80f6ffd700 1 -- 172.21.15.31:0/2248835095 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f810002cce0 con 0x7f80ec06ed10 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:51:28.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout: "crash" 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "7/23 daemons upgraded", 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:51:28.070 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80ec06ed10 msgr2=0x7f80ec0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80ec06ed10 0x7f80ec0711d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00b5c0 tx=0x7f80fc01a040 comp rx=0 tx=0).stop 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 msgr2=0x7f8100197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 0x7f8100197c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80f000d900 tx=0x7f80f000dcc0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 shutdown_connections 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80ec06ed10 0x7f80ec0711d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81000ffda0 0x7f81001976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 --2- 172.21.15.31:0/2248835095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8100100750 0x7f8100197c10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 >> 172.21.15.31:0/2248835095 conn(0x7f81000fb650 msgr2=0x7f81000fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 shutdown_connections 2024-04-22T23:51:28.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.080+0000 7f8107ab2700 1 -- 172.21.15.31:0/2248835095 wait complete. 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 -- 172.21.15.31:0/988090914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 msgr2=0x7f3ab4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 --2- 172.21.15.31:0/988090914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4105fb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009910 tx=0x7f3aa4009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 -- 172.21.15.31:0/988090914 shutdown_connections 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 --2- 172.21.15.31:0/988090914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab40ffc20 0x7f3ab4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 --2- 172.21.15.31:0/988090914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4105fb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 -- 172.21.15.31:0/988090914 >> 172.21.15.31:0/988090914 conn(0x7f3ab4075670 msgr2=0x7f3ab4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 -- 172.21.15.31:0/988090914 shutdown_connections 2024-04-22T23:51:28.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.196+0000 7f3ab956e700 1 -- 172.21.15.31:0/988090914 wait complete. 2024-04-22T23:51:28.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.197+0000 7f3ab956e700 1 Processor -- start 2024-04-22T23:51:28.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.197+0000 7f3ab956e700 1 -- start start 2024-04-22T23:51:28.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.197+0000 7f3ab956e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab40ffc20 0x7f3ab41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:28.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.197+0000 7f3ab956e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:28.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.197+0000 7f3ab956e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab41982d0 con 0x7f3ab4105bd0 2024-04-22T23:51:28.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.197+0000 7f3ab956e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab4198440 con 0x7f3ab40ffc20 2024-04-22T23:51:28.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34914/0 (socket says 172.21.15.31:34914) 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 -- 172.21.15.31:0/2355521215 learned_addr learned my addr 172.21.15.31:0/2355521215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab2ffd700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab40ffc20 0x7f3ab41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 -- 172.21.15.31:0/2355521215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab40ffc20 msgr2=0x7f3ab41976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab40ffc20 0x7f3ab41976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 -- 172.21.15.31:0/2355521215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40095f0 con 0x7f3ab4105bd0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab27fc700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4197bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e920 tx=0x7f3aa800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa800c9d0 con 0x7f3ab4105bd0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3aa8010430 con 0x7f3ab4105bd0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab41a0720 con 0x7f3ab4105bd0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.198+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa8018630 con 0x7f3ab4105bd0 2024-04-22T23:51:28.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.199+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab41a0c70 con 0x7f3ab4105bd0 2024-04-22T23:51:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.200+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab41080d0 con 0x7f3ab4105bd0 2024-04-22T23:51:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.200+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3aa800cb30 con 0x7f3ab4105bd0 2024-04-22T23:51:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.200+0000 7f3a9bfff700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3a9c06ecc0 0x7f3a9c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.200+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3aa8014070 con 0x7f3ab4105bd0 2024-04-22T23:51:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.201+0000 7f3ab2ffd700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3a9c06ecc0 0x7f3a9c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:28.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.201+0000 7f3ab2ffd700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3a9c06ecc0 0x7f3a9c071180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b760 tx=0x7f3aa4005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:28.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.205+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f3aa8054150 con 0x7f3ab4105bd0 2024-04-22T23:51:28.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:28 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2598482388' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:28.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:28 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1325135163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:51:28.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:28 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2598482388' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:28.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:28 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1325135163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:51:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.524+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3ab4107ab0 con 0x7f3ab4105bd0 2024-04-22T23:51:28.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.524+0000 7f3a9bfff700 1 -- 172.21.15.31:0/2355521215 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3aa8057eb0 con 0x7f3ab4105bd0 2024-04-22T23:51:28.517 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:51:28.517 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:51:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3a9c06ecc0 msgr2=0x7f3a9c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3a9c06ecc0 0x7f3a9c071180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b760 tx=0x7f3aa4005a90 comp rx=0 tx=0).stop 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 msgr2=0x7f3ab4197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4197bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e920 tx=0x7f3aa800ec30 comp rx=0 tx=0).stop 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 shutdown_connections 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3a9c06ecc0 0x7f3a9c071180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ab40ffc20 0x7f3ab41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 --2- 172.21.15.31:0/2355521215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ab4105bd0 0x7f3ab4197bf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 >> 172.21.15.31:0/2355521215 conn(0x7f3ab4075670 msgr2=0x7f3ab40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 shutdown_connections 2024-04-22T23:51:28.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:28.528+0000 7f3ab956e700 1 -- 172.21.15.31:0/2355521215 wait complete. 2024-04-22T23:51:28.778 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:28 smithi031 systemd[1]: Stopping Ceph osd.1 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:51:29.185 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:28 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[52945]: 2024-04-22T23:51:28.786+0000 7fb3b7cf3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:51:29.186 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:28 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[52945]: 2024-04-22T23:51:28.786+0000 7fb3b7cf3700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-22T23:51:29.186 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:28 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[52945]: 2024-04-22T23:51:28.786+0000 7fb3b7cf3700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:51:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:29 smithi031 ceph-mon[111289]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:29 smithi031 ceph-mon[111289]: pgmap v67: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:51:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:29 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2355521215' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:51:29.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:29 smithi031 ceph-mon[111289]: osd.1 marked itself down and dead 2024-04-22T23:51:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:29 smithi044 ceph-mon[82146]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:29 smithi044 ceph-mon[82146]: pgmap v67: 65 pgs: 65 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:51:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:29 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2355521215' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:51:29.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:29 smithi044 ceph-mon[82146]: osd.1 marked itself down and dead 2024-04-22T23:51:29.854 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:29 smithi031 podman[128006]: 2024-04-22 23:51:29.604270824 +0000 UTC m=+0.946845409 container died ac296b0010f93c7e1f0bc5c9540ca74f8bebe49d454432b8bf07f8beed25ad49 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-22T23:51:30.187 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:29 smithi031 podman[128006]: 2024-04-22 23:51:29.961173777 +0000 UTC m=+1.303748358 container cleanup ac296b0010f93c7e1f0bc5c9540ca74f8bebe49d454432b8bf07f8beed25ad49 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T23:51:30.188 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:29 smithi031 bash[128006]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1 2024-04-22T23:51:30.443 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:30 smithi031 podman[128023]: 2024-04-22 23:51:30.254404667 +0000 UTC m=+0.652139184 container remove ac296b0010f93c7e1f0bc5c9540ca74f8bebe49d454432b8bf07f8beed25ad49 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: osdmap e55: 6 total, 5 up, 6 in 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [0, 2]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 0]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 5]}]: dispatch 2024-04-22T23:51:30.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]: dispatch 2024-04-22T23:51:30.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:30.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: osdmap e55: 6 total, 5 up, 6 in 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [0, 2]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 0]}]: dispatch 2024-04-22T23:51:30.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]: dispatch 2024-04-22T23:51:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 5]}]: dispatch 2024-04-22T23:51:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]: dispatch 2024-04-22T23:51:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:30.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:30.943 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:30 smithi031 podman[128113]: 2024-04-22 23:51:30.494534383 +0000 UTC m=+0.019150537 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:51:30.944 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:30 smithi031 podman[128113]: 2024-04-22 23:51:30.597245154 +0000 UTC m=+0.121861274 container create f7f043de976d9f2bbd8dd8fd04d929fd7e37edaa09bca0782626a66573239935 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-22T23:51:31.355 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: pgmap v69: 65 pgs: 12 stale+active+clean, 53 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-22T23:51:31.355 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [0, 2]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 0]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 5]}]': finished 2024-04-22T23:51:31.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]': finished 2024-04-22T23:51:31.357 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:31 smithi031 ceph-mon[111289]: osdmap e56: 6 total, 5 up, 6 in 2024-04-22T23:51:31.357 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 podman[128113]: 2024-04-22 23:51:31.041994081 +0000 UTC m=+0.566610219 container init f7f043de976d9f2bbd8dd8fd04d929fd7e37edaa09bca0782626a66573239935 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T23:51:31.357 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 podman[128113]: 2024-04-22 23:51:31.045455037 +0000 UTC m=+0.570071156 container start f7f043de976d9f2bbd8dd8fd04d929fd7e37edaa09bca0782626a66573239935 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:51:31.357 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 podman[128113]: 2024-04-22 23:51:31.124977321 +0000 UTC m=+0.649593445 container attach f7f043de976d9f2bbd8dd8fd04d929fd7e37edaa09bca0782626a66573239935 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:51:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: pgmap v69: 65 pgs: 12 stale+active+clean, 53 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-22T23:51:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 5]}]': finished 2024-04-22T23:51:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.7", "id": [3, 1]}]': finished 2024-04-22T23:51:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [3, 1]}]': finished 2024-04-22T23:51:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [3, 1]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [0, 2]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [0, 1]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [3, 0]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 1]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.d", "id": [4, 5]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 1]}]': finished 2024-04-22T23:51:31.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:31 smithi044 ceph-mon[82146]: osdmap e56: 6 total, 5 up, 6 in 2024-04-22T23:51:31.660 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 podman[128138]: 2024-04-22 23:51:31.394249335 +0000 UTC m=+0.019777391 container died f7f043de976d9f2bbd8dd8fd04d929fd7e37edaa09bca0782626a66573239935 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-22T23:51:32.182 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 podman[128138]: 2024-04-22 23:51:31.937067946 +0000 UTC m=+0.562595988 container remove f7f043de976d9f2bbd8dd8fd04d929fd7e37edaa09bca0782626a66573239935 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:51:32.183 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-22T23:51:32.183 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 systemd[1]: Stopped Ceph osd.1 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:51:32.183 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:31 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.1.service: Consumed 58.099s CPU time. 2024-04-22T23:51:32.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:32 smithi031 ceph-mon[111289]: osdmap e57: 6 total, 5 up, 6 in 2024-04-22T23:51:32.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:32 smithi031 systemd[1]: Starting Ceph osd.1 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:51:32.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:32 smithi031 podman[128238]: 2024-04-22 23:51:32.335479242 +0000 UTC m=+0.019797124 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:51:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:32 smithi044 ceph-mon[82146]: osdmap e57: 6 total, 5 up, 6 in 2024-04-22T23:51:32.764 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:32 smithi031 podman[128238]: 2024-04-22 23:51:32.454701974 +0000 UTC m=+0.139019850 container create 063b618163b8c2bd8433525035036986b73107554b7d4f7f437bae8d06c1a742 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T23:51:32.764 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:32 smithi031 podman[128238]: 2024-04-22 23:51:32.683068131 +0000 UTC m=+0.367386020 container init 063b618163b8c2bd8433525035036986b73107554b7d4f7f437bae8d06c1a742 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:51:32.764 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:32 smithi031 podman[128238]: 2024-04-22 23:51:32.686388609 +0000 UTC m=+0.370706487 container start 063b618163b8c2bd8433525035036986b73107554b7d4f7f437bae8d06c1a742 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:51:33.192 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:32 smithi031 podman[128238]: 2024-04-22 23:51:32.792513049 +0000 UTC m=+0.476830927 container attach 063b618163b8c2bd8433525035036986b73107554b7d4f7f437bae8d06c1a742 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T23:51:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-mon[111289]: pgmap v72: 65 pgs: 4 unknown, 3 peering, 10 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 28 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/264 objects degraded (10.606%) 2024-04-22T23:51:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-mon[111289]: Health check failed: Degraded data redundancy: 28/264 objects degraded (10.606%), 10 pgs degraded (PG_DEGRADED) 2024-04-22T23:51:33.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-mon[111289]: osdmap e58: 6 total, 5 up, 6 in 2024-04-22T23:51:33.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:51:33.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:51:33.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:51:33.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:51:33.444 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 bash[128238]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate[128253]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-22T23:51:33.445 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 podman[128238]: 2024-04-22 23:51:33.23181451 +0000 UTC m=+0.916132388 container died 063b618163b8c2bd8433525035036986b73107554b7d4f7f437bae8d06c1a742 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T23:51:33.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:33 smithi044 ceph-mon[82146]: pgmap v72: 65 pgs: 4 unknown, 3 peering, 10 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 28 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/264 objects degraded (10.606%) 2024-04-22T23:51:33.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:33 smithi044 ceph-mon[82146]: Health check failed: Degraded data redundancy: 28/264 objects degraded (10.606%), 10 pgs degraded (PG_DEGRADED) 2024-04-22T23:51:33.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:33 smithi044 ceph-mon[82146]: osdmap e58: 6 total, 5 up, 6 in 2024-04-22T23:51:33.944 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:33 smithi031 podman[128394]: 2024-04-22 23:51:33.786395208 +0000 UTC m=+0.549123366 container remove 063b618163b8c2bd8433525035036986b73107554b7d4f7f437bae8d06c1a742 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:51:34.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:34 smithi031 ceph-mon[111289]: osdmap e59: 6 total, 5 up, 6 in 2024-04-22T23:51:34.247 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 podman[128456]: 2024-04-22 23:51:33.91006395 +0000 UTC m=+0.019239067 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:51:34.248 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 podman[128456]: 2024-04-22 23:51:34.012366667 +0000 UTC m=+0.121541774 container create fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T23:51:34.248 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 podman[128456]: 2024-04-22 23:51:34.256933037 +0000 UTC m=+0.366108146 container init fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-22T23:51:34.694 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 podman[128456]: 2024-04-22 23:51:34.26032346 +0000 UTC m=+0.369498568 container start fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-22T23:51:34.694 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 ceph-osd[128475]: -- 172.21.15.31:0/4051005027 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d16061680 con 0x556d1602ac00 2024-04-22T23:51:34.694 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 bash[128456]: fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e 2024-04-22T23:51:34.694 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 systemd[1]: Started Ceph osd.1 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:51:34.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:34 smithi044 ceph-mon[82146]: osdmap e59: 6 total, 5 up, 6 in 2024-04-22T23:51:35.193 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:34 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-22T23:51:34.804+0000 7feab31c27c0 -1 Falling back to public interface 2024-04-22T23:51:35.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:35 smithi031 ceph-mon[111289]: pgmap v75: 65 pgs: 4 unknown, 3 peering, 10 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 28 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/264 objects degraded (10.606%) 2024-04-22T23:51:35.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:35 smithi031 ceph-mon[111289]: osdmap e60: 6 total, 5 up, 6 in 2024-04-22T23:51:35.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:35.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:35.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:35 smithi044 ceph-mon[82146]: pgmap v75: 65 pgs: 4 unknown, 3 peering, 10 active+undersized+degraded, 3 stale+active+clean, 17 active+undersized, 28 active+clean; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/264 objects degraded (10.606%) 2024-04-22T23:51:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:35 smithi044 ceph-mon[82146]: osdmap e60: 6 total, 5 up, 6 in 2024-04-22T23:51:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:35 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:35 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:35.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:35 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:37.602 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:37 smithi031 ceph-mon[111289]: pgmap v77: 65 pgs: 7 active+clean+remapped, 1 remapped+peering, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%); 4.7 MiB/s, 91 objects/s recovering 2024-04-22T23:51:37.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:37 smithi044 ceph-mon[82146]: pgmap v77: 65 pgs: 7 active+clean+remapped, 1 remapped+peering, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%); 4.7 MiB/s, 91 objects/s recovering 2024-04-22T23:51:39.319 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:39 smithi031 ceph-mon[111289]: pgmap v78: 65 pgs: 7 active+clean+remapped, 1 remapped+peering, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%); 3.8 MiB/s, 74 objects/s recovering 2024-04-22T23:51:39.693 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:39 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-22T23:51:39.331+0000 7feab31c27c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-22T23:51:39.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:39 smithi044 ceph-mon[82146]: pgmap v78: 65 pgs: 7 active+clean+remapped, 1 remapped+peering, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%); 3.8 MiB/s, 74 objects/s recovering 2024-04-22T23:51:40.194 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:39 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-22T23:51:39.713+0000 7feab31c27c0 -1 osd.1 54 log_to_monitors true 2024-04-22T23:51:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:40 smithi044 ceph-mon[82146]: from='osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:51:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:40 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:40.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:40 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:40.693 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 22 23:51:40 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-22T23:51:40.259+0000 7feaa461a700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:51:40.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:40 smithi031 ceph-mon[111289]: from='osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:51:40.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:40 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:40.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:40 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:41.492 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:41 smithi031 ceph-mon[111289]: pgmap v79: 65 pgs: 8 active+clean+remapped, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%); 2.9 MiB/s, 56 objects/s recovering 2024-04-22T23:51:41.492 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:41 smithi031 ceph-mon[111289]: from='osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:51:41.493 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:41 smithi031 ceph-mon[111289]: osdmap e61: 6 total, 5 up, 6 in 2024-04-22T23:51:41.493 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:41 smithi031 ceph-mon[111289]: from='osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:51:41.493 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:41.493 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:41 smithi044 ceph-mon[82146]: pgmap v79: 65 pgs: 8 active+clean+remapped, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%); 2.9 MiB/s, 56 objects/s recovering 2024-04-22T23:51:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:41 smithi044 ceph-mon[82146]: from='osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:51:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:41 smithi044 ceph-mon[82146]: osdmap e61: 6 total, 5 up, 6 in 2024-04-22T23:51:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:41 smithi044 ceph-mon[82146]: from='osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:51:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:41.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:42.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:42 smithi031 ceph-mon[111289]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:51:42.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:42 smithi031 ceph-mon[111289]: osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875] boot 2024-04-22T23:51:42.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:42 smithi031 ceph-mon[111289]: osdmap e62: 6 total, 6 up, 6 in 2024-04-22T23:51:42.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:42 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:51:42.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:42 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 30/264 objects degraded (11.364%), 12 pgs degraded (PG_DEGRADED) 2024-04-22T23:51:42.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:42 smithi031 ceph-mon[111289]: osdmap e63: 6 total, 6 up, 6 in 2024-04-22T23:51:42.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:42 smithi044 ceph-mon[82146]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:51:42.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:42 smithi044 ceph-mon[82146]: osd.1 [v2:172.21.15.31:6810/753330875,v1:172.21.15.31:6811/753330875] boot 2024-04-22T23:51:42.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:42 smithi044 ceph-mon[82146]: osdmap e62: 6 total, 6 up, 6 in 2024-04-22T23:51:42.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:42 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:51:42.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:42 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 30/264 objects degraded (11.364%), 12 pgs degraded (PG_DEGRADED) 2024-04-22T23:51:42.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:42 smithi044 ceph-mon[82146]: osdmap e63: 6 total, 6 up, 6 in 2024-04-22T23:51:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:43 smithi044 ceph-mon[82146]: pgmap v83: 65 pgs: 8 active+clean+remapped, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%) 2024-04-22T23:51:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:43 smithi044 ceph-mon[82146]: osdmap e64: 6 total, 6 up, 6 in 2024-04-22T23:51:43.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:43 smithi031 ceph-mon[111289]: pgmap v83: 65 pgs: 8 active+clean+remapped, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%) 2024-04-22T23:51:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:43 smithi031 ceph-mon[111289]: osdmap e64: 6 total, 6 up, 6 in 2024-04-22T23:51:44.680 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:44 smithi031 ceph-mon[111289]: osdmap e65: 6 total, 6 up, 6 in 2024-04-22T23:51:44.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:44 smithi044 ceph-mon[82146]: osdmap e65: 6 total, 6 up, 6 in 2024-04-22T23:51:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: pgmap v86: 65 pgs: 8 active+clean+remapped, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%) 2024-04-22T23:51:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: osdmap e66: 6 total, 6 up, 6 in 2024-04-22T23:51:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:51:45.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:51:45.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:45.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:45.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:45.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:45.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:51:45.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:51:45.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:51:45.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: pgmap v86: 65 pgs: 8 active+clean+remapped, 25 active+clean, 20 active+undersized, 12 active+undersized+degraded; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s; 30/264 objects degraded (11.364%); 15/264 objects misplaced (5.682%) 2024-04-22T23:51:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: osdmap e66: 6 total, 6 up, 6 in 2024-04-22T23:51:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:51:45.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:51:45.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:51:45.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:45.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:45.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:51:45.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:51:45.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:51:45.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:51:45.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:45.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:51:46.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:46 smithi031 ceph-mon[111289]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/264 objects degraded (11.364%), 12 pgs degraded) 2024-04-22T23:51:46.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:46 smithi044 ceph-mon[82146]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/264 objects degraded (11.364%), 12 pgs degraded) 2024-04-22T23:51:47.932 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:47 smithi031 ceph-mon[111289]: pgmap v88: 65 pgs: 1 peering, 64 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 6.0 MiB/s, 0 keys/s, 98 objects/s recovering 2024-04-22T23:51:47.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:47 smithi044 ceph-mon[82146]: pgmap v88: 65 pgs: 1 peering, 64 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 6.0 MiB/s, 0 keys/s, 98 objects/s recovering 2024-04-22T23:51:49.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:49 smithi031 ceph-mon[111289]: pgmap v89: 65 pgs: 1 peering, 64 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 4.7 MiB/s, 0 keys/s, 76 objects/s recovering 2024-04-22T23:51:49.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:49 smithi044 ceph-mon[82146]: pgmap v89: 65 pgs: 1 peering, 64 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 4.7 MiB/s, 0 keys/s, 76 objects/s recovering 2024-04-22T23:51:51.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:51 smithi031 ceph-mon[111289]: pgmap v90: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 668 B/s rd, 1 op/s; 3.7 MiB/s, 0 keys/s, 60 objects/s recovering 2024-04-22T23:51:51.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:51 smithi044 ceph-mon[82146]: pgmap v90: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 668 B/s rd, 1 op/s; 3.7 MiB/s, 0 keys/s, 60 objects/s recovering 2024-04-22T23:51:53.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:53 smithi031 ceph-mon[111289]: pgmap v91: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 3.2 MiB/s, 0 keys/s, 53 objects/s recovering 2024-04-22T23:51:53.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:53 smithi044 ceph-mon[82146]: pgmap v91: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 3.2 MiB/s, 0 keys/s, 53 objects/s recovering 2024-04-22T23:51:55.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:55 smithi031 ceph-mon[111289]: pgmap v92: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 2.8 MiB/s, 0 keys/s, 45 objects/s recovering 2024-04-22T23:51:55.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:55 smithi044 ceph-mon[82146]: pgmap v92: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 2.8 MiB/s, 0 keys/s, 45 objects/s recovering 2024-04-22T23:51:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:51:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:57 smithi031 ceph-mon[111289]: pgmap v93: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 2.4 MiB/s, 0 keys/s, 21 objects/s recovering 2024-04-22T23:51:57.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:57 smithi044 ceph-mon[82146]: pgmap v93: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 2.4 MiB/s, 0 keys/s, 21 objects/s recovering 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 -- 172.21.15.31:0/2077328450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 msgr2=0x7febd01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 --2- 172.21.15.31:0/2077328450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd01047a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febc4009910 tx=0x7febc4009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 -- 172.21.15.31:0/2077328450 shutdown_connections 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 --2- 172.21.15.31:0/2077328450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd01047a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 --2- 172.21.15.31:0/2077328450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd00ffda0 0x7febd0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 -- 172.21.15.31:0/2077328450 >> 172.21.15.31:0/2077328450 conn(0x7febd00fb650 msgr2=0x7febd00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:58.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 -- 172.21.15.31:0/2077328450 shutdown_connections 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 -- 172.21.15.31:0/2077328450 wait complete. 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 Processor -- start 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.653+0000 7febd6493700 1 -- start start 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febd6493700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd00ffda0 0x7febd019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febd6493700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febd6493700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd019d240 con 0x7febd0100750 2024-04-22T23:51:58.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febd6493700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd01966f0 con 0x7febd00ffda0 2024-04-22T23:51:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51924/0 (socket says 172.21.15.31:51924) 2024-04-22T23:51:58.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcf7fe700 1 -- 172.21.15.31:0/2500084481 learned_addr learned my addr 172.21.15.31:0/2500084481 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcffff700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd00ffda0 0x7febd019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcf7fe700 1 -- 172.21.15.31:0/2500084481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd00ffda0 msgr2=0x7febd019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcf7fe700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd00ffda0 0x7febd019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.654+0000 7febcf7fe700 1 -- 172.21.15.31:0/2500084481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40095f0 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.655+0000 7febcf7fe700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd019cbb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febc40092e0 tx=0x7febc400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.655+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc401d070 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.655+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febc400f460 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.655+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febc40175d0 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.655+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd0196970 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.655+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd0196e60 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.656+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd01091b0 con 0x7febd0100750 2024-04-22T23:51:58.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.656+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7febc400f5d0 con 0x7febd0100750 2024-04-22T23:51:58.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.657+0000 7febcd7fa700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7febb806ecc0 0x7febb8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:58.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.657+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7febc40903f0 con 0x7febd0100750 2024-04-22T23:51:58.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.657+0000 7febcffff700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7febb806ecc0 0x7febb8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:58.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.658+0000 7febcffff700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7febb806ecc0 0x7febb8071180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7febc0005950 tx=0x7febc000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:58.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.663+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7febc405cc10 con 0x7febd0100750 2024-04-22T23:51:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.892+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febd0197080 con 0x7febb806ecc0 2024-04-22T23:51:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.894+0000 7febcd7fa700 1 -- 172.21.15.31:0/2500084481 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7febd0197080 con 0x7febb806ecc0 2024-04-22T23:51:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7febb806ecc0 msgr2=0x7febb8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7febb806ecc0 0x7febb8071180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7febc0005950 tx=0x7febc000a300 comp rx=0 tx=0).stop 2024-04-22T23:51:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 msgr2=0x7febd019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd019cbb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7febc40092e0 tx=0x7febc400bd70 comp rx=0 tx=0).stop 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 shutdown_connections 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7febb806ecc0 0x7febb8071180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd00ffda0 0x7febd019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 --2- 172.21.15.31:0/2500084481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febd0100750 0x7febd019cbb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 >> 172.21.15.31:0/2500084481 conn(0x7febd00fb650 msgr2=0x7febd0069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 shutdown_connections 2024-04-22T23:51:58.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:58.896+0000 7febd6493700 1 -- 172.21.15.31:0/2500084481 wait complete. 2024-04-22T23:51:58.898 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:51:58.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2887884031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e80ffb40 msgr2=0x7fa1e80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2887884031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e80ffb40 0x7fa1e80fffc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009910 tx=0x7fa1dc009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2887884031 shutdown_connections 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2887884031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e80ffb40 0x7fa1e80fffc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2887884031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e8105bf0 0x7fa1e8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2887884031 >> 172.21.15.31:0/2887884031 conn(0x7fa1e80fb650 msgr2=0x7fa1e80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2887884031 shutdown_connections 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.004+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2887884031 wait complete. 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1ee0d3700 1 Processor -- start 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1ee0d3700 1 -- start start 2024-04-22T23:51:58.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1ee0d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e80ffb40 0x7fa1e819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1ee0d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 0x7fa1e819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1ee0d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e819d1a0 con 0x7fa1e8105bf0 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1ee0d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8196650 con 0x7fa1e80ffb40 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 0x7fa1e819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 0x7fa1e819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51946/0 (socket says 172.21.15.31:51946) 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1e6ffd700 1 -- 172.21.15.31:0/2945416613 learned_addr learned my addr 172.21.15.31:0/2945416613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.005+0000 7fa1e77fe700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e80ffb40 0x7fa1e819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e6ffd700 1 -- 172.21.15.31:0/2945416613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e80ffb40 msgr2=0x7fa1e819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e6ffd700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e80ffb40 0x7fa1e819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e6ffd700 1 -- 172.21.15.31:0/2945416613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0095f0 con 0x7fa1e8105bf0 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e6ffd700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 0x7fa1e819cb10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc006010 tx=0x7fa1dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1dc01d070 con 0x7fa1e8105bf0 2024-04-22T23:51:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1dc005dc0 con 0x7fa1e8105bf0 2024-04-22T23:51:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e81968d0 con 0x7fa1e8105bf0 2024-04-22T23:51:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1dc017400 con 0x7fa1e8105bf0 2024-04-22T23:51:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.006+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8196dc0 con 0x7fa1e8105bf0 2024-04-22T23:51:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.007+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e804f2e0 con 0x7fa1e8105bf0 2024-04-22T23:51:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.008+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa1dc003a60 con 0x7fa1e8105bf0 2024-04-22T23:51:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.008+0000 7fa1e4ff9700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa1d006ecc0 0x7fa1d0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.008+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa1dc090270 con 0x7fa1e8105bf0 2024-04-22T23:51:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.008+0000 7fa1e77fe700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa1d006ecc0 0x7fa1d0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.008+0000 7fa1e77fe700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa1d006ecc0 0x7fa1d0071180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8007be0 tx=0x7fa1d80073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.013+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa1dc05cb10 con 0x7fa1e8105bf0 2024-04-22T23:51:59.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.243+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1e81971e0 con 0x7fa1d006ecc0 2024-04-22T23:51:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.244+0000 7fa1e4ff9700 1 -- 172.21.15.31:0/2945416613 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa1e81971e0 con 0x7fa1d006ecc0 2024-04-22T23:51:59.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa1d006ecc0 msgr2=0x7fa1d0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa1d006ecc0 0x7fa1d0071180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8007be0 tx=0x7fa1d80073d0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 msgr2=0x7fa1e819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 0x7fa1e819cb10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc006010 tx=0x7fa1dc00bd70 comp rx=0 tx=0).stop 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 shutdown_connections 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa1d006ecc0 0x7fa1d0071180 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1e80ffb40 0x7fa1e819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 --2- 172.21.15.31:0/2945416613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1e8105bf0 0x7fa1e819cb10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 >> 172.21.15.31:0/2945416613 conn(0x7fa1e80fb650 msgr2=0x7fa1e8068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 shutdown_connections 2024-04-22T23:51:59.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.246+0000 7fa1ee0d3700 1 -- 172.21.15.31:0/2945416613 wait complete. 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.353+0000 7f224d7cc700 1 -- 172.21.15.31:0/851254785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 msgr2=0x7f2248101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.353+0000 7f224d7cc700 1 --2- 172.21.15.31:0/851254785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f2248101280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2238009910 tx=0x7f2238009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 -- 172.21.15.31:0/851254785 shutdown_connections 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 --2- 172.21.15.31:0/851254785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2248101850 0x7f22481058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 --2- 172.21.15.31:0/851254785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f2248101280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 -- 172.21.15.31:0/851254785 >> 172.21.15.31:0/851254785 conn(0x7f22480fc750 msgr2=0x7f22480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 -- 172.21.15.31:0/851254785 shutdown_connections 2024-04-22T23:51:59.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 -- 172.21.15.31:0/851254785 wait complete. 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 Processor -- start 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.354+0000 7f224d7cc700 1 -- start start 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f224d7cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f224819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f224d7cc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2248101850 0x7f224819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f224d7cc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224819d2c0 con 0x7f2248100ea0 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f224d7cc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248196770 con 0x7f2248101850 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f224819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f224819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51960/0 (socket says 172.21.15.31:51960) 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 -- 172.21.15.31:0/3294101442 learned_addr learned my addr 172.21.15.31:0/3294101442 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f22467fc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2248101850 0x7f224819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 -- 172.21.15.31:0/3294101442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2248101850 msgr2=0x7f224819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2248101850 0x7f224819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 -- 172.21.15.31:0/3294101442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380095f0 con 0x7f2248100ea0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f2246ffd700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f224819c6f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2238005f50 tx=0x7f223800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223801d070 con 0x7f2248100ea0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.355+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f223800b610 con 0x7f2248100ea0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.356+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22481969f0 con 0x7f2248100ea0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.356+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2238022470 con 0x7f2248100ea0 2024-04-22T23:51:59.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.356+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248196ee0 con 0x7f2248100ea0 2024-04-22T23:51:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.357+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248109240 con 0x7f2248100ea0 2024-04-22T23:51:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.357+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2238003780 con 0x7f2248100ea0 2024-04-22T23:51:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.358+0000 7f222ffff700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f223006ecc0 0x7f2230071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.358+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f2238090200 con 0x7f2248100ea0 2024-04-22T23:51:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.358+0000 7f22467fc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f223006ecc0 0x7f2230071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.358+0000 7f22467fc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f223006ecc0 0x7f2230071180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2248197be0 tx=0x7f223c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.363+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f223805caa0 con 0x7f2248100ea0 2024-04-22T23:51:59.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.583+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2248197150 con 0x7f223006ecc0 2024-04-22T23:51:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.594+0000 7f222ffff700 1 -- 172.21.15.31:0/3294101442 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2248197150 con 0x7f223006ecc0 2024-04-22T23:51:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (4m) 19s ago 14m 25.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (15m) 19s ago 15m 8396k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (9m) 79s ago 14m 7964k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (87s) 19s ago 15m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (82s) 79s ago 14m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 19s ago 14m 79.0M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (9m) 19s ago 9m 16.6M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (9m) 19s ago 9m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (9m) 79s ago 9m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (9m) 79s ago 9m 20.0M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (6m) 19s ago 16m 506M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (5m) 79s ago 13m 450M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (2m) 19s ago 16m 48.8M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:51:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (2m) 79s ago 13m 34.9M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (5m) 19s ago 14m 14.7M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (4m) 79s ago 13m 15.8M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (61s) 19s ago 13m 93.1M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (25s) 19s ago 12m 12.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (12m) 19s ago 12m 178M 1217M 18.2.1 5be31c24972a c98ed1b23d72 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (11m) 79s ago 11m 237M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (11m) 79s ago 11m 211M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (10m) 79s ago 10m 196M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:51:59.587 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (4m) 19s ago 14m 73.8M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f223006ecc0 msgr2=0x7f2230071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f223006ecc0 0x7f2230071180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2248197be0 tx=0x7f223c009260 comp rx=0 tx=0).stop 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 msgr2=0x7f224819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f224819c6f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2238005f50 tx=0x7f223800f690 comp rx=0 tx=0).stop 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 shutdown_connections 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f223006ecc0 0x7f2230071180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2248100ea0 0x7f224819c6f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 --2- 172.21.15.31:0/3294101442 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2248101850 0x7f224819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.596+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 >> 172.21.15.31:0/3294101442 conn(0x7f22480fc750 msgr2=0x7f22480fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.597+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 shutdown_connections 2024-04-22T23:51:59.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.597+0000 7f224d7cc700 1 -- 172.21.15.31:0/3294101442 wait complete. 2024-04-22T23:51:59.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3389807256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f0105bd0 msgr2=0x7fb5f0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3389807256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f0105bd0 0x7fb5f0105fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009910 tx=0x7fb5e0009c20 comp rx=0 tx=0).stop 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3389807256 shutdown_connections 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3389807256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f00ffc20 0x7fb5f0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3389807256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f0105bd0 0x7fb5f0105fb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3389807256 >> 172.21.15.31:0/3389807256 conn(0x7fb5f0075670 msgr2=0x7fb5f0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3389807256 shutdown_connections 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.703+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3389807256 wait complete. 2024-04-22T23:51:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5f5ffe700 1 Processor -- start 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5f5ffe700 1 -- start start 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5f5ffe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 0x7fb5f019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5f5ffe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0105bd0 0x7fb5f019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5f5ffe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f019d1f0 con 0x7fb5f00ffc20 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5f5ffe700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01966a0 con 0x7fb5f0105bd0 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 0x7fb5f019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5ef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 0x7fb5f019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51982/0 (socket says 172.21.15.31:51982) 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5ef7fe700 1 -- 172.21.15.31:0/3349439329 learned_addr learned my addr 172.21.15.31:0/3349439329 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5eeffd700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0105bd0 0x7fb5f019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5ef7fe700 1 -- 172.21.15.31:0/3349439329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0105bd0 msgr2=0x7fb5f019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5ef7fe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0105bd0 0x7fb5f019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.704+0000 7fb5ef7fe700 1 -- 172.21.15.31:0/3349439329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e00095f0 con 0x7fb5f00ffc20 2024-04-22T23:51:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.705+0000 7fb5ef7fe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 0x7fb5f019c620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.705+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e001d070 con 0x7fb5f00ffc20 2024-04-22T23:51:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.705+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5e0005dc0 con 0x7fb5f00ffc20 2024-04-22T23:51:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.705+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0196920 con 0x7fb5f00ffc20 2024-04-22T23:51:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.705+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5e0017400 con 0x7fb5f00ffc20 2024-04-22T23:51:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.705+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f0196e10 con 0x7fb5f00ffc20 2024-04-22T23:51:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.706+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f004f2e0 con 0x7fb5f00ffc20 2024-04-22T23:51:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.706+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb5e0003a60 con 0x7fb5f00ffc20 2024-04-22T23:51:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.707+0000 7fb5ecff9700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb5d806ecc0 0x7fb5d8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:51:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.707+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb5e00902c0 con 0x7fb5f00ffc20 2024-04-22T23:51:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.707+0000 7fb5eeffd700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb5d806ecc0 0x7fb5d8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:51:59.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.707+0000 7fb5eeffd700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb5d806ecc0 0x7fb5d8071180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00ffa40 tx=0x7fb5e400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:51:59.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:51:59.712+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb5e005cb60 con 0x7fb5f00ffc20 2024-04-22T23:51:59.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:59 smithi031 ceph-mon[111289]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:51:59.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:51:59 smithi031 ceph-mon[111289]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:51:59.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:59 smithi044 ceph-mon[82146]: pgmap v94: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:51:59.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:51:59 smithi044 ceph-mon[82146]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:00.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.012+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5f01970b0 con 0x7fb5f00ffc20 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.012+0000 7fb5ecff9700 1 -- 172.21.15.31:0/3349439329 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fb5e005c6f0 con 0x7fb5f00ffc20 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:52:00.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:52:00.005 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb5d806ecc0 msgr2=0x7fb5d8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb5d806ecc0 0x7fb5d8071180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00ffa40 tx=0x7fb5e400b410 comp rx=0 tx=0).stop 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 msgr2=0x7fb5f019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 0x7fb5f019c620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006010 tx=0x7fb5e000bd70 comp rx=0 tx=0).stop 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 shutdown_connections 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb5d806ecc0 0x7fb5d8071180 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f00ffc20 0x7fb5f019c620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 --2- 172.21.15.31:0/3349439329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5f0105bd0 0x7fb5f019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 >> 172.21.15.31:0/3349439329 conn(0x7fb5f0075670 msgr2=0x7fb5f0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 shutdown_connections 2024-04-22T23:52:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.015+0000 7fb5f5ffe700 1 -- 172.21.15.31:0/3349439329 wait complete. 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/3550759143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c105bd0 msgr2=0x7ffa9c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/3550759143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c105bd0 0x7ffa9c105fb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009910 tx=0x7ffa90009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/3550759143 shutdown_connections 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/3550759143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/3550759143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c105bd0 0x7ffa9c105fb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/3550759143 >> 172.21.15.31:0/3550759143 conn(0x7ffa9c075670 msgr2=0x7ffa9c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:00.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/3550759143 shutdown_connections 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.126+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/3550759143 wait complete. 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa2f9d700 1 Processor -- start 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa2f9d700 1 -- start start 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa2f9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa2f9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa9c105bd0 0x7ffa9c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa2f9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c1982f0 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa2f9d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c198460 con 0x7ffa9c105bd0 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa0d39700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa0d39700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c1976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51994/0 (socket says 172.21.15.31:51994) 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa0d39700 1 -- 172.21.15.31:0/1562126506 learned_addr learned my addr 172.21.15.31:0/1562126506 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa9c105bd0 0x7ffa9c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa0d39700 1 -- 172.21.15.31:0/1562126506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa9c105bd0 msgr2=0x7ffa9c197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa0d39700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa9c105bd0 0x7ffa9c197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.127+0000 7ffaa0d39700 1 -- 172.21.15.31:0/1562126506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900095f0 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.128+0000 7ffaa0d39700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c1976d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa900092e0 tx=0x7ffa9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.128+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9001d070 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.128+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa90005e00 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.128+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c1a0740 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.128+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa90017400 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.128+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c1a0bb0 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.129+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c1080f0 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.130+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffa9000f460 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.130+0000 7ffa99ffb700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffa8c06ecc0 0x7ffa8c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.130+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ffa900903d0 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.130+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffa8c06ecc0 0x7ffa8c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.131+0000 7ffa9bfff700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffa8c06ecc0 0x7ffa8c071180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa88005950 tx=0x7ffa880058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:00.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.135+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ffa900590c0 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.397+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa9c068c10 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.398+0000 7ffa99ffb700 1 -- 172.21.15.31:0/1562126506 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7ffa9005c780 con 0x7ffa9c0ffc20 2024-04-22T23:52:00.389 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:52:00.389 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:52:00.389 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:52:00.390 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:52:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffa8c06ecc0 msgr2=0x7ffa8c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffa8c06ecc0 0x7ffa8c071180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa88005950 tx=0x7ffa880058e0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 msgr2=0x7ffa9c1976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c1976d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffa900092e0 tx=0x7ffa9000be20 comp rx=0 tx=0).stop 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 shutdown_connections 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffa8c06ecc0 0x7ffa8c071180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.400+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa9c0ffc20 0x7ffa9c1976d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.401+0000 7ffaa2f9d700 1 --2- 172.21.15.31:0/1562126506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa9c105bd0 0x7ffa9c197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.401+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 >> 172.21.15.31:0/1562126506 conn(0x7ffa9c075670 msgr2=0x7ffa9c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.401+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 shutdown_connections 2024-04-22T23:52:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.401+0000 7ffaa2f9d700 1 -- 172.21.15.31:0/1562126506 wait complete. 2024-04-22T23:52:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:52:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.514+0000 7f5be38ba700 1 -- 172.21.15.31:0/287172453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc1027a0 msgr2=0x7f5bdc10ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.514+0000 7f5be38ba700 1 --2- 172.21.15.31:0/287172453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc1027a0 0x7f5bdc10ac90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009910 tx=0x7f5bcc009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.514+0000 7f5be38ba700 1 -- 172.21.15.31:0/287172453 shutdown_connections 2024-04-22T23:52:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.514+0000 7f5be38ba700 1 --2- 172.21.15.31:0/287172453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc1027a0 0x7f5bdc10ac90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.514+0000 7f5be38ba700 1 --2- 172.21.15.31:0/287172453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bdc101e80 0x7f5bdc102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.514+0000 7f5be38ba700 1 -- 172.21.15.31:0/287172453 >> 172.21.15.31:0/287172453 conn(0x7f5bdc0fb650 msgr2=0x7f5bdc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 -- 172.21.15.31:0/287172453 shutdown_connections 2024-04-22T23:52:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 -- 172.21.15.31:0/287172453 wait complete. 2024-04-22T23:52:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 Processor -- start 2024-04-22T23:52:00.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 -- start start 2024-04-22T23:52:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 0x7f5bdc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bdc1027a0 0x7f5bdc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc19d290 con 0x7f5bdc101e80 2024-04-22T23:52:00.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.515+0000 7f5be38ba700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc196740 con 0x7f5bdc1027a0 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 0x7f5bdc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 0x7f5bdc19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52022/0 (socket says 172.21.15.31:52022) 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 -- 172.21.15.31:0/3847674247 learned_addr learned my addr 172.21.15.31:0/3847674247 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be0e55700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bdc1027a0 0x7f5bdc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 -- 172.21.15.31:0/3847674247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bdc1027a0 msgr2=0x7f5bdc19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bdc1027a0 0x7f5bdc19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 -- 172.21.15.31:0/3847674247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0095f0 con 0x7f5bdc101e80 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be1656700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 0x7f5bdc19c6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d8d0 tx=0x7f5bd800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd80098a0 con 0x7f5bdc101e80 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bd8010460 con 0x7f5bdc101e80 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc196a20 con 0x7f5bdc101e80 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd800f5f0 con 0x7f5bdc101e80 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.516+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc196f70 con 0x7f5bdc101e80 2024-04-22T23:52:00.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.517+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc108390 con 0x7f5bdc101e80 2024-04-22T23:52:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.518+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5bd8009a00 con 0x7f5bdc101e80 2024-04-22T23:52:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.518+0000 7f5bd27fc700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5bc806ecc0 0x7f5bc8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.518+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f5bd808ecf0 con 0x7f5bdc101e80 2024-04-22T23:52:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.518+0000 7f5be0e55700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5bc806ecc0 0x7f5bc8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.519+0000 7f5be0e55700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5bc806ecc0 0x7f5bc8071180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc197bb0 tx=0x7f5bcc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:00.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.523+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5bd805b590 con 0x7f5bdc101e80 2024-04-22T23:52:00.645 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: Upgrade: osd.2 is safe to restart 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3349439329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:00.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:00 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1562126506' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:52:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.761+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bdc197d60 con 0x7f5bc806ecc0 2024-04-22T23:52:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.763+0000 7f5bd27fc700 1 -- 172.21.15.31:0/3847674247 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5bdc197d60 con 0x7f5bc806ecc0 2024-04-22T23:52:00.754 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:52:00.754 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:52:00.754 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:52:00.754 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "crash" 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "8/23 daemons upgraded", 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:52:00.755 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5bc806ecc0 msgr2=0x7f5bc8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5bc806ecc0 0x7f5bc8071180 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc197bb0 tx=0x7f5bcc01a040 comp rx=0 tx=0).stop 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 msgr2=0x7f5bdc19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 0x7f5bdc19c6c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d8d0 tx=0x7f5bd800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 shutdown_connections 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5bc806ecc0 0x7f5bc8071180 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bdc101e80 0x7f5bdc19c6c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 --2- 172.21.15.31:0/3847674247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5bdc1027a0 0x7f5bdc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 >> 172.21.15.31:0/3847674247 conn(0x7f5bdc0fb650 msgr2=0x7f5bdc1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.765+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 shutdown_connections 2024-04-22T23:52:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.766+0000 7f5be38ba700 1 -- 172.21.15.31:0/3847674247 wait complete. 2024-04-22T23:52:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 -- 172.21.15.31:0/462837168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 msgr2=0x7fa304105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 --2- 172.21.15.31:0/462837168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304105840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009910 tx=0x7fa2ec009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 -- 172.21.15.31:0/462837168 shutdown_connections 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 --2- 172.21.15.31:0/462837168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304105840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 --2- 172.21.15.31:0/462837168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3040ff5a0 0x7fa3040ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 -- 172.21.15.31:0/462837168 >> 172.21.15.31:0/462837168 conn(0x7fa304074e30 msgr2=0x7fa304075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 -- 172.21.15.31:0/462837168 shutdown_connections 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 -- 172.21.15.31:0/462837168 wait complete. 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 Processor -- start 2024-04-22T23:52:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.880+0000 7fa30a71c700 1 -- start start 2024-04-22T23:52:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa30a71c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3040ff5a0 0x7fa3041969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa30a71c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa30a71c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa304197570 con 0x7fa3040fff50 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa30a71c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419b380 con 0x7fa3040ff5a0 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48734/0 (socket says 172.21.15.31:48734) 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 -- 172.21.15.31:0/4020094149 learned_addr learned my addr 172.21.15.31:0/4020094149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa303fff700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3040ff5a0 0x7fa3041969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 -- 172.21.15.31:0/4020094149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3040ff5a0 msgr2=0x7fa3041969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3040ff5a0 0x7fa3041969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 -- 172.21.15.31:0/4020094149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0095f0 con 0x7fa3040fff50 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3037fe700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304196ee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0092e0 tx=0x7fa2ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ec01d070 con 0x7fa3040fff50 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.881+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2ec00f460 con 0x7fa3040fff50 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.882+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ec0175f0 con 0x7fa3040fff50 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.882+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30419b600 con 0x7fa3040fff50 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.882+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30419baf0 con 0x7fa3040fff50 2024-04-22T23:52:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.882+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa304109220 con 0x7fa3040fff50 2024-04-22T23:52:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.883+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa2ec00f5d0 con 0x7fa3040fff50 2024-04-22T23:52:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.884+0000 7fa3017fa700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa2f006ecc0 0x7fa2f0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.884+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fa2ec090390 con 0x7fa3040fff50 2024-04-22T23:52:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.884+0000 7fa303fff700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa2f006ecc0 0x7fa2f0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.884+0000 7fa303fff700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa2f006ecc0 0x7fa2f0071180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:00.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:00.888+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa2ec05cc30 con 0x7fa3040fff50 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: Upgrade: osd.2 is safe to restart 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3349439329' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:00.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:00 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1562126506' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:52:01.187 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:52:01.187 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:52:01.188 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:52:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.196+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa30404f2e0 con 0x7fa3040fff50 2024-04-22T23:52:01.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.196+0000 7fa3017fa700 1 -- 172.21.15.31:0/4020094149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fa2ec05c7c0 con 0x7fa3040fff50 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa2f006ecc0 msgr2=0x7fa2f0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa2f006ecc0 0x7fa2f0071180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4006fd0 tx=0x7fa2f4008040 comp rx=0 tx=0).stop 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 msgr2=0x7fa304196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304196ee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0092e0 tx=0x7fa2ec00be20 comp rx=0 tx=0).stop 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 shutdown_connections 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa2f006ecc0 0x7fa2f0071180 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3040ff5a0 0x7fa3041969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 --2- 172.21.15.31:0/4020094149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3040fff50 0x7fa304196ee0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 >> 172.21.15.31:0/4020094149 conn(0x7fa304074e30 msgr2=0x7fa3040fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 shutdown_connections 2024-04-22T23:52:01.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:01.198+0000 7fa30a71c700 1 -- 172.21.15.31:0/4020094149 wait complete. 2024-04-22T23:52:01.762 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:01 smithi031 ceph-mon[111289]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:52:01.763 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:01 smithi031 ceph-mon[111289]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:01.763 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:01 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4020094149' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:52:01.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:01 smithi044 ceph-mon[82146]: pgmap v95: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:52:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:01 smithi044 ceph-mon[82146]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:01.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:01 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4020094149' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:52:03.093 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:02 smithi031 ceph-mon[111289]: Upgrade: Updating osd.2 2024-04-22T23:52:03.094 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:03.094 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:52:03.094 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:03.094 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:02 smithi031 ceph-mon[111289]: Deploying daemon osd.2 on smithi031 2024-04-22T23:52:03.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:02 smithi044 ceph-mon[82146]: Upgrade: Updating osd.2 2024-04-22T23:52:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:52:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:03.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:02 smithi044 ceph-mon[82146]: Deploying daemon osd.2 on smithi031 2024-04-22T23:52:04.098 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:03 smithi031 ceph-mon[111289]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:52:04.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:03 smithi044 ceph-mon[82146]: pgmap v96: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:52:05.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:05 smithi031 systemd[1]: Stopping Ceph osd.2 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:52:05.693 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:05 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[60808]: 2024-04-22T23:52:05.364+0000 7f9fc6c1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:52:05.693 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:05 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[60808]: 2024-04-22T23:52:05.364+0000 7f9fc6c1c700 -1 osd.2 66 *** Got signal Terminated *** 2024-04-22T23:52:05.693 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:05 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[60808]: 2024-04-22T23:52:05.364+0000 7f9fc6c1c700 -1 osd.2 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:52:06.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:05 smithi031 ceph-mon[111289]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:06.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:05 smithi031 ceph-mon[111289]: osd.2 marked itself down and dead 2024-04-22T23:52:06.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:05 smithi044 ceph-mon[82146]: pgmap v97: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:06.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:05 smithi044 ceph-mon[82146]: osd.2 marked itself down and dead 2024-04-22T23:52:06.565 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:06 smithi031 podman[133853]: 2024-04-22 23:52:06.275134573 +0000 UTC m=+1.037534622 container died c98ed1b23d7230b5d12f2a35332f2f5fd5d294e6885244ed1681e9ab54afb152 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:52:06.829 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:06 smithi031 podman[133853]: 2024-04-22 23:52:06.689621763 +0000 UTC m=+1.452021809 container cleanup c98ed1b23d7230b5d12f2a35332f2f5fd5d294e6885244ed1681e9ab54afb152 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, RELEASE=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-22T23:52:06.829 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:06 smithi031 bash[133853]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2 2024-04-22T23:52:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:06 smithi031 ceph-mon[111289]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:52:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:06 smithi031 ceph-mon[111289]: osdmap e67: 6 total, 5 up, 6 in 2024-04-22T23:52:07.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:06 smithi031 podman[133870]: 2024-04-22 23:52:06.891179131 +0000 UTC m=+0.611197470 container remove c98ed1b23d7230b5d12f2a35332f2f5fd5d294e6885244ed1681e9ab54afb152 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-04-22T23:52:07.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:06 smithi044 ceph-mon[82146]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:52:07.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:06 smithi044 ceph-mon[82146]: osdmap e67: 6 total, 5 up, 6 in 2024-04-22T23:52:07.585 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:07 smithi031 podman[133962]: 2024-04-22 23:52:07.234087968 +0000 UTC m=+0.019062950 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:52:07.586 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:07 smithi031 podman[133962]: 2024-04-22 23:52:07.350371616 +0000 UTC m=+0.135346596 container create eeb4dafdfa0a38d66ebd3836577e34606b0627671234033b33502386c8f6519c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-22T23:52:07.586 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:07 smithi031 podman[133962]: 2024-04-22 23:52:07.595024955 +0000 UTC m=+0.379999953 container init eeb4dafdfa0a38d66ebd3836577e34606b0627671234033b33502386c8f6519c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-22T23:52:07.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:07 smithi031 ceph-mon[111289]: pgmap v99: 65 pgs: 11 peering, 8 stale+active+clean, 46 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-22T23:52:07.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:07 smithi031 ceph-mon[111289]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:52:07.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:07 smithi031 ceph-mon[111289]: osdmap e68: 6 total, 5 up, 6 in 2024-04-22T23:52:07.899 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:07 smithi031 podman[133962]: 2024-04-22 23:52:07.598694689 +0000 UTC m=+0.383669679 container start eeb4dafdfa0a38d66ebd3836577e34606b0627671234033b33502386c8f6519c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5) 2024-04-22T23:52:07.899 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:07 smithi031 podman[133962]: 2024-04-22 23:52:07.694471806 +0000 UTC m=+0.479446789 container attach eeb4dafdfa0a38d66ebd3836577e34606b0627671234033b33502386c8f6519c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-22T23:52:08.152 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:07 smithi031 podman[133962]: 2024-04-22 23:52:07.909090281 +0000 UTC m=+0.694065272 container died eeb4dafdfa0a38d66ebd3836577e34606b0627671234033b33502386c8f6519c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:52:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:07 smithi044 ceph-mon[82146]: pgmap v99: 65 pgs: 11 peering, 8 stale+active+clean, 46 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-22T23:52:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:07 smithi044 ceph-mon[82146]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:52:08.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:07 smithi044 ceph-mon[82146]: osdmap e68: 6 total, 5 up, 6 in 2024-04-22T23:52:08.935 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 podman[133988]: 2024-04-22 23:52:08.483723169 +0000 UTC m=+0.568485651 container remove eeb4dafdfa0a38d66ebd3836577e34606b0627671234033b33502386c8f6519c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:52:08.935 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-22T23:52:08.935 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 systemd[1]: Stopped Ceph osd.2 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:52:08.935 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.2.service: Consumed 48.498s CPU time. 2024-04-22T23:52:08.936 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 systemd[1]: Starting Ceph osd.2 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:52:09.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 podman[134096]: 2024-04-22 23:52:08.847261172 +0000 UTC m=+0.019451376 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:52:09.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:08 smithi031 podman[134096]: 2024-04-22 23:52:08.967192003 +0000 UTC m=+0.139382200 container create eb80ca88827be35db8c3215374161200e2d310f200bce18a79a23aa21fc8fc89 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:52:09.538 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 podman[134096]: 2024-04-22 23:52:09.261202714 +0000 UTC m=+0.433392918 container init eb80ca88827be35db8c3215374161200e2d310f200bce18a79a23aa21fc8fc89 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:52:09.538 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 podman[134096]: 2024-04-22 23:52:09.264891141 +0000 UTC m=+0.437081345 container start eb80ca88827be35db8c3215374161200e2d310f200bce18a79a23aa21fc8fc89 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-22T23:52:09.538 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 podman[134096]: 2024-04-22 23:52:09.343994437 +0000 UTC m=+0.516184649 container attach eb80ca88827be35db8c3215374161200e2d310f200bce18a79a23aa21fc8fc89 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:52:09.805 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:09.805 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 bash[134096]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:52:09.806 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:52:09.807 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-22T23:52:09.807 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:52:09.807 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate[134111]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-22T23:52:09.807 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:09 smithi031 podman[134096]: 2024-04-22 23:52:09.81468017 +0000 UTC m=+0.986870373 container died eb80ca88827be35db8c3215374161200e2d310f200bce18a79a23aa21fc8fc89 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T23:52:10.164 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:09 smithi031 ceph-mon[111289]: pgmap v101: 65 pgs: 11 peering, 8 stale+active+clean, 46 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:52:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:09 smithi044 ceph-mon[82146]: pgmap v101: 65 pgs: 11 peering, 8 stale+active+clean, 46 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:52:10.832 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:10 smithi031 podman[134252]: 2024-04-22 23:52:10.50024047 +0000 UTC m=+0.679647319 container remove eb80ca88827be35db8c3215374161200e2d310f200bce18a79a23aa21fc8fc89 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T23:52:10.832 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:10 smithi031 podman[134319]: 2024-04-22 23:52:10.620872764 +0000 UTC m=+0.018765941 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:52:10.832 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:10 smithi031 podman[134319]: 2024-04-22 23:52:10.724408831 +0000 UTC m=+0.122301981 container create 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-22T23:52:11.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:10 smithi031 ceph-mon[111289]: Health check failed: Degraded data redundancy: 10/264 objects degraded (3.788%), 4 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:11.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:10 smithi031 podman[134319]: 2024-04-22 23:52:10.977380768 +0000 UTC m=+0.375273931 container init 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T23:52:11.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:10 smithi031 podman[134319]: 2024-04-22 23:52:10.980864307 +0000 UTC m=+0.378757460 container start 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-22T23:52:11.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:11 smithi031 ceph-osd[134338]: -- 172.21.15.31:0/286580028 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555898a15680 con 0x5558989df000 2024-04-22T23:52:11.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:11 smithi031 bash[134319]: 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 2024-04-22T23:52:11.194 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:11 smithi031 systemd[1]: Started Ceph osd.2 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:52:11.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:10 smithi044 ceph-mon[82146]: Health check failed: Degraded data redundancy: 10/264 objects degraded (3.788%), 4 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:11.886 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:11 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-22T23:52:11.532+0000 7f02c82927c0 -1 Falling back to public interface 2024-04-22T23:52:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:11 smithi031 ceph-mon[111289]: pgmap v102: 65 pgs: 4 active+undersized, 4 active+undersized+degraded, 11 peering, 5 stale+active+clean, 41 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 10/264 objects degraded (3.788%) 2024-04-22T23:52:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:12.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:12.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:11 smithi044 ceph-mon[82146]: pgmap v102: 65 pgs: 4 active+undersized, 4 active+undersized+degraded, 11 peering, 5 stale+active+clean, 41 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 10/264 objects degraded (3.788%) 2024-04-22T23:52:12.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:12.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:12.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:13.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:12 smithi031 ceph-mon[111289]: pgmap v103: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:13.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:12 smithi031 ceph-mon[111289]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:52:13.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:12 smithi044 ceph-mon[82146]: pgmap v103: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:13.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:12 smithi044 ceph-mon[82146]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:52:15.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:15 smithi031 ceph-mon[111289]: pgmap v104: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:15.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:15.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:15.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:15 smithi044 ceph-mon[82146]: pgmap v104: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:15.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:15.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:15.943 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:15 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-22T23:52:15.679+0000 7f02c82927c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-22T23:52:16.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:16 smithi031 ceph-mon[111289]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:52:16.444 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:15 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-22T23:52:15.971+0000 7f02c82927c0 -1 osd.2 66 log_to_monitors true 2024-04-22T23:52:16.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:16 smithi044 ceph-mon[82146]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:52:17.443 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-22T23:52:17.146+0000 7f02b56e2700 -1 osd.2 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: pgmap v105: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: osdmap e69: 6 total, 5 up, 6 in 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 37/264 objects degraded (14.015%), 13 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:17 smithi031 ceph-mon[111289]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' 2024-04-22T23:52:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: pgmap v105: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:52:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: osdmap e69: 6 total, 5 up, 6 in 2024-04-22T23:52:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-22T23:52:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 37/264 objects degraded (14.015%), 13 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:17.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:17.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:17 smithi044 ceph-mon[82146]: from='osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235]' entity='osd.2' 2024-04-22T23:52:18.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:18.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:18.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:18 smithi031 ceph-mon[111289]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:52:18.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:18 smithi031 ceph-mon[111289]: osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235] boot 2024-04-22T23:52:18.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:18 smithi031 ceph-mon[111289]: osdmap e70: 6 total, 6 up, 6 in 2024-04-22T23:52:18.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:18 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:52:19.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:19.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:19.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:18 smithi044 ceph-mon[82146]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:52:19.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:18 smithi044 ceph-mon[82146]: osd.2 [v2:172.21.15.31:6818/1746177235,v1:172.21.15.31:6819/1746177235] boot 2024-04-22T23:52:19.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:18 smithi044 ceph-mon[82146]: osdmap e70: 6 total, 6 up, 6 in 2024-04-22T23:52:19.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:18 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:52:19.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:19 smithi031 ceph-mon[111289]: pgmap v107: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:19.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:19 smithi031 ceph-mon[111289]: osdmap e71: 6 total, 6 up, 6 in 2024-04-22T23:52:20.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:19 smithi044 ceph-mon[82146]: pgmap v107: 65 pgs: 15 active+undersized, 13 active+undersized+degraded, 37 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 37/264 objects degraded (14.015%) 2024-04-22T23:52:20.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:19 smithi044 ceph-mon[82146]: osdmap e71: 6 total, 6 up, 6 in 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: pgmap v110: 65 pgs: 12 active+undersized, 11 active+undersized+degraded, 42 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 33/264 objects degraded (12.500%) 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:22.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:22.195 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:21 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: pgmap v110: 65 pgs: 12 active+undersized, 11 active+undersized+degraded, 42 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 33/264 objects degraded (12.500%) 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:22.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:21 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:23.112 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:22 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:23.112 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:22 smithi044 ceph-mon[82146]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-22T23:52:23.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:22 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:23.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:22 smithi031 ceph-mon[111289]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-22T23:52:24.127 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:23 smithi044 ceph-mon[82146]: pgmap v111: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-22T23:52:24.127 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:23 smithi044 ceph-mon[82146]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded) 2024-04-22T23:52:24.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:23 smithi031 ceph-mon[111289]: pgmap v111: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-22T23:52:24.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:23 smithi031 ceph-mon[111289]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/264 objects degraded (12.500%), 11 pgs degraded) 2024-04-22T23:52:26.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:25 smithi031 ceph-mon[111289]: pgmap v112: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-22T23:52:26.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:25 smithi044 ceph-mon[82146]: pgmap v112: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-22T23:52:28.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:27 smithi031 ceph-mon[111289]: pgmap v113: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:28.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:27 smithi044 ceph-mon[82146]: pgmap v113: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:30.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:29 smithi031 ceph-mon[111289]: pgmap v114: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:30.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:29 smithi044 ceph-mon[82146]: pgmap v114: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 -- 172.21.15.31:0/3765132767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 msgr2=0x7f463410cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 --2- 172.21.15.31:0/3765132767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463410cfa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4628009910 tx=0x7f4628009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 -- 172.21.15.31:0/3765132767 shutdown_connections 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 --2- 172.21.15.31:0/3765132767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463410cfa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 --2- 172.21.15.31:0/3765132767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4634073160 0x7f4634073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 -- 172.21.15.31:0/3765132767 >> 172.21.15.31:0/3765132767 conn(0x7f46340fc750 msgr2=0x7f46340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 -- 172.21.15.31:0/3765132767 shutdown_connections 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 -- 172.21.15.31:0/3765132767 wait complete. 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.319+0000 7f463c3d9700 1 Processor -- start 2024-04-22T23:52:31.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f463c3d9700 1 -- start start 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f463c3d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4634073160 0x7f463419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f463c3d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f463c3d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463419f370 con 0x7f4634073a80 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f463c3d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341988b0 con 0x7f4634073160 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48980/0 (socket says 172.21.15.31:48980) 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 -- 172.21.15.31:0/1360554745 learned_addr learned my addr 172.21.15.31:0/1360554745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f463a175700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4634073160 0x7f463419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 -- 172.21.15.31:0/1360554745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4634073160 msgr2=0x7f463419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4634073160 0x7f463419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 -- 172.21.15.31:0/1360554745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46280095f0 con 0x7f4634073a80 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.320+0000 7f4639974700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463419ed70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4628006010 tx=0x7f462800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.321+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462801d070 con 0x7f4634073a80 2024-04-22T23:52:31.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.321+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f462800f460 con 0x7f4634073a80 2024-04-22T23:52:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.321+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4634198b30 con 0x7f4634073a80 2024-04-22T23:52:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.321+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46280175f0 con 0x7f4634073a80 2024-04-22T23:52:31.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.321+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4634199020 con 0x7f4634073a80 2024-04-22T23:52:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.322+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463410a6a0 con 0x7f4634073a80 2024-04-22T23:52:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.322+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f462800f5d0 con 0x7f4634073a80 2024-04-22T23:52:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.323+0000 7f46337fe700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f461c06ecc0 0x7f461c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.323+0000 7f463a175700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f461c06ecc0 0x7f461c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.323+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f4628090390 con 0x7f4634073a80 2024-04-22T23:52:31.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.323+0000 7f463a175700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f461c06ecc0 0x7f461c071180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f462400f440 tx=0x7f46240079e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:31.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.328+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f462805cc30 con 0x7f4634073a80 2024-04-22T23:52:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:31 smithi031 ceph-mon[111289]: pgmap v115: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:31 smithi044 ceph-mon[82146]: pgmap v115: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:31.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:31.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.562+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4634199e10 con 0x7f461c06ecc0 2024-04-22T23:52:31.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.563+0000 7f46337fe700 1 -- 172.21.15.31:0/1360554745 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4634199e10 con 0x7f461c06ecc0 2024-04-22T23:52:31.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f461c06ecc0 msgr2=0x7f461c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f461c06ecc0 0x7f461c071180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f462400f440 tx=0x7f46240079e0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 msgr2=0x7f463419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463419ed70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4628006010 tx=0x7f462800bd70 comp rx=0 tx=0).stop 2024-04-22T23:52:31.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 shutdown_connections 2024-04-22T23:52:31.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f461c06ecc0 0x7f461c071180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4634073160 0x7f463419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 --2- 172.21.15.31:0/1360554745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4634073a80 0x7f463419ed70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 >> 172.21.15.31:0/1360554745 conn(0x7f46340fc750 msgr2=0x7f46341077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:31.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 shutdown_connections 2024-04-22T23:52:31.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.565+0000 7f463c3d9700 1 -- 172.21.15.31:0/1360554745 wait complete. 2024-04-22T23:52:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:52:31.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2751965978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48101840 msgr2=0x7fef48105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2751965978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48101840 0x7fef48105890 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009910 tx=0x7fef3c009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:31.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2751965978 shutdown_connections 2024-04-22T23:52:31.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2751965978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48101840 0x7fef48105890 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2751965978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef48100e90 0x7fef48101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2751965978 >> 172.21.15.31:0/2751965978 conn(0x7fef480fc740 msgr2=0x7fef480feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:31.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2751965978 shutdown_connections 2024-04-22T23:52:31.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2751965978 wait complete. 2024-04-22T23:52:31.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 Processor -- start 2024-04-22T23:52:31.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.674+0000 7fef4e9f8700 1 -- start start 2024-04-22T23:52:31.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef4e9f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 0x7fef48073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef4e9f8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef48101840 0x7fef480735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef4e9f8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef48074e40 con 0x7fef48100e90 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef4e9f8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef48074fb0 con 0x7fef48101840 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef47fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 0x7fef48073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef47fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 0x7fef48073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48996/0 (socket says 172.21.15.31:48996) 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef47fff700 1 -- 172.21.15.31:0/2024707269 learned_addr learned my addr 172.21.15.31:0/2024707269 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef477fe700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef48101840 0x7fef480735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef47fff700 1 -- 172.21.15.31:0/2024707269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef48101840 msgr2=0x7fef480735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef47fff700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef48101840 0x7fef480735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.675+0000 7fef47fff700 1 -- 172.21.15.31:0/2024707269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c0095f0 con 0x7fef48100e90 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.676+0000 7fef47fff700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 0x7fef48073090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d900 tx=0x7fef3800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.676+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef38004a00 con 0x7fef48100e90 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.676+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef380054e0 con 0x7fef48100e90 2024-04-22T23:52:31.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.676+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef48073bd0 con 0x7fef48100e90 2024-04-22T23:52:31.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.676+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef38009d00 con 0x7fef48100e90 2024-04-22T23:52:31.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.676+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef481a73e0 con 0x7fef48100e90 2024-04-22T23:52:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.677+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef481091a0 con 0x7fef48100e90 2024-04-22T23:52:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.677+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fef38005020 con 0x7fef48100e90 2024-04-22T23:52:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.678+0000 7fef457fa700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fef3006ecc0 0x7fef30071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.678+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fef3808eda0 con 0x7fef48100e90 2024-04-22T23:52:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.678+0000 7fef477fe700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fef3006ecc0 0x7fef30071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:31.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.678+0000 7fef477fe700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fef3006ecc0 0x7fef30071180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006010 tx=0x7fef3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:31.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.684+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fef38057b10 con 0x7fef48100e90 2024-04-22T23:52:31.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.914+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef480746a0 con 0x7fef3006ecc0 2024-04-22T23:52:31.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.915+0000 7fef457fa700 1 -- 172.21.15.31:0/2024707269 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fef480746a0 con 0x7fef3006ecc0 2024-04-22T23:52:31.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fef3006ecc0 msgr2=0x7fef30071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fef3006ecc0 0x7fef30071180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef3c006010 tx=0x7fef3c01a040 comp rx=0 tx=0).stop 2024-04-22T23:52:31.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 msgr2=0x7fef48073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:31.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 0x7fef48073090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d900 tx=0x7fef3800dc10 comp rx=0 tx=0).stop 2024-04-22T23:52:31.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 shutdown_connections 2024-04-22T23:52:31.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fef3006ecc0 0x7fef30071180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef48100e90 0x7fef48073090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 --2- 172.21.15.31:0/2024707269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef48101840 0x7fef480735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:31.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 >> 172.21.15.31:0/2024707269 conn(0x7fef480fc740 msgr2=0x7fef48105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:31.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 shutdown_connections 2024-04-22T23:52:31.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:31.918+0000 7fef4e9f8700 1 -- 172.21.15.31:0/2024707269 wait complete. 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 -- 172.21.15.31:0/395766763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700ffc20 msgr2=0x7fd670100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 --2- 172.21.15.31:0/395766763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700ffc20 0x7fd670100080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd660009910 tx=0x7fd660009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 -- 172.21.15.31:0/395766763 shutdown_connections 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 --2- 172.21.15.31:0/395766763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6700ffc20 0x7fd670100080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 --2- 172.21.15.31:0/395766763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd670105bd0 0x7fd670105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 -- 172.21.15.31:0/395766763 >> 172.21.15.31:0/395766763 conn(0x7fd670075670 msgr2=0x7fd670075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 -- 172.21.15.31:0/395766763 shutdown_connections 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.025+0000 7fd67860d700 1 -- 172.21.15.31:0/395766763 wait complete. 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd67860d700 1 Processor -- start 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd67860d700 1 -- start start 2024-04-22T23:52:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd67860d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6700ffc20 0x7fd670195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd67860d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 0x7fd670195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd67860d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670069520 con 0x7fd670105bd0 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd67860d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670195fd0 con 0x7fd6700ffc20 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd675ba8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 0x7fd670195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd675ba8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 0x7fd670195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49012/0 (socket says 172.21.15.31:49012) 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd675ba8700 1 -- 172.21.15.31:0/3634989197 learned_addr learned my addr 172.21.15.31:0/3634989197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.026+0000 7fd6763a9700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6700ffc20 0x7fd670195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd675ba8700 1 -- 172.21.15.31:0/3634989197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6700ffc20 msgr2=0x7fd670195550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd675ba8700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6700ffc20 0x7fd670195550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd675ba8700 1 -- 172.21.15.31:0/3634989197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6600095f0 con 0x7fd670105bd0 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd6763a9700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6700ffc20 0x7fd670195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:52:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd675ba8700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 0x7fd670195a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b5c0 tx=0x7fd66000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66001d070 con 0x7fd670105bd0 2024-04-22T23:52:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd66000f460 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd670196250 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd660017610 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.027+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6701a2d90 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.028+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6701080d0 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.028+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd66000f5d0 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.029+0000 7fd6677fe700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd65c06ecc0 0x7fd65c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.029+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fd660090510 con 0x7fd670105bd0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.029+0000 7fd6763a9700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd65c06ecc0 0x7fd65c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.029+0000 7fd6763a9700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd65c06ecc0 0x7fd65c071180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd66c005fd0 tx=0x7fd66c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.034+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fd660059200 con 0x7fd670105bd0 2024-04-22T23:52:32.156 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:32 smithi031 ceph-mon[111289]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:32.156 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:32 smithi031 ceph-mon[111289]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:32.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.255+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd670068be0 con 0x7fd65c06ecc0 2024-04-22T23:52:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.266+0000 7fd6677fe700 1 -- 172.21.15.31:0/3634989197 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd670068be0 con 0x7fd65c06ecc0 2024-04-22T23:52:32.257 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:52:32.257 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (4m) 15s ago 15m 25.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:52:32.257 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (15m) 15s ago 15m 8594k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:52:32.257 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (9m) 111s ago 14m 7964k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (2m) 15s ago 15m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (115s) 111s ago 14m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (4m) 15s ago 15m 80.1M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (10m) 15s ago 9m 16.9M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (9m) 15s ago 9m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (9m) 111s ago 9m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (9m) 111s ago 9m 20.0M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (7m) 15s ago 16m 508M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (5m) 111s ago 14m 450M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (3m) 15s ago 16m 50.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (2m) 111s ago 14m 34.9M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (5m) 15s ago 15m 15.1M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (5m) 111s ago 14m 15.8M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:52:32.258 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (94s) 15s ago 13m 96.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (58s) 15s ago 13m 126M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (21s) 15s ago 12m 15.1M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (12m) 111s ago 12m 237M 2241M 18.2.1 5be31c24972a 393d4900ec3c 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (11m) 111s ago 11m 211M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (11m) 111s ago 11m 196M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 15s ago 14m 76.0M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:52:32.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd65c06ecc0 msgr2=0x7fd65c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd65c06ecc0 0x7fd65c071180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd66c005fd0 tx=0x7fd66c005e30 comp rx=0 tx=0).stop 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 msgr2=0x7fd670195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 0x7fd670195a90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b5c0 tx=0x7fd66000be00 comp rx=0 tx=0).stop 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 shutdown_connections 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd65c06ecc0 0x7fd65c071180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6700ffc20 0x7fd670195550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 --2- 172.21.15.31:0/3634989197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd670105bd0 0x7fd670195a90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 >> 172.21.15.31:0/3634989197 conn(0x7fd670075670 msgr2=0x7fd6700fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 shutdown_connections 2024-04-22T23:52:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.268+0000 7fd67860d700 1 -- 172.21.15.31:0/3634989197 wait complete. 2024-04-22T23:52:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 -- 172.21.15.31:0/2946305541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 msgr2=0x7f5f2c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 --2- 172.21.15.31:0/2946305541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c1070b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009910 tx=0x7f5f28009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 -- 172.21.15.31:0/2946305541 shutdown_connections 2024-04-22T23:52:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 --2- 172.21.15.31:0/2946305541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f2c068580 0x7f5f2c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 --2- 172.21.15.31:0/2946305541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c1070b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 -- 172.21.15.31:0/2946305541 >> 172.21.15.31:0/2946305541 conn(0x7f5f2c075670 msgr2=0x7f5f2c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 -- 172.21.15.31:0/2946305541 shutdown_connections 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.374+0000 7f5f33276700 1 -- 172.21.15.31:0/2946305541 wait complete. 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f33276700 1 Processor -- start 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f33276700 1 -- start start 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f33276700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f2c068580 0x7f5f2c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f33276700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f33276700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c19d230 con 0x7f5f2c106cd0 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f33276700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1966e0 con 0x7f5f2c068580 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f30811700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f30811700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49034/0 (socket says 172.21.15.31:49034) 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f30811700 1 -- 172.21.15.31:0/1415681597 learned_addr learned my addr 172.21.15.31:0/1415681597 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:32.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.375+0000 7f5f31012700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f2c068580 0x7f5f2c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f30811700 1 -- 172.21.15.31:0/1415681597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f2c068580 msgr2=0x7f5f2c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f30811700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f2c068580 0x7f5f2c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f30811700 1 -- 172.21.15.31:0/1415681597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f280095f0 con 0x7f5f2c106cd0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f30811700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c19cba0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00d8d0 tx=0x7f5f1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1c009940 con 0x7f5f2c106cd0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f1c010460 con 0x7f5f2c106cd0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c1969c0 con 0x7f5f2c106cd0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1c00f5f0 con 0x7f5f2c106cd0 2024-04-22T23:52:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.376+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c196f10 con 0x7f5f2c106cd0 2024-04-22T23:52:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.377+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c1091d0 con 0x7f5f2c106cd0 2024-04-22T23:52:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.378+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5f1c0105d0 con 0x7f5f2c106cd0 2024-04-22T23:52:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.378+0000 7f5f227fc700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5f1806ecc0 0x7f5f18071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.378+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f5f1c08ee70 con 0x7f5f2c106cd0 2024-04-22T23:52:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.378+0000 7f5f31012700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5f1806ecc0 0x7f5f18071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.379+0000 7f5f31012700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5f1806ecc0 0x7f5f18071180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800b5c0 tx=0x7f5f2801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:32.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.383+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5f1c05b6d0 con 0x7f5f2c106cd0 2024-04-22T23:52:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:32 smithi044 ceph-mon[82146]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:32.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:32 smithi044 ceph-mon[82146]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.686+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f2c197180 con 0x7f5f2c106cd0 2024-04-22T23:52:32.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.686+0000 7f5f227fc700 1 -- 172.21.15.31:0/1415681597 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f5f1c05b260 con 0x7f5f2c106cd0 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 3 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 7 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:52:32.679 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5f1806ecc0 msgr2=0x7f5f18071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5f1806ecc0 0x7f5f18071180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800b5c0 tx=0x7f5f2801a040 comp rx=0 tx=0).stop 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 msgr2=0x7f5f2c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c19cba0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00d8d0 tx=0x7f5f1c00dc90 comp rx=0 tx=0).stop 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 shutdown_connections 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5f1806ecc0 0x7f5f18071180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f2c068580 0x7f5f2c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 --2- 172.21.15.31:0/1415681597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c106cd0 0x7f5f2c19cba0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.688+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 >> 172.21.15.31:0/1415681597 conn(0x7f5f2c075670 msgr2=0x7f5f2c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.689+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 shutdown_connections 2024-04-22T23:52:32.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.689+0000 7f5f33276700 1 -- 172.21.15.31:0/1415681597 wait complete. 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/223914177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073a80 msgr2=0x7f3d6810cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/223914177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073a80 0x7f3d6810cf90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009910 tx=0x7f3d50009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/223914177 shutdown_connections 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/223914177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073a80 0x7f3d6810cf90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/223914177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d68073160 0x7f3d68073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/223914177 >> 172.21.15.31:0/223914177 conn(0x7f3d680fc740 msgr2=0x7f3d680feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:32.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/223914177 shutdown_connections 2024-04-22T23:52:32.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.795+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/223914177 wait complete. 2024-04-22T23:52:32.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d6e5d3700 1 Processor -- start 2024-04-22T23:52:32.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d6e5d3700 1 -- start start 2024-04-22T23:52:32.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d6e5d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 0x7f3d6819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d6e5d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d68073a80 0x7f3d6819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d6e5d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819f400 con 0x7f3d68073160 2024-04-22T23:52:32.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d6e5d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d681988b0 con 0x7f3d68073a80 2024-04-22T23:52:32.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 0x7f3d6819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 0x7f3d6819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49058/0 (socket says 172.21.15.31:49058) 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d67fff700 1 -- 172.21.15.31:0/692539363 learned_addr learned my addr 172.21.15.31:0/692539363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.796+0000 7f3d677fe700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d68073a80 0x7f3d6819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d67fff700 1 -- 172.21.15.31:0/692539363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d68073a80 msgr2=0x7f3d6819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d67fff700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d68073a80 0x7f3d6819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d67fff700 1 -- 172.21.15.31:0/692539363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500095f0 con 0x7f3d68073160 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d67fff700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 0x7f3d6819e830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800d8d0 tx=0x7f3d5800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d580098a0 con 0x7f3d68073160 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d58010460 con 0x7f3d68073160 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d68198b90 con 0x7f3d68073160 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d5800f5f0 con 0x7f3d68073160 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.797+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d681990b0 con 0x7f3d68073160 2024-04-22T23:52:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.798+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6810a690 con 0x7f3d68073160 2024-04-22T23:52:32.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.799+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3d58009a00 con 0x7f3d68073160 2024-04-22T23:52:32.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.799+0000 7f3d657fa700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d5406ecc0 0x7f3d54071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:32.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.799+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f3d5808edd0 con 0x7f3d68073160 2024-04-22T23:52:32.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.799+0000 7f3d677fe700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d5406ecc0 0x7f3d54071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:32.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.800+0000 7f3d677fe700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d5406ecc0 0x7f3d54071180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006010 tx=0x7f3d5001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:32.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:32.804+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f3d5805b5f0 con 0x7f3d68073160 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.061+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d681992d0 con 0x7f3d68073160 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.062+0000 7f3d657fa700 1 -- 172.21.15.31:0/692539363 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f3d5805b180 con 0x7f3d68073160 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:33.054 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:52:33.055 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:33.056 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:33.057 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:52:33.057 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:52:33.057 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:33.057 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d5406ecc0 msgr2=0x7f3d54071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d5406ecc0 0x7f3d54071180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006010 tx=0x7f3d5001a040 comp rx=0 tx=0).stop 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 msgr2=0x7f3d6819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 0x7f3d6819e830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800d8d0 tx=0x7f3d5800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 shutdown_connections 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d5406ecc0 0x7f3d54071180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d68073160 0x7f3d6819e830 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 --2- 172.21.15.31:0/692539363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d68073a80 0x7f3d6819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 >> 172.21.15.31:0/692539363 conn(0x7f3d680fc740 msgr2=0x7f3d681077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 shutdown_connections 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.064+0000 7f3d6e5d3700 1 -- 172.21.15.31:0/692539363 wait complete. 2024-04-22T23:52:33.058 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:52:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.172+0000 7fb03aef1700 1 -- 172.21.15.31:0/4166522722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034101850 msgr2=0x7fb0341058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.172+0000 7fb03aef1700 1 --2- 172.21.15.31:0/4166522722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034101850 0x7fb0341058a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b0a0 tx=0x7fb02400b3b0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.172+0000 7fb03aef1700 1 -- 172.21.15.31:0/4166522722 shutdown_connections 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.172+0000 7fb03aef1700 1 --2- 172.21.15.31:0/4166522722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034101850 0x7fb0341058a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.172+0000 7fb03aef1700 1 --2- 172.21.15.31:0/4166522722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb034100ea0 0x7fb034101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.172+0000 7fb03aef1700 1 -- 172.21.15.31:0/4166522722 >> 172.21.15.31:0/4166522722 conn(0x7fb0340fc750 msgr2=0x7fb0340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 -- 172.21.15.31:0/4166522722 shutdown_connections 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 -- 172.21.15.31:0/4166522722 wait complete. 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 Processor -- start 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 -- start start 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 0x7fb034073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb034101850 0x7fb0340735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034073b10 con 0x7fb034100ea0 2024-04-22T23:52:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb03aef1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034073c50 con 0x7fb034101850 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb038c8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 0x7fb034073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb038c8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 0x7fb034073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49066/0 (socket says 172.21.15.31:49066) 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.173+0000 7fb038c8d700 1 -- 172.21.15.31:0/1042819952 learned_addr learned my addr 172.21.15.31:0/1042819952 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb033fff700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb034101850 0x7fb0340735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb038c8d700 1 -- 172.21.15.31:0/1042819952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb034101850 msgr2=0x7fb0340735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb038c8d700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb034101850 0x7fb0340735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb038c8d700 1 -- 172.21.15.31:0/1042819952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024009d00 con 0x7fb034100ea0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb038c8d700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 0x7fb034073090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00d8d0 tx=0x7fb02c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02c0098a0 con 0x7fb034100ea0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0341a2b00 con 0x7fb034100ea0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb02c010460 con 0x7fb034100ea0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02c00f5f0 con 0x7fb034100ea0 2024-04-22T23:52:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.174+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0341a2eb0 con 0x7fb034100ea0 2024-04-22T23:52:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.175+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0341092a0 con 0x7fb034100ea0 2024-04-22T23:52:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.176+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb02c009a00 con 0x7fb034100ea0 2024-04-22T23:52:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.176+0000 7fb031ffb700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb02006ecc0 0x7fb020071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.176+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb02c08edf0 con 0x7fb034100ea0 2024-04-22T23:52:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.176+0000 7fb033fff700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb02006ecc0 0x7fb020071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.177+0000 7fb033fff700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb02006ecc0 0x7fb020071180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb024009fd0 tx=0x7fb0240060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.181+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb02c057ae0 con 0x7fb034100ea0 2024-04-22T23:52:33.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.413+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0340611d0 con 0x7fb02006ecc0 2024-04-22T23:52:33.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:33 smithi031 ceph-mon[111289]: pgmap v116: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:33.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:33 smithi031 ceph-mon[111289]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:33.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:33 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1415681597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:33.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:33 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/692539363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:52:33.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.414+0000 7fb031ffb700 1 -- 172.21.15.31:0/1042819952 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb0340611d0 con 0x7fb02006ecc0 2024-04-22T23:52:33.406 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "crash" 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "9/23 daemons upgraded", 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:52:33.407 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.416+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb02006ecc0 msgr2=0x7fb020071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.416+0000 7fb03aef1700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb02006ecc0 0x7fb020071180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb024009fd0 tx=0x7fb0240060c0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 msgr2=0x7fb034073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 0x7fb034073090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00d8d0 tx=0x7fb02c00dbe0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 shutdown_connections 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb02006ecc0 0x7fb020071180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb034100ea0 0x7fb034073090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 --2- 172.21.15.31:0/1042819952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb034101850 0x7fb0340735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 >> 172.21.15.31:0/1042819952 conn(0x7fb0340fc750 msgr2=0x7fb0340fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 shutdown_connections 2024-04-22T23:52:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.417+0000 7fb03aef1700 1 -- 172.21.15.31:0/1042819952 wait complete. 2024-04-22T23:52:33.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:33 smithi044 ceph-mon[82146]: pgmap v116: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-04-22T23:52:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:33 smithi044 ceph-mon[82146]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:33 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1415681597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:33.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:33 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/692539363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:52:33.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 -- 172.21.15.31:0/2986420086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 msgr2=0x7f152c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 --2- 172.21.15.31:0/2986420086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c105890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1520009910 tx=0x7f1520009c20 comp rx=0 tx=0).stop 2024-04-22T23:52:33.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 -- 172.21.15.31:0/2986420086 shutdown_connections 2024-04-22T23:52:33.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 --2- 172.21.15.31:0/2986420086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c105890 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 --2- 172.21.15.31:0/2986420086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f152c100e90 0x7f152c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 -- 172.21.15.31:0/2986420086 >> 172.21.15.31:0/2986420086 conn(0x7f152c0fc740 msgr2=0x7f152c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:33.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 -- 172.21.15.31:0/2986420086 shutdown_connections 2024-04-22T23:52:33.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.536+0000 7f15320fb700 1 -- 172.21.15.31:0/2986420086 wait complete. 2024-04-22T23:52:33.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f15320fb700 1 Processor -- start 2024-04-22T23:52:33.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f15320fb700 1 -- start start 2024-04-22T23:52:33.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f15320fb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f152c100e90 0x7f152c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:33.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f15320fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:33.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f15320fb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c19d2e0 con 0x7f152c101840 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f15320fb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c196790 con 0x7f152c100e90 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f152affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f152affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49092/0 (socket says 172.21.15.31:49092) 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f152affd700 1 -- 172.21.15.31:0/4252211536 learned_addr learned my addr 172.21.15.31:0/4252211536 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.537+0000 7f152b7fe700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f152c100e90 0x7f152c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f152affd700 1 -- 172.21.15.31:0/4252211536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f152c100e90 msgr2=0x7f152c19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f152affd700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f152c100e90 0x7f152c19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f152affd700 1 -- 172.21.15.31:0/4252211536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200095f0 con 0x7f152c101840 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f152affd700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c19cc50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f152000bda0 tx=0x7f152000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f152001d070 con 0x7f152c101840 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1520005dc0 con 0x7f152c101840 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1520017400 con 0x7f152c101840 2024-04-22T23:52:33.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c196a10 con 0x7f152c101840 2024-04-22T23:52:33.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.538+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c196f00 con 0x7f152c101840 2024-04-22T23:52:33.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.539+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c109290 con 0x7f152c101840 2024-04-22T23:52:33.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.540+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f152000f6f0 con 0x7f152c101840 2024-04-22T23:52:33.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.540+0000 7f1528ff9700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f151406ecc0 0x7f1514071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:52:33.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.540+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f15200902b0 con 0x7f152c101840 2024-04-22T23:52:33.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.540+0000 7f152b7fe700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f151406ecc0 0x7f1514071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:52:33.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.541+0000 7f152b7fe700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f151406ecc0 0x7f1514071180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f151c007be0 tx=0x7f151c0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:52:33.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.545+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f152005cad0 con 0x7f152c101840 2024-04-22T23:52:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.841+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f152c04f2e0 con 0x7f152c101840 2024-04-22T23:52:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.841+0000 7f1528ff9700 1 -- 172.21.15.31:0/4252211536 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f152005c660 con 0x7f152c101840 2024-04-22T23:52:33.834 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:52:33.834 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:52:33.834 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f151406ecc0 msgr2=0x7f1514071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f151406ecc0 0x7f1514071180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f151c007be0 tx=0x7f151c0073d0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 msgr2=0x7f152c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c19cc50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f152000bda0 tx=0x7f152000bdd0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 shutdown_connections 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f151406ecc0 0x7f1514071180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f152c100e90 0x7f152c19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 --2- 172.21.15.31:0/4252211536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f152c101840 0x7f152c19cc50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.843+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 >> 172.21.15.31:0/4252211536 conn(0x7f152c0fc740 msgr2=0x7f152c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.844+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 shutdown_connections 2024-04-22T23:52:33.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:52:33.844+0000 7f15320fb700 1 -- 172.21.15.31:0/4252211536 wait complete. 2024-04-22T23:52:34.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:34 smithi031 ceph-mon[111289]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:34.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:34 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4252211536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:52:34.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:34 smithi044 ceph-mon[82146]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:52:34.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:34 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4252211536' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:52:35.431 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:35 smithi044 ceph-mon[82146]: pgmap v117: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:35.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:35 smithi031 ceph-mon[111289]: pgmap v117: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:37.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:37 smithi044 ceph-mon[82146]: pgmap v118: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:52:37.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:37 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:37.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:37 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:37.341 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:37 smithi044 ceph-mon[82146]: Upgrade: osd.3 is safe to restart 2024-04-22T23:52:37.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:37 smithi031 ceph-mon[111289]: pgmap v118: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:52:37.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:37.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:37 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:52:37.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:37 smithi031 ceph-mon[111289]: Upgrade: osd.3 is safe to restart 2024-04-22T23:52:39.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:39 smithi031 ceph-mon[111289]: Upgrade: Updating osd.3 2024-04-22T23:52:39.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:39 smithi031 ceph-mon[111289]: pgmap v119: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:39.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:39 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:39.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:39 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:52:39.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:39 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:39.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:39 smithi031 ceph-mon[111289]: Deploying daemon osd.3 on smithi044 2024-04-22T23:52:39.351 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:39 smithi044 ceph-mon[82146]: Upgrade: Updating osd.3 2024-04-22T23:52:39.351 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:39 smithi044 ceph-mon[82146]: pgmap v119: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:39.351 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:39 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:39.352 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:39 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:52:39.352 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:39 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:39.352 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:39 smithi044 ceph-mon[82146]: Deploying daemon osd.3 on smithi044 2024-04-22T23:52:41.366 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:41 smithi044 ceph-mon[82146]: pgmap v120: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:41.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:41 smithi031 ceph-mon[111289]: pgmap v120: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:52:41.666 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:41 smithi044 systemd[1]: Stopping Ceph osd.3 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:52:41.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:41 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[40534]: 2024-04-22T23:52:41.673+0000 7fd4a316e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:52:41.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:41 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[40534]: 2024-04-22T23:52:41.673+0000 7fd4a316e700 -1 osd.3 71 *** Got signal Terminated *** 2024-04-22T23:52:41.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:41 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[40534]: 2024-04-22T23:52:41.673+0000 7fd4a316e700 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:52:42.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:42 smithi031 ceph-mon[111289]: osd.3 marked itself down and dead 2024-04-22T23:52:42.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:42 smithi044 ceph-mon[82146]: osd.3 marked itself down and dead 2024-04-22T23:52:42.750 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:42 smithi044 podman[91456]: 2024-04-22 23:52:42.495165538 +0000 UTC m=+0.936002706 container died 393d4900ec3cf6c92e59d4383ec62f7fad3ac73dbb8e91eb79cd220bb675e563 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T23:52:43.129 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:42 smithi044 podman[91456]: 2024-04-22 23:52:42.848792652 +0000 UTC m=+1.289629820 container cleanup 393d4900ec3cf6c92e59d4383ec62f7fad3ac73dbb8e91eb79cd220bb675e563 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.29.1) 2024-04-22T23:52:43.129 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:42 smithi044 bash[91456]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3 2024-04-22T23:52:43.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:43 smithi031 ceph-mon[111289]: pgmap v121: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:52:43.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:43 smithi031 ceph-mon[111289]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:52:43.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:43 smithi031 ceph-mon[111289]: osdmap e72: 6 total, 5 up, 6 in 2024-04-22T23:52:43.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:43 smithi044 ceph-mon[82146]: pgmap v121: 65 pgs: 65 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:52:43.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:43 smithi044 ceph-mon[82146]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:52:43.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:43 smithi044 ceph-mon[82146]: osdmap e72: 6 total, 5 up, 6 in 2024-04-22T23:52:43.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:43 smithi044 podman[91474]: 2024-04-22 23:52:43.317077297 +0000 UTC m=+0.821944337 container remove 393d4900ec3cf6c92e59d4383ec62f7fad3ac73dbb8e91eb79cd220bb675e563 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, RELEASE=HEAD) 2024-04-22T23:52:43.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:43 smithi044 podman[91563]: 2024-04-22 23:52:43.718110261 +0000 UTC m=+0.105310844 container create c233632be9c28ed5c1273f7c2d61be30e915c9d58cee6b55b05de903d48a7245 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T23:52:43.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:43 smithi044 podman[91563]: 2024-04-22 23:52:43.631156538 +0000 UTC m=+0.018357144 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:52:43.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:43 smithi044 podman[91563]: 2024-04-22 23:52:43.945477 +0000 UTC m=+0.332677579 container init c233632be9c28ed5c1273f7c2d61be30e915c9d58cee6b55b05de903d48a7245 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T23:52:43.960 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:43 smithi044 podman[91563]: 2024-04-22 23:52:43.949019291 +0000 UTC m=+0.336219876 container start c233632be9c28ed5c1273f7c2d61be30e915c9d58cee6b55b05de903d48a7245 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:52:44.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:44 smithi031 ceph-mon[111289]: osdmap e73: 6 total, 5 up, 6 in 2024-04-22T23:52:44.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:44 smithi044 ceph-mon[82146]: osdmap e73: 6 total, 5 up, 6 in 2024-04-22T23:52:44.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:44 smithi044 podman[91563]: 2024-04-22 23:52:44.04564601 +0000 UTC m=+0.432846594 container attach c233632be9c28ed5c1273f7c2d61be30e915c9d58cee6b55b05de903d48a7245 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True) 2024-04-22T23:52:44.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:44 smithi044 podman[91563]: 2024-04-22 23:52:44.256302346 +0000 UTC m=+0.643502929 container died c233632be9c28ed5c1273f7c2d61be30e915c9d58cee6b55b05de903d48a7245 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T23:52:45.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:45 smithi031 ceph-mon[111289]: pgmap v124: 65 pgs: 13 stale+active+clean, 52 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-22T23:52:45.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:45 smithi044 ceph-mon[82146]: pgmap v124: 65 pgs: 13 stale+active+clean, 52 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-22T23:52:45.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 podman[91589]: 2024-04-22 23:52:45.082339675 +0000 UTC m=+0.820798961 container remove c233632be9c28ed5c1273f7c2d61be30e915c9d58cee6b55b05de903d48a7245 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-22T23:52:45.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-22T23:52:45.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 systemd[1]: Stopped Ceph osd.3 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:52:45.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 7.684s CPU time. 2024-04-22T23:52:45.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 systemd[1]: Starting Ceph osd.3 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:52:45.785 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 podman[91693]: 2024-04-22 23:52:45.44618425 +0000 UTC m=+0.018606422 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:52:45.785 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 podman[91693]: 2024-04-22 23:52:45.584003656 +0000 UTC m=+0.156425874 container create 9eda48a19d4450202ec4fa1c07ac00201ca29bcea33385f696889af0fe21a0ae (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-22T23:52:45.785 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 podman[91693]: 2024-04-22 23:52:45.794970962 +0000 UTC m=+0.367393135 container init 9eda48a19d4450202ec4fa1c07ac00201ca29bcea33385f696889af0fe21a0ae (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:52:46.156 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 podman[91693]: 2024-04-22 23:52:45.798194786 +0000 UTC m=+0.370616959 container start 9eda48a19d4450202ec4fa1c07ac00201ca29bcea33385f696889af0fe21a0ae (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:52:46.156 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:45 smithi044 podman[91693]: 2024-04-22 23:52:45.894990817 +0000 UTC m=+0.467412998 container attach 9eda48a19d4450202ec4fa1c07ac00201ca29bcea33385f696889af0fe21a0ae (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-22T23:52:46.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:46 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:46.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:46 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:46.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:46.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-22T23:52:46.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 bash[91693]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate[91708]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-22T23:52:46.461 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 podman[91693]: 2024-04-22 23:52:46.315755455 +0000 UTC m=+0.888177625 container died 9eda48a19d4450202ec4fa1c07ac00201ca29bcea33385f696889af0fe21a0ae (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T23:52:47.157 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:46 smithi044 podman[91840]: 2024-04-22 23:52:46.890451989 +0000 UTC m=+0.568950458 container remove 9eda48a19d4450202ec4fa1c07ac00201ca29bcea33385f696889af0fe21a0ae (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-22T23:52:47.157 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 podman[91903]: 2024-04-22 23:52:47.011838377 +0000 UTC m=+0.018831713 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:52:47.157 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 podman[91903]: 2024-04-22 23:52:47.11638294 +0000 UTC m=+0.123376277 container create bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:52:47.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:47 smithi031 ceph-mon[111289]: pgmap v125: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 1 stale+active+clean, 34 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 48/264 objects degraded (18.182%) 2024-04-22T23:52:47.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:47 smithi031 ceph-mon[111289]: Health check failed: Degraded data redundancy: 48/264 objects degraded (18.182%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:47.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:47 smithi044 ceph-mon[82146]: pgmap v125: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 1 stale+active+clean, 34 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 48/264 objects degraded (18.182%) 2024-04-22T23:52:47.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:47 smithi044 ceph-mon[82146]: Health check failed: Degraded data redundancy: 48/264 objects degraded (18.182%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:47.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 podman[91903]: 2024-04-22 23:52:47.310767039 +0000 UTC m=+0.317760376 container init bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T23:52:47.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 podman[91903]: 2024-04-22 23:52:47.313867047 +0000 UTC m=+0.320860390 container start bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-22T23:52:47.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 ceph-osd[91922]: -- 172.21.15.44:0/4115971493 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557167507680 con 0x5571674d0c00 2024-04-22T23:52:47.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 bash[91903]: bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec 2024-04-22T23:52:47.460 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 systemd[1]: Started Ceph osd.3 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:52:48.210 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:47 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-22T23:52:47.854+0000 7f0bfd82f7c0 -1 Falling back to public interface 2024-04-22T23:52:48.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:48 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:48.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:48 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:48.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:48 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:48 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:48 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:48.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:48 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:49.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:49 smithi044 ceph-mon[82146]: pgmap v126: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:49.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:49 smithi031 ceph-mon[111289]: pgmap v126: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:51.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:51 smithi031 ceph-mon[111289]: pgmap v127: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:51.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:51 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:51.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:51 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:51.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:51 smithi044 ceph-mon[82146]: pgmap v127: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:51.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:51 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:51.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:51 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:52.524 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:52 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 51/264 objects degraded (19.318%), 15 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:52.524 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:52 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:52.524 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:52 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:52.524 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:52 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-22T23:52:52.381+0000 7f0bfd82f7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-22T23:52:52.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:52 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 51/264 objects degraded (19.318%), 15 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:52.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:52 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:52.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:52 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:52.959 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:52 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-22T23:52:52.712+0000 7f0bfd82f7c0 -1 osd.3 71 log_to_monitors true 2024-04-22T23:52:53.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:53 smithi031 ceph-mon[111289]: pgmap v128: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:53.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:53 smithi031 ceph-mon[111289]: from='osd.3 [v2:172.21.15.44:6800/2576217506,v1:172.21.15.44:6801/2576217506]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:52:53.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:53 smithi031 ceph-mon[111289]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:52:53.959 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 22 23:52:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-22T23:52:53.632+0000 7f0beec87700 -1 osd.3 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:52:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:53 smithi044 ceph-mon[82146]: pgmap v128: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:53 smithi044 ceph-mon[82146]: from='osd.3 [v2:172.21.15.44:6800/2576217506,v1:172.21.15.44:6801/2576217506]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:52:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:53 smithi044 ceph-mon[82146]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:52:54.803 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:54 smithi044 ceph-mon[82146]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:52:54.804 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:54 smithi044 ceph-mon[82146]: from='osd.3 [v2:172.21.15.44:6800/2576217506,v1:172.21.15.44:6801/2576217506]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:52:54.804 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:54 smithi044 ceph-mon[82146]: osdmap e74: 6 total, 5 up, 6 in 2024-04-22T23:52:54.804 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:54 smithi044 ceph-mon[82146]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:52:54.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:54 smithi031 ceph-mon[111289]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:52:54.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:54 smithi031 ceph-mon[111289]: from='osd.3 [v2:172.21.15.44:6800/2576217506,v1:172.21.15.44:6801/2576217506]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:52:54.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:54 smithi031 ceph-mon[111289]: osdmap e74: 6 total, 5 up, 6 in 2024-04-22T23:52:54.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:54 smithi031 ceph-mon[111289]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:52:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: osd.3 [v2:172.21.15.44:6800/2576217506,v1:172.21.15.44:6801/2576217506] boot 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: osdmap e75: 6 total, 6 up, 6 in 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:55.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:52:55.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:55 smithi031 ceph-mon[111289]: osdmap e76: 6 total, 6 up, 6 in 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: pgmap v130: 65 pgs: 18 active+undersized, 15 active+undersized+degraded, 32 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/264 objects degraded (19.318%) 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: osd.3 [v2:172.21.15.44:6800/2576217506,v1:172.21.15.44:6801/2576217506] boot 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: osdmap e75: 6 total, 6 up, 6 in 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:52:55.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:52:55.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:55.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:55.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:52:55.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:52:55.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:55 smithi044 ceph-mon[82146]: osdmap e76: 6 total, 6 up, 6 in 2024-04-22T23:52:56.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:56 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:52:56.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:56 smithi031 ceph-mon[111289]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-04-22T23:52:56.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:56 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:52:56.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:56 smithi044 ceph-mon[82146]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-04-22T23:52:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:57 smithi031 ceph-mon[111289]: pgmap v133: 65 pgs: 8 peering, 10 active+undersized, 8 active+undersized+degraded, 39 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 25/264 objects degraded (9.470%) 2024-04-22T23:52:57.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:57 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 25/264 objects degraded (9.470%), 8 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:57 smithi044 ceph-mon[82146]: pgmap v133: 65 pgs: 8 peering, 10 active+undersized, 8 active+undersized+degraded, 39 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 25/264 objects degraded (9.470%) 2024-04-22T23:52:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:57 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 25/264 objects degraded (9.470%), 8 pgs degraded (PG_DEGRADED) 2024-04-22T23:52:59.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:52:59 smithi031 ceph-mon[111289]: pgmap v134: 65 pgs: 8 peering, 9 active+undersized, 7 active+undersized+degraded, 41 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 22/264 objects degraded (8.333%) 2024-04-22T23:52:59.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:52:59 smithi044 ceph-mon[82146]: pgmap v134: 65 pgs: 8 peering, 9 active+undersized, 7 active+undersized+degraded, 41 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 22/264 objects degraded (8.333%) 2024-04-22T23:53:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:01 smithi031 ceph-mon[111289]: pgmap v135: 65 pgs: 8 peering, 57 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 942 B/s rd, 2 op/s 2024-04-22T23:53:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:01 smithi031 ceph-mon[111289]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/264 objects degraded (8.333%), 7 pgs degraded) 2024-04-22T23:53:01.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:01 smithi044 ceph-mon[82146]: pgmap v135: 65 pgs: 8 peering, 57 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 942 B/s rd, 2 op/s 2024-04-22T23:53:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:01.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:01 smithi044 ceph-mon[82146]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/264 objects degraded (8.333%), 7 pgs degraded) 2024-04-22T23:53:03.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:03 smithi031 ceph-mon[111289]: pgmap v136: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:53:03.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:03 smithi044 ceph-mon[82146]: pgmap v136: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 -- 172.21.15.31:0/420249218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 msgr2=0x7f0270105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 --2- 172.21.15.31:0/420249218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270105fb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0260009910 tx=0x7f0260009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 -- 172.21.15.31:0/420249218 shutdown_connections 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 --2- 172.21.15.31:0/420249218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 0x7f0270100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 --2- 172.21.15.31:0/420249218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270105fb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 -- 172.21.15.31:0/420249218 >> 172.21.15.31:0/420249218 conn(0x7f0270075670 msgr2=0x7f0270075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 -- 172.21.15.31:0/420249218 shutdown_connections 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.959+0000 7f0274c98700 1 -- 172.21.15.31:0/420249218 wait complete. 2024-04-22T23:53:03.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.960+0000 7f0274c98700 1 Processor -- start 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.960+0000 7f0274c98700 1 -- start start 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.960+0000 7f0274c98700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 0x7f0270101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.960+0000 7f0274c98700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.960+0000 7f0274c98700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02701022c0 con 0x7f0270105bd0 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.960+0000 7f0274c98700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0270102400 con 0x7f02700ffc20 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37328/0 (socket says 172.21.15.31:37328) 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 -- 172.21.15.31:0/553814806 learned_addr learned my addr 172.21.15.31:0/553814806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026e59c700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 0x7f0270101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 -- 172.21.15.31:0/553814806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 msgr2=0x7f0270101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 0x7f0270101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 -- 172.21.15.31:0/553814806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600095f0 con 0x7f0270105bd0 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026e59c700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 0x7f0270101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:53:03.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f026dd9b700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270101d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f026400c930 tx=0x7f026400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0264007ad0 con 0x7f0270105bd0 2024-04-22T23:53:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0264007c30 con 0x7f0270105bd0 2024-04-22T23:53:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02701a2a70 con 0x7f0270105bd0 2024-04-22T23:53:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0264018610 con 0x7f0270105bd0 2024-04-22T23:53:03.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.961+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02701a2dd0 con 0x7f0270105bd0 2024-04-22T23:53:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.962+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027004f2e0 con 0x7f0270105bd0 2024-04-22T23:53:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.963+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f026400f450 con 0x7f0270105bd0 2024-04-22T23:53:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.963+0000 7f025f7fe700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f025806ecc0 0x7f0258071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.964+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f026408f160 con 0x7f0270105bd0 2024-04-22T23:53:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.964+0000 7f026e59c700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f025806ecc0 0x7f0258071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:03.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.964+0000 7f026e59c700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f025806ecc0 0x7f0258071180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0260005b40 tx=0x7f026001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:03.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:03.969+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0264057e50 con 0x7f0270105bd0 2024-04-22T23:53:04.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.198+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0270068d70 con 0x7f025806ecc0 2024-04-22T23:53:04.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.199+0000 7f025f7fe700 1 -- 172.21.15.31:0/553814806 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0270068d70 con 0x7f025806ecc0 2024-04-22T23:53:04.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f025806ecc0 msgr2=0x7f0258071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f025806ecc0 0x7f0258071180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0260005b40 tx=0x7f026001a040 comp rx=0 tx=0).stop 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 msgr2=0x7f0270101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270101d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f026400c930 tx=0x7f026400ccf0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 shutdown_connections 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f025806ecc0 0x7f0258071180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02700ffc20 0x7f0270101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 --2- 172.21.15.31:0/553814806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0270105bd0 0x7f0270101d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.201+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 >> 172.21.15.31:0/553814806 conn(0x7f0270075670 msgr2=0x7f0270068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.202+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 shutdown_connections 2024-04-22T23:53:04.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.202+0000 7f0274c98700 1 -- 172.21.15.31:0/553814806 wait complete. 2024-04-22T23:53:04.204 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:53:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 -- 172.21.15.31:0/1102875224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 msgr2=0x7f259410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 --2- 172.21.15.31:0/1102875224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259410ac10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f257c009910 tx=0x7f257c009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 -- 172.21.15.31:0/1102875224 shutdown_connections 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 --2- 172.21.15.31:0/1102875224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259410ac10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 --2- 172.21.15.31:0/1102875224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2594101e00 0x7f25941021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 -- 172.21.15.31:0/1102875224 >> 172.21.15.31:0/1102875224 conn(0x7f25940fb650 msgr2=0x7f25940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 -- 172.21.15.31:0/1102875224 shutdown_connections 2024-04-22T23:53:04.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.310+0000 7f2598d9c700 1 -- 172.21.15.31:0/1102875224 wait complete. 2024-04-22T23:53:04.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2598d9c700 1 Processor -- start 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2598d9c700 1 -- start start 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2598d9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2594101e00 0x7f259419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2598d9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2598d9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259419d260 con 0x7f2594102720 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2598d9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594196710 con 0x7f2594101e00 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2591d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2591d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37352/0 (socket says 172.21.15.31:37352) 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f2591d9b700 1 -- 172.21.15.31:0/3135817474 learned_addr learned my addr 172.21.15.31:0/3135817474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.311+0000 7f259259c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2594101e00 0x7f259419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f2591d9b700 1 -- 172.21.15.31:0/3135817474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2594101e00 msgr2=0x7f259419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f2591d9b700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2594101e00 0x7f259419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f2591d9b700 1 -- 172.21.15.31:0/3135817474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0095f0 con 0x7f2594102720 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f2591d9b700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259419cbd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f257c000c00 tx=0x7f257c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f257c01d070 con 0x7f2594102720 2024-04-22T23:53:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2594196990 con 0x7f2594102720 2024-04-22T23:53:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f257c005dc0 con 0x7f2594102720 2024-04-22T23:53:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f257c00fe70 con 0x7f2594102720 2024-04-22T23:53:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.312+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2594196e80 con 0x7f2594102720 2024-04-22T23:53:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.313+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2594108310 con 0x7f2594102720 2024-04-22T23:53:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.314+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f257c003a60 con 0x7f2594102720 2024-04-22T23:53:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.314+0000 7f258b7fe700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f258006ecc0 0x7f2580071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.314+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f257c090310 con 0x7f2594102720 2024-04-22T23:53:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.314+0000 7f259259c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f258006ecc0 0x7f2580071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.315+0000 7f259259c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f258006ecc0 0x7f2580071180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2584009550 tx=0x7f25840091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:04.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.319+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f257c05cb30 con 0x7f2594102720 2024-04-22T23:53:04.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.551+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2594197c90 con 0x7f258006ecc0 2024-04-22T23:53:04.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.552+0000 7f258b7fe700 1 -- 172.21.15.31:0/3135817474 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2594197c90 con 0x7f258006ecc0 2024-04-22T23:53:04.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f258006ecc0 msgr2=0x7f2580071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f258006ecc0 0x7f2580071180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2584009550 tx=0x7f25840091b0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 msgr2=0x7f259419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259419cbd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f257c000c00 tx=0x7f257c00bd70 comp rx=0 tx=0).stop 2024-04-22T23:53:04.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 shutdown_connections 2024-04-22T23:53:04.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f258006ecc0 0x7f2580071180 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2594101e00 0x7f259419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 --2- 172.21.15.31:0/3135817474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2594102720 0x7f259419cbd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 >> 172.21.15.31:0/3135817474 conn(0x7f25940fb650 msgr2=0x7f2594105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:04.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 shutdown_connections 2024-04-22T23:53:04.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.555+0000 7f2598d9c700 1 -- 172.21.15.31:0/3135817474 wait complete. 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 -- 172.21.15.31:0/1108434733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0101e00 msgr2=0x7f10d01021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/1108434733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0101e00 0x7f10d01021e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009910 tx=0x7f10c4009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 -- 172.21.15.31:0/1108434733 shutdown_connections 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/1108434733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0102720 0x7f10d010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/1108434733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0101e00 0x7f10d01021e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 -- 172.21.15.31:0/1108434733 >> 172.21.15.31:0/1108434733 conn(0x7f10d00fb650 msgr2=0x7f10d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 -- 172.21.15.31:0/1108434733 shutdown_connections 2024-04-22T23:53:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 -- 172.21.15.31:0/1108434733 wait complete. 2024-04-22T23:53:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 Processor -- start 2024-04-22T23:53:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.663+0000 7f10d6ad1700 1 -- start start 2024-04-22T23:53:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10d6ad1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101e00 0x7f10d00ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10d6ad1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 0x7f10d00fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10d6ad1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d01004e0 con 0x7f10d0102720 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10d6ad1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0100620 con 0x7f10d0101e00 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 0x7f10d00fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 0x7f10d00fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37382/0 (socket says 172.21.15.31:37382) 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 -- 172.21.15.31:0/406711615 learned_addr learned my addr 172.21.15.31:0/406711615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 -- 172.21.15.31:0/406711615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101e00 msgr2=0x7f10d00ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10d486d700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101e00 0x7f10d00ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101e00 0x7f10d00ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 -- 172.21.15.31:0/406711615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40095f0 con 0x7f10d0102720 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10d486d700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101e00 0x7f10d00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.664+0000 7f10cffff700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 0x7f10d00fffa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c8f0 tx=0x7f10bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.665+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10bc0041f0 con 0x7f10d0102720 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.665+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10bc007dc0 con 0x7f10d0102720 2024-04-22T23:53:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.665+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d01a2a90 con 0x7f10d0102720 2024-04-22T23:53:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.665+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10bc003ea0 con 0x7f10d0102720 2024-04-22T23:53:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.665+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d01a2df0 con 0x7f10d0102720 2024-04-22T23:53:04.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.666+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d0108310 con 0x7f10d0102720 2024-04-22T23:53:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.666+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f10bc004350 con 0x7f10d0102720 2024-04-22T23:53:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.667+0000 7f10cdffb700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f10c006ecc0 0x7f10c0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.667+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f10bc08ebc0 con 0x7f10d0102720 2024-04-22T23:53:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.667+0000 7f10d486d700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f10c006ecc0 0x7f10c0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:04.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.667+0000 7f10d486d700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f10c006ecc0 0x7f10c0071180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10c400b5c0 tx=0x7f10c4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:04.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.672+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f10bc057930 con 0x7f10d0102720 2024-04-22T23:53:04.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.890+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f10d002cce0 con 0x7f10c006ecc0 2024-04-22T23:53:04.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.901+0000 7f10cdffb700 1 -- 172.21.15.31:0/406711615 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f10d002cce0 con 0x7f10c006ecc0 2024-04-22T23:53:04.892 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 48s ago 15m 25.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (16m) 48s ago 16m 8594k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (10m) 14s ago 15m 8136k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (2m) 48s ago 16m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (2m) 14s ago 15m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (4m) 48s ago 15m 80.1M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (10m) 48s ago 10m 16.9M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (10m) 48s ago 10m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (10m) 14s ago 10m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (10m) 14s ago 10m 20.9M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (7m) 48s ago 17m 508M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (6m) 14s ago 14m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (3m) 48s ago 17m 50.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:53:04.893 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (3m) 14s ago 14m 41.6M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 48s ago 16m 15.1M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (5m) 14s ago 15m 15.4M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (2m) 48s ago 14m 96.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (90s) 48s ago 13m 126M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (54s) 48s ago 13m 15.1M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (17s) 14s ago 12m 11.7M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (12m) 14s ago 12m 216M 2241M 18.2.1 5be31c24972a 67592bd335a2 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (11m) 14s ago 11m 206M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:53:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 48s ago 15m 76.0M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.903+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f10c006ecc0 msgr2=0x7f10c0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.903+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f10c006ecc0 0x7f10c0071180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f10c400b5c0 tx=0x7f10c4005960 comp rx=0 tx=0).stop 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.903+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 msgr2=0x7f10d00fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.903+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 0x7f10d00fffa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c8f0 tx=0x7f10bc00ccb0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 shutdown_connections 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f10c006ecc0 0x7f10c0071180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f10d0101e00 0x7f10d00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 --2- 172.21.15.31:0/406711615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10d0102720 0x7f10d00fffa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 >> 172.21.15.31:0/406711615 conn(0x7f10d00fb650 msgr2=0x7f10d0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:04.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 shutdown_connections 2024-04-22T23:53:04.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:04.904+0000 7f10d6ad1700 1 -- 172.21.15.31:0/406711615 wait complete. 2024-04-22T23:53:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 -- 172.21.15.31:0/3054195488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 msgr2=0x7fdca80ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 --2- 172.21.15.31:0/3054195488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca80ff7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009910 tx=0x7fdc9c009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 -- 172.21.15.31:0/3054195488 shutdown_connections 2024-04-22T23:53:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 --2- 172.21.15.31:0/3054195488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca80ff7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 --2- 172.21.15.31:0/3054195488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 0x7fdca8105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 -- 172.21.15.31:0/3054195488 >> 172.21.15.31:0/3054195488 conn(0x7fdca8074e30 msgr2=0x7fdca8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 -- 172.21.15.31:0/3054195488 shutdown_connections 2024-04-22T23:53:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.011+0000 7fdcae861700 1 -- 172.21.15.31:0/3054195488 wait complete. 2024-04-22T23:53:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdcae861700 1 Processor -- start 2024-04-22T23:53:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdcae861700 1 -- start start 2024-04-22T23:53:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdcae861700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca81976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdcae861700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 0x7fdca8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdcae861700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8198200 con 0x7fdca80ff340 2024-04-22T23:53:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdcae861700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8198370 con 0x7fdca8105350 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca81976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca81976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37388/0 (socket says 172.21.15.31:37388) 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca7fff700 1 -- 172.21.15.31:0/383072918 learned_addr learned my addr 172.21.15.31:0/383072918 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca77fe700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 0x7fdca8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca7fff700 1 -- 172.21.15.31:0/383072918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 msgr2=0x7fdca8197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca7fff700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 0x7fdca8197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca7fff700 1 -- 172.21.15.31:0/383072918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0095f0 con 0x7fdca80ff340 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.012+0000 7fdca77fe700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 0x7fdca8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.013+0000 7fdca7fff700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca81976a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d8d0 tx=0x7fdc9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.013+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc98009940 con 0x7fdca80ff340 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.013+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca81a0710 con 0x7fdca80ff340 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.013+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc98010460 con 0x7fdca80ff340 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.013+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc9800f5f0 con 0x7fdca80ff340 2024-04-22T23:53:05.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.013+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca81a0c00 con 0x7fdca80ff340 2024-04-22T23:53:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.014+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8103c00 con 0x7fdca80ff340 2024-04-22T23:53:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.014+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdc980105d0 con 0x7fdca80ff340 2024-04-22T23:53:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.015+0000 7fdca57fa700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdc9006ecc0 0x7fdc90071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.015+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fdc9808ef40 con 0x7fdca80ff340 2024-04-22T23:53:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.015+0000 7fdca77fe700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdc9006ecc0 0x7fdc90071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.015+0000 7fdca77fe700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdc9006ecc0 0x7fdc90071180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdca8101c20 tx=0x7fdc9c00b580 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:05.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.020+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fdc98057c30 con 0x7fdca80ff340 2024-04-22T23:53:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.322+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdca804f2e0 con 0x7fdca80ff340 2024-04-22T23:53:05.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.322+0000 7fdca57fa700 1 -- 172.21.15.31:0/383072918 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fdc9805b2f0 con 0x7fdca80ff340 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 8 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:53:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.324+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdc9006ecc0 msgr2=0x7fdc90071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.324+0000 7fdcae861700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdc9006ecc0 0x7fdc90071180 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdca8101c20 tx=0x7fdc9c00b580 comp rx=0 tx=0).stop 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 msgr2=0x7fdca81976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca81976a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d8d0 tx=0x7fdc9800dc90 comp rx=0 tx=0).stop 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 shutdown_connections 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdc9006ecc0 0x7fdc90071180 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca80ff340 0x7fdca81976a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 --2- 172.21.15.31:0/383072918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdca8105350 0x7fdca8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 >> 172.21.15.31:0/383072918 conn(0x7fdca8074e30 msgr2=0x7fdca80686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:05.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 shutdown_connections 2024-04-22T23:53:05.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.325+0000 7fdcae861700 1 -- 172.21.15.31:0/383072918 wait complete. 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 -- 172.21.15.31:0/1341543374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 msgr2=0x7f05d0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 --2- 172.21.15.31:0/1341543374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0105fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009910 tx=0x7f05c0009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 -- 172.21.15.31:0/1341543374 shutdown_connections 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 --2- 172.21.15.31:0/1341543374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d00ffc20 0x7f05d0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 --2- 172.21.15.31:0/1341543374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0105fb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 -- 172.21.15.31:0/1341543374 >> 172.21.15.31:0/1341543374 conn(0x7f05d0075670 msgr2=0x7f05d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.432+0000 7f05d6159700 1 -- 172.21.15.31:0/1341543374 shutdown_connections 2024-04-22T23:53:05.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.433+0000 7f05d6159700 1 -- 172.21.15.31:0/1341543374 wait complete. 2024-04-22T23:53:05.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:05 smithi031 ceph-mon[111289]: pgmap v137: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:53:05.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:05 smithi031 ceph-mon[111289]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:05.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:05 smithi031 ceph-mon[111289]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:05.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:05 smithi031 ceph-mon[111289]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:05.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.433+0000 7f05d6159700 1 Processor -- start 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05d6159700 1 -- start start 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05d6159700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d00ffc20 0x7f05d01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05d6159700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05d6159700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0198340 con 0x7f05d0105bd0 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05d6159700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0198480 con 0x7f05d00ffc20 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37408/0 (socket says 172.21.15.31:37408) 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05ceffd700 1 -- 172.21.15.31:0/2229038863 learned_addr learned my addr 172.21.15.31:0/2229038863 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05cf7fe700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d00ffc20 0x7f05d01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05ceffd700 1 -- 172.21.15.31:0/2229038863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d00ffc20 msgr2=0x7f05d01977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05ceffd700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d00ffc20 0x7f05d01977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.434+0000 7f05ceffd700 1 -- 172.21.15.31:0/2229038863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c00095f0 con 0x7f05d0105bd0 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.435+0000 7f05ceffd700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0197d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05c400b4b0 tx=0x7f05c400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:05.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.435+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c4011800 con 0x7f05d0105bd0 2024-04-22T23:53:05.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.435+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d01a0850 con 0x7f05d0105bd0 2024-04-22T23:53:05.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.435+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05c4011e20 con 0x7f05d0105bd0 2024-04-22T23:53:05.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.435+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c400f2e0 con 0x7f05d0105bd0 2024-04-22T23:53:05.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.435+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d01a0d10 con 0x7f05d0105bd0 2024-04-22T23:53:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.436+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d01080d0 con 0x7f05d0105bd0 2024-04-22T23:53:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.436+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f05c4010380 con 0x7f05d0105bd0 2024-04-22T23:53:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.437+0000 7f05ccff9700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f05b806eab0 0x7f05b8070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.437+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f05c408e660 con 0x7f05d0105bd0 2024-04-22T23:53:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.437+0000 7f05cf7fe700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f05b806eab0 0x7f05b8070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.438+0000 7f05cf7fe700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f05b806eab0 0x7f05b8070f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006010 tx=0x7f05c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:05.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.442+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f05c40573d0 con 0x7f05d0105bd0 2024-04-22T23:53:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:05 smithi044 ceph-mon[82146]: pgmap v137: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:53:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:05 smithi044 ceph-mon[82146]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:05 smithi044 ceph-mon[82146]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:05 smithi044 ceph-mon[82146]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.701+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f05d0068b40 con 0x7f05d0105bd0 2024-04-22T23:53:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.702+0000 7f05ccff9700 1 -- 172.21.15.31:0/2229038863 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7f05c405aa90 con 0x7f05d0105bd0 2024-04-22T23:53:05.693 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:53:05.693 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:53:05.693 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:53:05.693 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:53:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:53:05.695 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:05.696 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f05b806eab0 msgr2=0x7f05b8070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f05b806eab0 0x7f05b8070f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05c0006010 tx=0x7f05c001a040 comp rx=0 tx=0).stop 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 msgr2=0x7f05d0197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0197d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f05c400b4b0 tx=0x7f05c400b7c0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 shutdown_connections 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f05b806eab0 0x7f05b8070f70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05d00ffc20 0x7f05d01977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 --2- 172.21.15.31:0/2229038863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d0105bd0 0x7f05d0197d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 >> 172.21.15.31:0/2229038863 conn(0x7f05d0075670 msgr2=0x7f05d00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:05.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 shutdown_connections 2024-04-22T23:53:05.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.704+0000 7f05d6159700 1 -- 172.21.15.31:0/2229038863 wait complete. 2024-04-22T23:53:05.698 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1811380516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0100ea0 msgr2=0x7f67a0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1811380516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0100ea0 0x7f67a0101280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6794009910 tx=0x7f6794009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1811380516 shutdown_connections 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1811380516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a0101850 0x7f67a01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1811380516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0100ea0 0x7f67a0101280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1811380516 >> 172.21.15.31:0/1811380516 conn(0x7f67a00fc750 msgr2=0x7f67a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1811380516 shutdown_connections 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1811380516 wait complete. 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.812+0000 7f67a6ddf700 1 Processor -- start 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f67a6ddf700 1 -- start start 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f67a6ddf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a0100ea0 0x7f67a0196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f67a6ddf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 0x7f67a0196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f67a6ddf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0197560 con 0x7f67a0101850 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f67a6ddf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a019b370 con 0x7f67a0100ea0 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 0x7f67a0196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 0x7f67a0196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37438/0 (socket says 172.21.15.31:37438) 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 -- 172.21.15.31:0/1464968093 learned_addr learned my addr 172.21.15.31:0/1464968093 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f67a4b7b700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a0100ea0 0x7f67a0196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 -- 172.21.15.31:0/1464968093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a0100ea0 msgr2=0x7f67a0196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a0100ea0 0x7f67a0196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 -- 172.21.15.31:0/1464968093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940095f0 con 0x7f67a0101850 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.813+0000 7f679ffff700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 0x7f67a0196ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f678c00b480 tx=0x7f678c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.814+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f678c011800 con 0x7f67a0101850 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.814+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f678c011e20 con 0x7f67a0101850 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.814+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a019b650 con 0x7f67a0101850 2024-04-22T23:53:05.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.814+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f678c010ea0 con 0x7f67a0101850 2024-04-22T23:53:05.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.814+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a019bba0 con 0x7f67a0101850 2024-04-22T23:53:05.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.815+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a0109240 con 0x7f67a0101850 2024-04-22T23:53:05.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.815+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f678c010380 con 0x7f67a0101850 2024-04-22T23:53:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.816+0000 7f679dffb700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f679006ecc0 0x7f6790071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.816+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f678c08e9f0 con 0x7f67a0101850 2024-04-22T23:53:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.816+0000 7f67a4b7b700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f679006ecc0 0x7f6790071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:05.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.816+0000 7f67a4b7b700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f679006ecc0 0x7f6790071180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6794006010 tx=0x7f679401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:05.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:05.820+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f678c05b210 con 0x7f67a0101850 2024-04-22T23:53:06.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.054+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67a0197d90 con 0x7f679006ecc0 2024-04-22T23:53:06.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.055+0000 7f679dffb700 1 -- 172.21.15.31:0/1464968093 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f67a0197d90 con 0x7f679006ecc0 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "crash" 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "10/23 daemons upgraded", 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:53:06.047 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f679006ecc0 msgr2=0x7f6790071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f679006ecc0 0x7f6790071180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6794006010 tx=0x7f679401a040 comp rx=0 tx=0).stop 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 msgr2=0x7f67a0196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 0x7f67a0196ed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f678c00b480 tx=0x7f678c00b790 comp rx=0 tx=0).stop 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 shutdown_connections 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f679006ecc0 0x7f6790071180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67a0100ea0 0x7f67a0196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 --2- 172.21.15.31:0/1464968093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67a0101850 0x7f67a0196ed0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 >> 172.21.15.31:0/1464968093 conn(0x7f67a00fc750 msgr2=0x7f67a00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 shutdown_connections 2024-04-22T23:53:06.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.058+0000 7f67a6ddf700 1 -- 172.21.15.31:0/1464968093 wait complete. 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 -- 172.21.15.31:0/2596357582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe4100750 msgr2=0x7fefe41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 --2- 172.21.15.31:0/2596357582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe4100750 0x7fefe41047a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009910 tx=0x7fefd4009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 -- 172.21.15.31:0/2596357582 shutdown_connections 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 --2- 172.21.15.31:0/2596357582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe4100750 0x7fefe41047a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 --2- 172.21.15.31:0/2596357582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe40ffda0 0x7fefe4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 -- 172.21.15.31:0/2596357582 >> 172.21.15.31:0/2596357582 conn(0x7fefe40fb650 msgr2=0x7fefe40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 -- 172.21.15.31:0/2596357582 shutdown_connections 2024-04-22T23:53:06.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.168+0000 7fefec10a700 1 -- 172.21.15.31:0/2596357582 wait complete. 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefec10a700 1 Processor -- start 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefec10a700 1 -- start start 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefec10a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 0x7fefe4195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefec10a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe4100750 0x7fefe4195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefec10a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4196010 con 0x7fefe40ffda0 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefec10a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe4196180 con 0x7fefe4100750 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefe9ea6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 0x7fefe4195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefe9ea6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 0x7fefe4195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37458/0 (socket says 172.21.15.31:37458) 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefe9ea6700 1 -- 172.21.15.31:0/1846035675 learned_addr learned my addr 172.21.15.31:0/1846035675 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.169+0000 7fefe96a5700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe4100750 0x7fefe4195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefe9ea6700 1 -- 172.21.15.31:0/1846035675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe4100750 msgr2=0x7fefe4195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefe9ea6700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe4100750 0x7fefe4195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefe9ea6700 1 -- 172.21.15.31:0/1846035675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd40095f0 con 0x7fefe40ffda0 2024-04-22T23:53:06.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefe9ea6700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 0x7fefe4195590 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fefe000d8d0 tx=0x7fefe000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe00098a0 con 0x7fefe40ffda0 2024-04-22T23:53:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefe0010460 con 0x7fefe40ffda0 2024-04-22T23:53:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe41a2c00 con 0x7fefe40ffda0 2024-04-22T23:53:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe000f5f0 con 0x7fefe40ffda0 2024-04-22T23:53:06.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.170+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe41a2f60 con 0x7fefe40ffda0 2024-04-22T23:53:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.171+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe41081a0 con 0x7fefe40ffda0 2024-04-22T23:53:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.171+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fefe0009a00 con 0x7fefe40ffda0 2024-04-22T23:53:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.172+0000 7fefdaffd700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fefd006ecc0 0x7fefd0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.172+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fefe008ec50 con 0x7fefe40ffda0 2024-04-22T23:53:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.172+0000 7fefe96a5700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fefd006ecc0 0x7fefd0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:06.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.173+0000 7fefe96a5700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fefd006ecc0 0x7fefd0071180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefe4069800 tx=0x7fefd401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:06.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.177+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fefe005b4f0 con 0x7fefe40ffda0 2024-04-22T23:53:06.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:06 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/383072918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:06.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:06 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2229038863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:53:06.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:06 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/383072918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:06 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2229038863' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:53:06.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.475+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fefe404f2e0 con 0x7fefe40ffda0 2024-04-22T23:53:06.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.475+0000 7fefdaffd700 1 -- 172.21.15.31:0/1846035675 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fefe00160e0 con 0x7fefe40ffda0 2024-04-22T23:53:06.467 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:53:06.467 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:53:06.467 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:53:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.477+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fefd006ecc0 msgr2=0x7fefd0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fefd006ecc0 0x7fefd0071180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefe4069800 tx=0x7fefd401a040 comp rx=0 tx=0).stop 2024-04-22T23:53:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 msgr2=0x7fefe4195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 0x7fefe4195590 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fefe000d8d0 tx=0x7fefe000dbe0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 shutdown_connections 2024-04-22T23:53:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fefd006ecc0 0x7fefd0071180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefe40ffda0 0x7fefe4195590 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 --2- 172.21.15.31:0/1846035675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefe4100750 0x7fefe4195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 >> 172.21.15.31:0/1846035675 conn(0x7fefe40fb650 msgr2=0x7fefe40fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 shutdown_connections 2024-04-22T23:53:06.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:06.478+0000 7fefec10a700 1 -- 172.21.15.31:0/1846035675 wait complete. 2024-04-22T23:53:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:07 smithi031 ceph-mon[111289]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:07 smithi031 ceph-mon[111289]: pgmap v138: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T23:53:07.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:07 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1846035675' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:53:07.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:07 smithi044 ceph-mon[82146]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:07.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:07 smithi044 ceph-mon[82146]: pgmap v138: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T23:53:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:07 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1846035675' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:53:09.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:09 smithi031 ceph-mon[111289]: pgmap v139: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-04-22T23:53:09.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:09 smithi044 ceph-mon[82146]: pgmap v139: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-04-22T23:53:11.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:11 smithi031 ceph-mon[111289]: pgmap v140: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-22T23:53:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:53:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:11 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:53:11.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:11 smithi031 ceph-mon[111289]: Upgrade: osd.4 is safe to restart 2024-04-22T23:53:11.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:11 smithi044 ceph-mon[82146]: pgmap v140: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-22T23:53:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:53:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:11 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:53:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:11 smithi044 ceph-mon[82146]: Upgrade: osd.4 is safe to restart 2024-04-22T23:53:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:13 smithi031 ceph-mon[111289]: pgmap v141: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:53:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:13 smithi031 ceph-mon[111289]: Upgrade: Updating osd.4 2024-04-22T23:53:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:53:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:53:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:13 smithi031 ceph-mon[111289]: Deploying daemon osd.4 on smithi044 2024-04-22T23:53:13.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:13 smithi044 ceph-mon[82146]: pgmap v141: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:53:13.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:13 smithi044 ceph-mon[82146]: Upgrade: Updating osd.4 2024-04-22T23:53:13.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:13.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:53:13.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:53:13.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:13 smithi044 ceph-mon[82146]: Deploying daemon osd.4 on smithi044 2024-04-22T23:53:15.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:15 smithi031 ceph-mon[111289]: pgmap v142: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:53:15.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:15.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:15.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:15 smithi044 ceph-mon[82146]: pgmap v142: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:53:15.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:15.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:15.912 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:15 smithi044 systemd[1]: Stopping Ceph osd.4 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:53:16.172 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:15 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[47021]: 2024-04-22T23:53:15.920+0000 7fe1d4ff6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:53:16.172 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:15 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[47021]: 2024-04-22T23:53:15.920+0000 7fe1d4ff6700 -1 osd.4 76 *** Got signal Terminated *** 2024-04-22T23:53:16.172 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:15 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[47021]: 2024-04-22T23:53:15.920+0000 7fe1d4ff6700 -1 osd.4 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:53:16.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:16 smithi031 ceph-mon[111289]: osd.4 marked itself down and dead 2024-04-22T23:53:16.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:16 smithi044 ceph-mon[82146]: osd.4 marked itself down and dead 2024-04-22T23:53:16.912 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:16 smithi044 podman[96361]: 2024-04-22 23:53:16.636268456 +0000 UTC m=+0.815315604 container died 67592bd335a28ef596f2d39b6f93f56e262465c4154330e5d100d9a31840f898 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1) 2024-04-22T23:53:17.171 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96361]: 2024-04-22 23:53:17.012112774 +0000 UTC m=+1.191159930 container cleanup 67592bd335a28ef596f2d39b6f93f56e262465c4154330e5d100d9a31840f898 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T23:53:17.171 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 bash[96361]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4 2024-04-22T23:53:17.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:17 smithi031 ceph-mon[111289]: pgmap v143: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:53:17.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:17 smithi031 ceph-mon[111289]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:53:17.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:17 smithi031 ceph-mon[111289]: osdmap e77: 6 total, 5 up, 6 in 2024-04-22T23:53:17.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:17 smithi044 ceph-mon[82146]: pgmap v143: 65 pgs: 65 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:53:17.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:17 smithi044 ceph-mon[82146]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:53:17.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:17 smithi044 ceph-mon[82146]: osdmap e77: 6 total, 5 up, 6 in 2024-04-22T23:53:17.460 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96379]: 2024-04-22 23:53:17.17990885 +0000 UTC m=+0.545143492 container remove 67592bd335a28ef596f2d39b6f93f56e262465c4154330e5d100d9a31840f898 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True) 2024-04-22T23:53:17.857 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96472]: 2024-04-22 23:53:17.502259788 +0000 UTC m=+0.018298001 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:53:17.857 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96472]: 2024-04-22 23:53:17.605848782 +0000 UTC m=+0.121886960 container create a57e08e79de0a3ca2ef8c933edbb40d7342a7b739dc106fe0475a94002bc5b08 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:53:17.857 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96472]: 2024-04-22 23:53:17.808611171 +0000 UTC m=+0.324649349 container init a57e08e79de0a3ca2ef8c933edbb40d7342a7b739dc106fe0475a94002bc5b08 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:53:17.857 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96472]: 2024-04-22 23:53:17.812004635 +0000 UTC m=+0.328042814 container start a57e08e79de0a3ca2ef8c933edbb40d7342a7b739dc106fe0475a94002bc5b08 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T23:53:17.857 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:17 smithi044 podman[96472]: 2024-04-22 23:53:17.866819572 +0000 UTC m=+0.382857760 container attach a57e08e79de0a3ca2ef8c933edbb40d7342a7b739dc106fe0475a94002bc5b08 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:53:18.184 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:18 smithi044 podman[96472]: 2024-04-22 23:53:18.11622621 +0000 UTC m=+0.632264397 container died a57e08e79de0a3ca2ef8c933edbb40d7342a7b739dc106fe0475a94002bc5b08 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T23:53:18.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:18 smithi031 ceph-mon[111289]: osdmap e78: 6 total, 5 up, 6 in 2024-04-22T23:53:18.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:18 smithi044 ceph-mon[82146]: osdmap e78: 6 total, 5 up, 6 in 2024-04-22T23:53:19.185 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:18 smithi044 podman[96498]: 2024-04-22 23:53:18.795799338 +0000 UTC m=+0.673415181 container remove a57e08e79de0a3ca2ef8c933edbb40d7342a7b739dc106fe0475a94002bc5b08 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-22T23:53:19.185 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:18 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-22T23:53:19.185 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:18 smithi044 systemd[1]: Stopped Ceph osd.4 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:53:19.185 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:18 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.4.service: Consumed 54.541s CPU time. 2024-04-22T23:53:19.185 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 systemd[1]: Starting Ceph osd.4 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:53:19.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-mon[82146]: pgmap v146: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-22T23:53:19.441 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-mon[82146]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:53:19.441 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 podman[96606]: 2024-04-22 23:53:19.139257328 +0000 UTC m=+0.018480140 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:53:19.441 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 podman[96606]: 2024-04-22 23:53:19.238937593 +0000 UTC m=+0.118160436 container create 077d1fbc7cde335612237e795c1d9f268dff994399cd36f44add70bdc6e237b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-22T23:53:19.441 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 podman[96606]: 2024-04-22 23:53:19.450206931 +0000 UTC m=+0.329429731 container init 077d1fbc7cde335612237e795c1d9f268dff994399cd36f44add70bdc6e237b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-22T23:53:19.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:19 smithi031 ceph-mon[111289]: pgmap v146: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-04-22T23:53:19.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:19 smithi031 ceph-mon[111289]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:53:19.709 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 podman[96606]: 2024-04-22 23:53:19.453920722 +0000 UTC m=+0.333143525 container start 077d1fbc7cde335612237e795c1d9f268dff994399cd36f44add70bdc6e237b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-22T23:53:19.710 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 podman[96606]: 2024-04-22 23:53:19.524905804 +0000 UTC m=+0.404128604 container attach 077d1fbc7cde335612237e795c1d9f268dff994399cd36f44add70bdc6e237b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:53:20.210 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 bash[96606]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate[96620]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-22T23:53:20.211 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:19 smithi044 podman[96606]: 2024-04-22 23:53:19.972027238 +0000 UTC m=+0.851250042 container died 077d1fbc7cde335612237e795c1d9f268dff994399cd36f44add70bdc6e237b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:53:20.585 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:20 smithi044 ceph-mon[82146]: Health check failed: Degraded data redundancy: 22/264 objects degraded (8.333%), 6 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:20.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:20 smithi031 ceph-mon[111289]: Health check failed: Degraded data redundancy: 22/264 objects degraded (8.333%), 6 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:20.836 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:20 smithi044 podman[96762]: 2024-04-22 23:53:20.595159089 +0000 UTC m=+0.617446086 container remove 077d1fbc7cde335612237e795c1d9f268dff994399cd36f44add70bdc6e237b6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T23:53:20.837 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:20 smithi044 podman[96824]: 2024-04-22 23:53:20.719828829 +0000 UTC m=+0.018376448 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:53:20.837 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:20 smithi044 podman[96824]: 2024-04-22 23:53:20.846258783 +0000 UTC m=+0.144806373 container create 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-22T23:53:21.105 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:21 smithi044 podman[96824]: 2024-04-22 23:53:21.090474415 +0000 UTC m=+0.389022007 container init 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=) 2024-04-22T23:53:21.105 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:21 smithi044 podman[96824]: 2024-04-22 23:53:21.094562048 +0000 UTC m=+0.393109640 container start 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:53:21.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:21 smithi044 ceph-mon[82146]: pgmap v147: 65 pgs: 8 active+undersized, 6 active+undersized+degraded, 2 peering, 2 stale+active+clean, 47 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 22/264 objects degraded (8.333%) 2024-04-22T23:53:21.460 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:21 smithi044 ceph-osd[96844]: -- 172.21.15.44:0/2877553185 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9684e5680 con 0x55e9684aec00 2024-04-22T23:53:21.460 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:21 smithi044 bash[96824]: 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 2024-04-22T23:53:21.460 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:21 smithi044 systemd[1]: Started Ceph osd.4 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:53:21.492 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:21 smithi031 ceph-mon[111289]: pgmap v147: 65 pgs: 8 active+undersized, 6 active+undersized+degraded, 2 peering, 2 stale+active+clean, 47 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 22/264 objects degraded (8.333%) 2024-04-22T23:53:21.959 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:21 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-22T23:53:21.636+0000 7faf69b5a7c0 -1 Falling back to public interface 2024-04-22T23:53:22.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:22 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:22.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:22 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:22.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:22 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:53:22.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:22 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:22.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:22 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:22.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:22 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:53:23.588 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:23 smithi044 ceph-mon[82146]: pgmap v148: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-22T23:53:23.588 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:23 smithi044 ceph-mon[82146]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:53:23.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:23 smithi031 ceph-mon[111289]: pgmap v148: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-22T23:53:23.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:23 smithi031 ceph-mon[111289]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:53:25.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:25 smithi031 ceph-mon[111289]: pgmap v149: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-22T23:53:25.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:25.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:25.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:25 smithi044 ceph-mon[82146]: pgmap v149: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 46/264 objects degraded (17.424%) 2024-04-22T23:53:25.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:25.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:26.346 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:26 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-22T23:53:26.210+0000 7faf69b5a7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-04-22T23:53:26.709 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:26 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-22T23:53:26.544+0000 7faf69b5a7c0 -1 osd.4 76 log_to_monitors true 2024-04-22T23:53:26.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:26.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:26 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:26.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:26.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:26 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:27.709 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 22 23:53:27 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-22T23:53:27.489+0000 7faf5afb2700 -1 osd.4 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:53:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:27 smithi044 ceph-mon[82146]: pgmap v150: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/264 objects degraded (17.424%) 2024-04-22T23:53:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:27 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 46/264 objects degraded (17.424%), 15 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:27 smithi044 ceph-mon[82146]: from='osd.4 [v2:172.21.15.44:6808/2616608091,v1:172.21.15.44:6809/2616608091]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:53:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:27 smithi044 ceph-mon[82146]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:53:27.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:27 smithi031 ceph-mon[111289]: pgmap v150: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/264 objects degraded (17.424%) 2024-04-22T23:53:27.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:27 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 46/264 objects degraded (17.424%), 15 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:27.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:27 smithi031 ceph-mon[111289]: from='osd.4 [v2:172.21.15.44:6808/2616608091,v1:172.21.15.44:6809/2616608091]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:53:27.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:27 smithi031 ceph-mon[111289]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:53:28.831 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:28 smithi044 ceph-mon[82146]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:53:28.832 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:28 smithi044 ceph-mon[82146]: from='osd.4 [v2:172.21.15.44:6808/2616608091,v1:172.21.15.44:6809/2616608091]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:53:28.832 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:28 smithi044 ceph-mon[82146]: osdmap e79: 6 total, 5 up, 6 in 2024-04-22T23:53:28.832 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:28 smithi044 ceph-mon[82146]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:53:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:28 smithi031 ceph-mon[111289]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:53:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:28 smithi031 ceph-mon[111289]: from='osd.4 [v2:172.21.15.44:6808/2616608091,v1:172.21.15.44:6809/2616608091]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:53:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:28 smithi031 ceph-mon[111289]: osdmap e79: 6 total, 5 up, 6 in 2024-04-22T23:53:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:28 smithi031 ceph-mon[111289]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:53:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/264 objects degraded (17.424%) 2024-04-22T23:53:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: osd.4 [v2:172.21.15.44:6808/2616608091,v1:172.21.15.44:6809/2616608091] boot 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: osdmap e80: 6 total, 6 up, 6 in 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: pgmap v152: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 46/264 objects degraded (17.424%) 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: osd.4 [v2:172.21.15.44:6808/2616608091,v1:172.21.15.44:6809/2616608091] boot 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: osdmap e80: 6 total, 6 up, 6 in 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:29.961 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:29.962 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:30.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:30 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:30.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:30 smithi031 ceph-mon[111289]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:53:30.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:30 smithi031 ceph-mon[111289]: osdmap e81: 6 total, 6 up, 6 in 2024-04-22T23:53:30.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:30.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:30.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:30 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:30.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:30 smithi044 ceph-mon[82146]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:53:30.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:30 smithi044 ceph-mon[82146]: osdmap e81: 6 total, 6 up, 6 in 2024-04-22T23:53:30.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:30.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:31.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:31 smithi031 ceph-mon[111289]: pgmap v155: 65 pgs: 11 peering, 11 active+undersized, 10 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 2 op/s; 27/264 objects degraded (10.227%) 2024-04-22T23:53:31.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:31 smithi044 ceph-mon[82146]: pgmap v155: 65 pgs: 11 peering, 11 active+undersized, 10 active+undersized+degraded, 33 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 2 op/s; 27/264 objects degraded (10.227%) 2024-04-22T23:53:32.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:32 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 27/264 objects degraded (10.227%), 10 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:32.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:32 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 27/264 objects degraded (10.227%), 10 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:33.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:33 smithi044 ceph-mon[82146]: pgmap v156: 65 pgs: 11 peering, 4 active+undersized, 2 active+undersized+degraded, 48 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 5/264 objects degraded (1.894%) 2024-04-22T23:53:33.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:33 smithi031 ceph-mon[111289]: pgmap v156: 65 pgs: 11 peering, 4 active+undersized, 2 active+undersized+degraded, 48 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 1 op/s; 5/264 objects degraded (1.894%) 2024-04-22T23:53:34.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:34 smithi031 ceph-mon[111289]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/264 objects degraded (1.894%), 2 pgs degraded) 2024-04-22T23:53:34.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:34 smithi044 ceph-mon[82146]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/264 objects degraded (1.894%), 2 pgs degraded) 2024-04-22T23:53:35.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:35 smithi031 ceph-mon[111289]: pgmap v157: 65 pgs: 5 peering, 60 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-22T23:53:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:35 smithi044 ceph-mon[82146]: pgmap v157: 65 pgs: 5 peering, 60 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 -- 172.21.15.31:0/1698747585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0068580 msgr2=0x7fbbb00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/1698747585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0068580 0x7fbbb00689e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009910 tx=0x7fbba4009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 -- 172.21.15.31:0/1698747585 shutdown_connections 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/1698747585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0068580 0x7fbbb00689e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/1698747585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbb0106cd0 0x7fbbb01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 -- 172.21.15.31:0/1698747585 >> 172.21.15.31:0/1698747585 conn(0x7fbbb0075670 msgr2=0x7fbbb0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 -- 172.21.15.31:0/1698747585 shutdown_connections 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.601+0000 7fbbb51d3700 1 -- 172.21.15.31:0/1698747585 wait complete. 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.602+0000 7fbbb51d3700 1 Processor -- start 2024-04-22T23:53:36.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.602+0000 7fbbb51d3700 1 -- start start 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.602+0000 7fbbb51d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbb0068580 0x7fbbb019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.602+0000 7fbbb51d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 0x7fbbb019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.602+0000 7fbbb51d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb019d200 con 0x7fbbb0106cd0 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.602+0000 7fbbb51d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01966b0 con 0x7fbbb0068580 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 0x7fbbb019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 0x7fbbb019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41312/0 (socket says 172.21.15.31:41312) 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbaed9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbb0068580 0x7fbbb019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:36.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 -- 172.21.15.31:0/3679681198 learned_addr learned my addr 172.21.15.31:0/3679681198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 -- 172.21.15.31:0/3679681198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbb0068580 msgr2=0x7fbbb019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbb0068580 0x7fbbb019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 -- 172.21.15.31:0/3679681198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40095f0 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbae59c700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 0x7fbbb019cb70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005f50 tx=0x7fbba400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbba401d070 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbba400b610 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb0196930 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.603+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbba4022470 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.604+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb0196e20 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.605+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb01091d0 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.605+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fbba40225d0 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.605+0000 7fbb97fff700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fbb9806ecc0 0x7fbb98071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.606+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fbba40901b0 con 0x7fbbb0106cd0 2024-04-22T23:53:36.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.606+0000 7fbbaed9d700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fbb9806ecc0 0x7fbb98071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:36.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.606+0000 7fbbaed9d700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fbb9806ecc0 0x7fbb98071180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009c60 tx=0x7fbba0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:36.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.611+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fbba405ca50 con 0x7fbbb0106cd0 2024-04-22T23:53:36.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.858+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbb0197cb0 con 0x7fbb9806ecc0 2024-04-22T23:53:36.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.859+0000 7fbb97fff700 1 -- 172.21.15.31:0/3679681198 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbbb0197cb0 con 0x7fbb9806ecc0 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.861+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fbb9806ecc0 msgr2=0x7fbb98071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.861+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fbb9806ecc0 0x7fbb98071180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009c60 tx=0x7fbba0009260 comp rx=0 tx=0).stop 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.861+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 msgr2=0x7fbbb019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.861+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 0x7fbbb019cb70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005f50 tx=0x7fbba400f690 comp rx=0 tx=0).stop 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 shutdown_connections 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fbb9806ecc0 0x7fbb98071180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbb0068580 0x7fbbb019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 --2- 172.21.15.31:0/3679681198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbb0106cd0 0x7fbbb019cb70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 >> 172.21.15.31:0/3679681198 conn(0x7fbbb0075670 msgr2=0x7fbbb00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 shutdown_connections 2024-04-22T23:53:36.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.862+0000 7fbbb51d3700 1 -- 172.21.15.31:0/3679681198 wait complete. 2024-04-22T23:53:36.865 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:53:36.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 -- 172.21.15.31:0/3272743554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 msgr2=0x7ffbcc1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/3272743554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc1047b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009910 tx=0x7ffbc0009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 -- 172.21.15.31:0/3272743554 shutdown_connections 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/3272743554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc1047b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/3272743554 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbcc1006b0 0x7ffbcc100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 -- 172.21.15.31:0/3272743554 >> 172.21.15.31:0/3272743554 conn(0x7ffbcc074e30 msgr2=0x7ffbcc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 -- 172.21.15.31:0/3272743554 shutdown_connections 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 -- 172.21.15.31:0/3272743554 wait complete. 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.971+0000 7ffbd21b0700 1 Processor -- start 2024-04-22T23:53:36.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbd21b0700 1 -- start start 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbd21b0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbcc1006b0 0x7ffbcc101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbd21b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbd21b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc1022d0 con 0x7ffbcc101060 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbd21b0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc102410 con 0x7ffbcc1006b0 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41338/0 (socket says 172.21.15.31:41338) 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcaffd700 1 -- 172.21.15.31:0/2299262396 learned_addr learned my addr 172.21.15.31:0/2299262396 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcb7fe700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbcc1006b0 0x7ffbcc101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcaffd700 1 -- 172.21.15.31:0/2299262396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbcc1006b0 msgr2=0x7ffbcc101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:36.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcaffd700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbcc1006b0 0x7ffbcc101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.972+0000 7ffbcaffd700 1 -- 172.21.15.31:0/2299262396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00095f0 con 0x7ffbcc101060 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.973+0000 7ffbcaffd700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc101d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00092e0 tx=0x7ffbc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.973+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbc001d070 con 0x7ffbcc101060 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.973+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc102690 con 0x7ffbcc101060 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.973+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbc000f460 con 0x7ffbcc101060 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.973+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbc00175f0 con 0x7ffbcc101060 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.973+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc1a2d60 con 0x7ffbcc101060 2024-04-22T23:53:36.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.974+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc1090a0 con 0x7ffbcc101060 2024-04-22T23:53:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.974+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffbc000f5d0 con 0x7ffbcc101060 2024-04-22T23:53:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.975+0000 7ffbc8ff9700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbb406ecc0 0x7ffbb4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.975+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ffbc00904f0 con 0x7ffbcc101060 2024-04-22T23:53:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.975+0000 7ffbcb7fe700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbb406ecc0 0x7ffbb4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:36.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.975+0000 7ffbcb7fe700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbb406ecc0 0x7ffbb4071180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005fd0 tx=0x7ffbbc00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:36.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:36.980+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ffbc0059160 con 0x7ffbcc101060 2024-04-22T23:53:37.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.208+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbcc0611d0 con 0x7ffbb406ecc0 2024-04-22T23:53:37.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.209+0000 7ffbc8ff9700 1 -- 172.21.15.31:0/2299262396 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ffbcc0611d0 con 0x7ffbb406ecc0 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbb406ecc0 msgr2=0x7ffbb4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbb406ecc0 0x7ffbb4071180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc005fd0 tx=0x7ffbbc00a560 comp rx=0 tx=0).stop 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 msgr2=0x7ffbcc101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc101d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00092e0 tx=0x7ffbc000be20 comp rx=0 tx=0).stop 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 shutdown_connections 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbb406ecc0 0x7ffbb4071180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbcc1006b0 0x7ffbcc101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 --2- 172.21.15.31:0/2299262396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbcc101060 0x7ffbcc101d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 >> 172.21.15.31:0/2299262396 conn(0x7ffbcc074e30 msgr2=0x7ffbcc103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 shutdown_connections 2024-04-22T23:53:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.212+0000 7ffbd21b0700 1 -- 172.21.15.31:0/2299262396 wait complete. 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 -- 172.21.15.31:0/2979767638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b00ffc20 msgr2=0x7f56b0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/2979767638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b00ffc20 0x7f56b0100080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009910 tx=0x7f56a4009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 -- 172.21.15.31:0/2979767638 shutdown_connections 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/2979767638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b00ffc20 0x7f56b0100080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/2979767638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b0105bd0 0x7f56b0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 -- 172.21.15.31:0/2979767638 >> 172.21.15.31:0/2979767638 conn(0x7f56b0075670 msgr2=0x7f56b0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 -- 172.21.15.31:0/2979767638 shutdown_connections 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.321+0000 7f56b4c5a700 1 -- 172.21.15.31:0/2979767638 wait complete. 2024-04-22T23:53:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56b4c5a700 1 Processor -- start 2024-04-22T23:53:37.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56b4c5a700 1 -- start start 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56b4c5a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b00ffc20 0x7f56b0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56b4c5a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 0x7f56b0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56b4c5a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0198320 con 0x7f56b0105bd0 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56b4c5a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0198460 con 0x7f56b00ffc20 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56add9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 0x7f56b0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56add9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 0x7f56b0197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41342/0 (socket says 172.21.15.31:41342) 2024-04-22T23:53:37.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.322+0000 7f56add9b700 1 -- 172.21.15.31:0/3794614571 learned_addr learned my addr 172.21.15.31:0/3794614571 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56ae59c700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b00ffc20 0x7f56b0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56add9b700 1 -- 172.21.15.31:0/3794614571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b00ffc20 msgr2=0x7f56b0197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56add9b700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b00ffc20 0x7f56b0197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56add9b700 1 -- 172.21.15.31:0/3794614571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40095f0 con 0x7f56b0105bd0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56ae59c700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b00ffc20 0x7f56b0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56add9b700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 0x7f56b0197c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006010 tx=0x7f56a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56a401d070 con 0x7f56b0105bd0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01a0770 con 0x7f56b0105bd0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56a400f460 con 0x7f56b0105bd0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56a4017630 con 0x7f56b0105bd0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.323+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b01a0c30 con 0x7f56b0105bd0 2024-04-22T23:53:37.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.324+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b01080d0 con 0x7f56b0105bd0 2024-04-22T23:53:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.325+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f56a4003a60 con 0x7f56b0105bd0 2024-04-22T23:53:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.325+0000 7f569f7fe700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f569806ecc0 0x7f5698071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.325+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f56a40903f0 con 0x7f56b0105bd0 2024-04-22T23:53:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.325+0000 7f56ae59c700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f569806ecc0 0x7f5698071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:37.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.326+0000 7f56ae59c700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f569806ecc0 0x7f5698071180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f56a000bc10 tx=0x7f56a000b5f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:37.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.330+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f56a40590e0 con 0x7f56b0105bd0 2024-04-22T23:53:37.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.548+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f56b0068c70 con 0x7f569806ecc0 2024-04-22T23:53:37.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.559+0000 7f569f7fe700 1 -- 172.21.15.31:0/3794614571 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f56b0068c70 con 0x7f569806ecc0 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 81s ago 16m 25.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (16m) 81s ago 16m 8594k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (10m) 13s ago 15m 8372k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (3m) 81s ago 16m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (3m) 13s ago 15m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 81s ago 16m 80.1M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (11m) 81s ago 11m 16.9M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (10m) 81s ago 10m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (11m) 13s ago 11m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (10m) 13s ago 10m 21.1M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:53:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (8m) 81s ago 17m 508M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (7m) 13s ago 15m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (4m) 81s ago 18m 50.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (3m) 13s ago 15m 41.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 81s ago 16m 15.1M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (6m) 13s ago 15m 15.8M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (2m) 81s ago 14m 96.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 81s ago 14m 126M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (86s) 81s ago 13m 15.1M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (50s) 13s ago 13m 82.0M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (16s) 13s ago 12m 13.6M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (12m) 13s ago 12m 206M 2241M 18.2.1 5be31c24972a a662ce217eae 2024-04-22T23:53:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 81s ago 16m 76.0M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:53:37.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.561+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f569806ecc0 msgr2=0x7f5698071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f569806ecc0 0x7f5698071180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f56a000bc10 tx=0x7f56a000b5f0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 msgr2=0x7f56b0197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 0x7f56b0197c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56a4006010 tx=0x7f56a400be20 comp rx=0 tx=0).stop 2024-04-22T23:53:37.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 shutdown_connections 2024-04-22T23:53:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f569806ecc0 0x7f5698071180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56b00ffc20 0x7f56b0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 --2- 172.21.15.31:0/3794614571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b0105bd0 0x7f56b0197c40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 >> 172.21.15.31:0/3794614571 conn(0x7f56b0075670 msgr2=0x7f56b00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 shutdown_connections 2024-04-22T23:53:37.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.562+0000 7f56b4c5a700 1 -- 172.21.15.31:0/3794614571 wait complete. 2024-04-22T23:53:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 -- 172.21.15.31:0/3006928187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 msgr2=0x7f6ca4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 --2- 172.21.15.31:0/3006928187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca4105fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009910 tx=0x7f6c94009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 -- 172.21.15.31:0/3006928187 shutdown_connections 2024-04-22T23:53:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 --2- 172.21.15.31:0/3006928187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ca40ffc20 0x7f6ca4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 --2- 172.21.15.31:0/3006928187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca4105fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 -- 172.21.15.31:0/3006928187 >> 172.21.15.31:0/3006928187 conn(0x7f6ca4075670 msgr2=0x7f6ca4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 -- 172.21.15.31:0/3006928187 shutdown_connections 2024-04-22T23:53:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.668+0000 7f6caa535700 1 -- 172.21.15.31:0/3006928187 wait complete. 2024-04-22T23:53:37.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6caa535700 1 Processor -- start 2024-04-22T23:53:37.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6caa535700 1 -- start start 2024-04-22T23:53:37.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6caa535700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ca40ffc20 0x7f6ca419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:37.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6caa535700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6caa535700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca419f3f0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6caa535700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41988a0 con 0x7f6ca40ffc20 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6ca37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6ca37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41358/0 (socket says 172.21.15.31:41358) 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6ca37fe700 1 -- 172.21.15.31:0/475414847 learned_addr learned my addr 172.21.15.31:0/475414847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6ca3fff700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ca40ffc20 0x7f6ca419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6ca37fe700 1 -- 172.21.15.31:0/475414847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ca40ffc20 msgr2=0x7f6ca419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.669+0000 7f6ca37fe700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ca40ffc20 0x7f6ca419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6ca37fe700 1 -- 172.21.15.31:0/475414847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940095f0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6ca37fe700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca419ed60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e920 tx=0x7f6c9800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9800c9d0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c98010430 con 0x7f6ca4105bd0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca4198b80 con 0x7f6ca4105bd0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c98018630 con 0x7f6ca4105bd0 2024-04-22T23:53:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.670+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca41990d0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.671+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca404f2e0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.672+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6c980105a0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.672+0000 7f6ca17fa700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6c8c06ecc0 0x7f6c8c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:37.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.672+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f6c98014070 con 0x7f6ca4105bd0 2024-04-22T23:53:37.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.672+0000 7f6ca3fff700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6c8c06ecc0 0x7f6c8c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:37.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.673+0000 7f6ca3fff700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6c8c06ecc0 0x7f6c8c071180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c940059d0 tx=0x7f6c94005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:37.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.677+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6c98054150 con 0x7f6ca4105bd0 2024-04-22T23:53:37.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:37 smithi031 ceph-mon[111289]: pgmap v158: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-22T23:53:37.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:37 smithi031 ceph-mon[111289]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:37.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:37 smithi044 ceph-mon[82146]: pgmap v158: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-22T23:53:37.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:37 smithi044 ceph-mon[82146]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.975+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ca4199db0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.975+0000 7f6ca17fa700 1 -- 172.21.15.31:0/475414847 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f6c9805b9b0 con 0x7f6ca4105bd0 2024-04-22T23:53:37.967 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:53:37.967 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:53:37.967 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:53:37.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 5 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 9 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:53:37.968 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:53:37.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6c8c06ecc0 msgr2=0x7f6c8c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6c8c06ecc0 0x7f6c8c071180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c940059d0 tx=0x7f6c94005940 comp rx=0 tx=0).stop 2024-04-22T23:53:37.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 msgr2=0x7f6ca419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca419ed60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800e920 tx=0x7f6c9800ec30 comp rx=0 tx=0).stop 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 shutdown_connections 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6c8c06ecc0 0x7f6c8c071180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ca40ffc20 0x7f6ca419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 --2- 172.21.15.31:0/475414847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ca4105bd0 0x7f6ca419ed60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 >> 172.21.15.31:0/475414847 conn(0x7f6ca4075670 msgr2=0x7f6ca40686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 shutdown_connections 2024-04-22T23:53:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:37.977+0000 7f6caa535700 1 -- 172.21.15.31:0/475414847 wait complete. 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.084+0000 7ffad07b3700 1 -- 172.21.15.31:0/3885683196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 msgr2=0x7ffac80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.084+0000 7ffad07b3700 1 --2- 172.21.15.31:0/3885683196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac80fffc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffab8009910 tx=0x7ffab8009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 -- 172.21.15.31:0/3885683196 shutdown_connections 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 --2- 172.21.15.31:0/3885683196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac80fffc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 --2- 172.21.15.31:0/3885683196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8105bf0 0x7ffac8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 -- 172.21.15.31:0/3885683196 >> 172.21.15.31:0/3885683196 conn(0x7ffac80fb650 msgr2=0x7ffac80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 -- 172.21.15.31:0/3885683196 shutdown_connections 2024-04-22T23:53:38.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 -- 172.21.15.31:0/3885683196 wait complete. 2024-04-22T23:53:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 Processor -- start 2024-04-22T23:53:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.085+0000 7ffad07b3700 1 -- start start 2024-04-22T23:53:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffad07b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffad07b3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8105bf0 0x7ffac8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffad07b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8102230 con 0x7ffac80ffb40 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffad07b3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8102370 con 0x7ffac8105bf0 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac81017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41378/0 (socket says 172.21.15.31:41378) 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 -- 172.21.15.31:0/2516252321 learned_addr learned my addr 172.21.15.31:0/2516252321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffacdd4e700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8105bf0 0x7ffac8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 -- 172.21.15.31:0/2516252321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8105bf0 msgr2=0x7ffac8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8105bf0 0x7ffac8101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 -- 172.21.15.31:0/2516252321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80095f0 con 0x7ffac80ffb40 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7fface54f700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac81017b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffac400d8d0 tx=0x7ffac400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac4009940 con 0x7ffac80ffb40 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffac4010460 con 0x7ffac80ffb40 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac81a29b0 con 0x7ffac80ffb40 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac400f5f0 con 0x7ffac80ffb40 2024-04-22T23:53:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.086+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac81a2dc0 con 0x7ffac80ffb40 2024-04-22T23:53:38.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.087+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac804f2e0 con 0x7ffac80ffb40 2024-04-22T23:53:38.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.088+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffac40105d0 con 0x7ffac80ffb40 2024-04-22T23:53:38.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.088+0000 7ffabf7fe700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffab406ecc0 0x7ffab4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.088+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ffac408efc0 con 0x7ffac80ffb40 2024-04-22T23:53:38.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.088+0000 7ffacdd4e700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffab406ecc0 0x7ffab4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.089+0000 7ffacdd4e700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffab406ecc0 0x7ffab4071180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffab800b5c0 tx=0x7ffab8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:38.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.093+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ffac4057c30 con 0x7ffac80ffb40 2024-04-22T23:53:38.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.350+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffac81a30e0 con 0x7ffac80ffb40 2024-04-22T23:53:38.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.351+0000 7ffabf7fe700 1 -- 172.21.15.31:0/2516252321 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1902 (secure 0 0 0) 0x7ffac405b2f0 con 0x7ffac80ffb40 2024-04-22T23:53:38.343 INFO:teuthology.orchestra.run.smithi031.stdout:e18 2024-04-22T23:53:38.343 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:53:38.343 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:42:46.131915+0000 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:53:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:53:38.345 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:14490} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.31:6828/748820754,v1:172.21.15.31:6829/748820754] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:14500} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2302742063,v1:172.21.15.31:6827/2302742063] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:53:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffab406ecc0 msgr2=0x7ffab4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffab406ecc0 0x7ffab4071180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffab800b5c0 tx=0x7ffab8005960 comp rx=0 tx=0).stop 2024-04-22T23:53:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 msgr2=0x7ffac81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac81017b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffac400d8d0 tx=0x7ffac400dc90 comp rx=0 tx=0).stop 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 shutdown_connections 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffab406ecc0 0x7ffab4071180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffac80ffb40 0x7ffac81017b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 --2- 172.21.15.31:0/2516252321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8105bf0 0x7ffac8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 >> 172.21.15.31:0/2516252321 conn(0x7ffac80fb650 msgr2=0x7ffac8068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 shutdown_connections 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.353+0000 7ffad07b3700 1 -- 172.21.15.31:0/2516252321 wait complete. 2024-04-22T23:53:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-22T23:53:38.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 -- 172.21.15.31:0/4111216215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda80ffc20 msgr2=0x7feda8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 --2- 172.21.15.31:0/4111216215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda80ffc20 0x7feda8100080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed98009910 tx=0x7fed98009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 -- 172.21.15.31:0/4111216215 shutdown_connections 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 --2- 172.21.15.31:0/4111216215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda80ffc20 0x7feda8100080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 --2- 172.21.15.31:0/4111216215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda8105bd0 0x7feda8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 -- 172.21.15.31:0/4111216215 >> 172.21.15.31:0/4111216215 conn(0x7feda8075670 msgr2=0x7feda8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 -- 172.21.15.31:0/4111216215 shutdown_connections 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.462+0000 7fedb01d2700 1 -- 172.21.15.31:0/4111216215 wait complete. 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedb01d2700 1 Processor -- start 2024-04-22T23:53:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedb01d2700 1 -- start start 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedb01d2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda80ffc20 0x7feda81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedb01d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 0x7feda8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedb01d2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda8198350 con 0x7feda8105bd0 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedb01d2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda8198490 con 0x7feda80ffc20 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedad76d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 0x7feda8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedad76d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 0x7feda8197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41398/0 (socket says 172.21.15.31:41398) 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedad76d700 1 -- 172.21.15.31:0/597335856 learned_addr learned my addr 172.21.15.31:0/597335856 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedadf6e700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda80ffc20 0x7feda81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedad76d700 1 -- 172.21.15.31:0/597335856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda80ffc20 msgr2=0x7feda81977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedad76d700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda80ffc20 0x7feda81977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedad76d700 1 -- 172.21.15.31:0/597335856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980095f0 con 0x7feda8105bd0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.463+0000 7fedadf6e700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda80ffc20 0x7feda81977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.464+0000 7fedad76d700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 0x7feda8197d30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed98006010 tx=0x7fed9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.464+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9801d070 con 0x7feda8105bd0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.464+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed98004fb0 con 0x7feda8105bd0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.464+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda81a0860 con 0x7feda8105bd0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.464+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed98022ea0 con 0x7feda8105bd0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.464+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda81a0c70 con 0x7feda8105bd0 2024-04-22T23:53:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.465+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda81080f0 con 0x7feda8105bd0 2024-04-22T23:53:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.465+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fed98022470 con 0x7feda8105bd0 2024-04-22T23:53:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.466+0000 7fed9effd700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fed9406ecc0 0x7fed94071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.466+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fed980903e0 con 0x7feda8105bd0 2024-04-22T23:53:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.466+0000 7fedadf6e700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fed9406ecc0 0x7fed94071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.466+0000 7fedadf6e700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fed9406ecc0 0x7fed94071180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feda4007900 tx=0x7feda4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.471+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fed98059200 con 0x7feda8105bd0 2024-04-22T23:53:38.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.700+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feda8068c50 con 0x7fed9406ecc0 2024-04-22T23:53:38.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:38 smithi031 ceph-mon[111289]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:38.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:38 smithi031 ceph-mon[111289]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:38.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:38 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/475414847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:38.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:38 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2516252321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:53:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.701+0000 7fed9effd700 1 -- 172.21.15.31:0/597335856 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7feda8068c50 con 0x7fed9406ecc0 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "crash" 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "11/23 daemons upgraded", 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:53:38.694 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:53:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fed9406ecc0 msgr2=0x7fed94071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fed9406ecc0 0x7fed94071180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feda4007900 tx=0x7feda4008040 comp rx=0 tx=0).stop 2024-04-22T23:53:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 msgr2=0x7feda8197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 0x7feda8197d30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed98006010 tx=0x7fed9800be20 comp rx=0 tx=0).stop 2024-04-22T23:53:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 shutdown_connections 2024-04-22T23:53:38.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fed9406ecc0 0x7fed94071180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feda80ffc20 0x7feda81977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 --2- 172.21.15.31:0/597335856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feda8105bd0 0x7feda8197d30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 >> 172.21.15.31:0/597335856 conn(0x7feda8075670 msgr2=0x7feda80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 shutdown_connections 2024-04-22T23:53:38.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.704+0000 7fedb01d2700 1 -- 172.21.15.31:0/597335856 wait complete. 2024-04-22T23:53:38.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 -- 172.21.15.31:0/4076466637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038068580 msgr2=0x7ff0380689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 --2- 172.21.15.31:0/4076466637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038068580 0x7ff0380689e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff020009910 tx=0x7ff020009c20 comp rx=0 tx=0).stop 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 -- 172.21.15.31:0/4076466637 shutdown_connections 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 --2- 172.21.15.31:0/4076466637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038068580 0x7ff0380689e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 --2- 172.21.15.31:0/4076466637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff038106cd0 0x7ff0381070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 -- 172.21.15.31:0/4076466637 >> 172.21.15.31:0/4076466637 conn(0x7ff038075670 msgr2=0x7ff038075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 -- 172.21.15.31:0/4076466637 shutdown_connections 2024-04-22T23:53:38.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.811+0000 7ff03d124700 1 -- 172.21.15.31:0/4076466637 wait complete. 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.812+0000 7ff03d124700 1 Processor -- start 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.812+0000 7ff03d124700 1 -- start start 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff03d124700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff038068580 0x7ff038196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff03d124700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 0x7ff038196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff03d124700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038197550 con 0x7ff038106cd0 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff03d124700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819b360 con 0x7ff038068580 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 0x7ff038196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 0x7ff038196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41416/0 (socket says 172.21.15.31:41416) 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 -- 172.21.15.31:0/1241252868 learned_addr learned my addr 172.21.15.31:0/1241252868 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff036d9d700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff038068580 0x7ff038196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 -- 172.21.15.31:0/1241252868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff038068580 msgr2=0x7ff038196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff038068580 0x7ff038196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 -- 172.21.15.31:0/1241252868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0200095f0 con 0x7ff038106cd0 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff02ffff700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 0x7ff038196ec0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff020006010 tx=0x7ff02000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff02001d070 con 0x7ff038106cd0 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff020004fb0 con 0x7ff038106cd0 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03819b5e0 con 0x7ff038106cd0 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.813+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff020022ea0 con 0x7ff038106cd0 2024-04-22T23:53:38.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.814+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03819bad0 con 0x7ff038106cd0 2024-04-22T23:53:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.814+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0381091f0 con 0x7ff038106cd0 2024-04-22T23:53:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.815+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff020022470 con 0x7ff038106cd0 2024-04-22T23:53:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.815+0000 7ff034d99700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff02406ecc0 0x7ff024071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:53:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.816+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6704+0+0 (secure 0 0 0) 0x7ff020090410 con 0x7ff038106cd0 2024-04-22T23:53:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.816+0000 7ff036d9d700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff02406ecc0 0x7ff024071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:53:38.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.816+0000 7ff036d9d700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff02406ecc0 0x7ff024071180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff028006fd0 tx=0x7ff028008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:53:38.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:38.820+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff02005cd60 con 0x7ff038106cd0 2024-04-22T23:53:38.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:38 smithi044 ceph-mon[82146]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:38.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:38 smithi044 ceph-mon[82146]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:38.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:38 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/475414847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:53:38.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:38 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2516252321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:53:39.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.119+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff038197ce0 con 0x7ff038106cd0 2024-04-22T23:53:39.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.120+0000 7ff034d99700 1 -- 172.21.15.31:0/1241252868 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff020022720 con 0x7ff038106cd0 2024-04-22T23:53:39.112 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:53:39.112 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:53:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:53:39.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff02406ecc0 msgr2=0x7ff024071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:39.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff02406ecc0 0x7ff024071180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff028006fd0 tx=0x7ff028008040 comp rx=0 tx=0).stop 2024-04-22T23:53:39.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 msgr2=0x7ff038196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:53:39.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 0x7ff038196ec0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff020006010 tx=0x7ff02000be20 comp rx=0 tx=0).stop 2024-04-22T23:53:39.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 shutdown_connections 2024-04-22T23:53:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff02406ecc0 0x7ff024071180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff038068580 0x7ff038196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 --2- 172.21.15.31:0/1241252868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff038106cd0 0x7ff038196ec0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:53:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 >> 172.21.15.31:0/1241252868 conn(0x7ff038075670 msgr2=0x7ff0380fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:53:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 shutdown_connections 2024-04-22T23:53:39.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:53:39.122+0000 7ff03d124700 1 -- 172.21.15.31:0/1241252868 wait complete. 2024-04-22T23:53:39.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:39 smithi031 ceph-mon[111289]: pgmap v159: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 953 B/s rd, 1 op/s 2024-04-22T23:53:39.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:39 smithi031 ceph-mon[111289]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:39.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:39 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1241252868' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:53:39.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:39 smithi044 ceph-mon[82146]: pgmap v159: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 953 B/s rd, 1 op/s 2024-04-22T23:53:39.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:39 smithi044 ceph-mon[82146]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:53:39.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:39 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1241252868' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:53:41.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:41 smithi031 ceph-mon[111289]: pgmap v160: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 960 B/s rd, 1 op/s 2024-04-22T23:53:41.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:41 smithi044 ceph-mon[82146]: pgmap v160: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 960 B/s rd, 1 op/s 2024-04-22T23:53:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:43 smithi031 ceph-mon[111289]: pgmap v161: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:53:43.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:43 smithi044 ceph-mon[82146]: pgmap v161: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:53:44.894 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:44 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:44.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:44 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:45.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:45 smithi031 ceph-mon[111289]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:45.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:45 smithi031 ceph-mon[111289]: Upgrade: osd.5 is safe to restart 2024-04-22T23:53:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:45 smithi031 ceph-mon[111289]: pgmap v162: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:53:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:45 smithi044 ceph-mon[82146]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:53:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:45 smithi044 ceph-mon[82146]: Upgrade: osd.5 is safe to restart 2024-04-22T23:53:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:45 smithi044 ceph-mon[82146]: pgmap v162: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:53:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:53:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:46.870 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:46 smithi044 ceph-mon[82146]: Upgrade: Updating osd.5 2024-04-22T23:53:46.870 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:46 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:46.870 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:46 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:53:46.870 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:46 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:53:46.870 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:46 smithi044 ceph-mon[82146]: Deploying daemon osd.5 on smithi044 2024-04-22T23:53:46.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:46 smithi031 ceph-mon[111289]: Upgrade: Updating osd.5 2024-04-22T23:53:46.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:46 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:46.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:46 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:53:46.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:46 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:53:46.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:46 smithi031 ceph-mon[111289]: Deploying daemon osd.5 on smithi044 2024-04-22T23:53:47.806 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:47 smithi044 ceph-mon[82146]: pgmap v163: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:53:47.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:47 smithi031 ceph-mon[111289]: pgmap v163: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:53:49.166 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:48 smithi044 systemd[1]: Stopping Ceph osd.5 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:53:49.459 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:49 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[53944]: 2024-04-22T23:53:49.174+0000 7fdae6841700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:53:49.459 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:49 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[53944]: 2024-04-22T23:53:49.174+0000 7fdae6841700 -1 osd.5 81 *** Got signal Terminated *** 2024-04-22T23:53:49.460 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:49 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[53944]: 2024-04-22T23:53:49.174+0000 7fdae6841700 -1 osd.5 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:53:49.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:49 smithi031 ceph-mon[111289]: pgmap v164: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:53:49.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:49 smithi031 ceph-mon[111289]: osd.5 marked itself down and dead 2024-04-22T23:53:49.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:49 smithi044 ceph-mon[82146]: pgmap v164: 65 pgs: 65 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:53:49.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:49 smithi044 ceph-mon[82146]: osd.5 marked itself down and dead 2024-04-22T23:53:50.459 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:50 smithi044 podman[101682]: 2024-04-22 23:53:50.232359092 +0000 UTC m=+1.166184888 container died a662ce217eaea9b765a70f27c928b2bdec4f921826f4be6bf165ae2afd7d0e17 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T23:53:50.866 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:50 smithi044 ceph-mon[82146]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:53:50.867 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:50 smithi044 ceph-mon[82146]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-22T23:53:50.867 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:50 smithi044 ceph-mon[82146]: osdmap e82: 6 total, 5 up, 6 in 2024-04-22T23:53:50.867 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:50 smithi044 podman[101682]: 2024-04-22 23:53:50.583625259 +0000 UTC m=+1.517451054 container cleanup a662ce217eaea9b765a70f27c928b2bdec4f921826f4be6bf165ae2afd7d0e17 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:53:50.867 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:50 smithi044 bash[101682]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5 2024-04-22T23:53:50.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:50 smithi031 ceph-mon[111289]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:53:50.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:50 smithi031 ceph-mon[111289]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-22T23:53:50.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:50 smithi031 ceph-mon[111289]: osdmap e82: 6 total, 5 up, 6 in 2024-04-22T23:53:51.201 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:50 smithi044 podman[101711]: 2024-04-22 23:53:50.876464391 +0000 UTC m=+0.642361498 container remove a662ce217eaea9b765a70f27c928b2bdec4f921826f4be6bf165ae2afd7d0e17 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, ceph=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-04-22T23:53:51.459 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:51 smithi044 podman[101800]: 2024-04-22 23:53:51.210932674 +0000 UTC m=+0.112071672 container create b3fe3759e85c96ed33bd9d0832e176f316f0e6bc4b486df6e0a31171d71bec44 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T23:53:51.460 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:51 smithi044 podman[101800]: 2024-04-22 23:53:51.117242436 +0000 UTC m=+0.018381463 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:53:51.460 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:51 smithi044 podman[101800]: 2024-04-22 23:53:51.430051139 +0000 UTC m=+0.331190137 container init b3fe3759e85c96ed33bd9d0832e176f316f0e6bc4b486df6e0a31171d71bec44 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:53:51.460 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:51 smithi044 podman[101800]: 2024-04-22 23:53:51.43343205 +0000 UTC m=+0.334571050 container start b3fe3759e85c96ed33bd9d0832e176f316f0e6bc4b486df6e0a31171d71bec44 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-22T23:53:51.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:51 smithi031 ceph-mon[111289]: pgmap v166: 65 pgs: 13 stale+active+clean, 52 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-22T23:53:51.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:51 smithi031 ceph-mon[111289]: osdmap e83: 6 total, 5 up, 6 in 2024-04-22T23:53:51.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:51 smithi044 ceph-mon[82146]: pgmap v166: 65 pgs: 13 stale+active+clean, 52 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-04-22T23:53:51.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:51 smithi044 ceph-mon[82146]: osdmap e83: 6 total, 5 up, 6 in 2024-04-22T23:53:51.960 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:51 smithi044 podman[101800]: 2024-04-22 23:53:51.522101416 +0000 UTC m=+0.423240414 container attach b3fe3759e85c96ed33bd9d0832e176f316f0e6bc4b486df6e0a31171d71bec44 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-22T23:53:51.960 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:51 smithi044 podman[101800]: 2024-04-22 23:53:51.746779235 +0000 UTC m=+0.647918245 container died b3fe3759e85c96ed33bd9d0832e176f316f0e6bc4b486df6e0a31171d71bec44 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:53:52.708 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 podman[101827]: 2024-04-22 23:53:52.300589423 +0000 UTC m=+0.545241191 container remove b3fe3759e85c96ed33bd9d0832e176f316f0e6bc4b486df6e0a31171d71bec44 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-22T23:53:52.708 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-22T23:53:52.708 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 systemd[1]: Stopped Ceph osd.5 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:53:52.708 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.5.service: Consumed 56.559s CPU time. 2024-04-22T23:53:52.708 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 systemd[1]: Starting Ceph osd.5 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-22T23:53:52.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:52 smithi044 ceph-mon[82146]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:53:52.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:52 smithi044 ceph-mon[82146]: Health check failed: Degraded data redundancy: 8/264 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:52.960 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 podman[101929]: 2024-04-22 23:53:52.657043286 +0000 UTC m=+0.018386844 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:53:52.960 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:52 smithi044 podman[101929]: 2024-04-22 23:53:52.776841823 +0000 UTC m=+0.138185369 container create 4330c06bd8fc4c9b372c2130bbf2a5d08f4abc16a2d73a2a905fc09c4cb66bc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:53:53.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:52 smithi031 ceph-mon[111289]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:53:53.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:52 smithi031 ceph-mon[111289]: Health check failed: Degraded data redundancy: 8/264 objects degraded (3.030%), 3 pgs degraded (PG_DEGRADED) 2024-04-22T23:53:53.459 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 podman[101929]: 2024-04-22 23:53:53.004623 +0000 UTC m=+0.365966546 container init 4330c06bd8fc4c9b372c2130bbf2a5d08f4abc16a2d73a2a905fc09c4cb66bc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:53:53.460 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 podman[101929]: 2024-04-22 23:53:53.008005604 +0000 UTC m=+0.369349151 container start 4330c06bd8fc4c9b372c2130bbf2a5d08f4abc16a2d73a2a905fc09c4cb66bc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-22T23:53:53.460 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 podman[101929]: 2024-04-22 23:53:53.079608102 +0000 UTC m=+0.440951662 container attach 4330c06bd8fc4c9b372c2130bbf2a5d08f4abc16a2d73a2a905fc09c4cb66bc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-22T23:53:53.933 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-mon[82146]: pgmap v168: 65 pgs: 3 active+undersized, 3 active+undersized+degraded, 4 peering, 9 stale+active+clean, 46 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8/264 objects degraded (3.030%) 2024-04-22T23:53:53.933 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:53.933 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:53.933 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:53.933 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:53:53.933 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:53:53.933 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 bash[101929]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate[101944]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-22T23:53:53.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:53 smithi044 podman[101929]: 2024-04-22 23:53:53.525859036 +0000 UTC m=+0.887202592 container died 4330c06bd8fc4c9b372c2130bbf2a5d08f4abc16a2d73a2a905fc09c4cb66bc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:53:54.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:53 smithi031 ceph-mon[111289]: pgmap v168: 65 pgs: 3 active+undersized, 3 active+undersized+degraded, 4 peering, 9 stale+active+clean, 46 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 8/264 objects degraded (3.030%) 2024-04-22T23:53:54.500 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 podman[102070]: 2024-04-22 23:53:54.25903094 +0000 UTC m=+0.727302779 container remove 4330c06bd8fc4c9b372c2130bbf2a5d08f4abc16a2d73a2a905fc09c4cb66bc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-activate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:53:54.500 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 podman[102133]: 2024-04-22 23:53:54.384790051 +0000 UTC m=+0.018863684 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:53:54.500 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 podman[102133]: 2024-04-22 23:53:54.509916305 +0000 UTC m=+0.143989936 container create 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-22T23:53:54.802 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 podman[102133]: 2024-04-22 23:53:54.720709086 +0000 UTC m=+0.354782717 container init 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:53:54.803 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 podman[102133]: 2024-04-22 23:53:54.72411419 +0000 UTC m=+0.358187820 container start 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:53:55.209 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 bash[102133]: 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 2024-04-22T23:53:55.209 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:54 smithi044 systemd[1]: Started Ceph osd.5 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-22T23:53:55.695 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:55 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-22T23:53:55.275+0000 7f15c12897c0 -1 Falling back to public interface 2024-04-22T23:53:55.948 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:55 smithi044 ceph-mon[82146]: pgmap v169: 65 pgs: 12 active+undersized, 11 active+undersized+degraded, 4 peering, 2 stale+active+clean, 36 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 37/264 objects degraded (14.015%) 2024-04-22T23:53:55.948 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:55.948 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:55.948 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:53:56.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:55 smithi031 ceph-mon[111289]: pgmap v169: 65 pgs: 12 active+undersized, 11 active+undersized+degraded, 4 peering, 2 stale+active+clean, 36 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 37/264 objects degraded (14.015%) 2024-04-22T23:53:56.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:56.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:56.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:53:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:57 smithi031 ceph-mon[111289]: pgmap v170: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-22T23:53:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:57 smithi031 ceph-mon[111289]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:53:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:57 smithi044 ceph-mon[82146]: pgmap v170: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-22T23:53:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:57 smithi044 ceph-mon[82146]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:53:59.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:59.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:53:59.709 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-22T23:53:59.452+0000 7f15c12897c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-04-22T23:54:00.102 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-22T23:53:59.816+0000 7f15c12897c0 -1 osd.5 81 log_to_monitors true 2024-04-22T23:54:00.103 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-mon[82146]: pgmap v171: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-22T23:54:00.103 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:00.103 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:00.103 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-mon[82146]: from='osd.5 [v2:172.21.15.44:6816/2666243870,v1:172.21.15.44:6817/2666243870]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:54:00.103 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:53:59 smithi044 ceph-mon[82146]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:54:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:59 smithi031 ceph-mon[111289]: pgmap v171: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 255 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-22T23:54:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:00.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:00.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:59 smithi031 ceph-mon[111289]: from='osd.5 [v2:172.21.15.44:6816/2666243870,v1:172.21.15.44:6817/2666243870]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:54:00.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:53:59 smithi031 ceph-mon[111289]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:54:00.459 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 22 23:54:00 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-22T23:54:00.133+0000 7f15b26e1700 -1 osd.5 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: from='osd.5 [v2:172.21.15.44:6816/2666243870,v1:172.21.15.44:6817/2666243870]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: osdmap e84: 6 total, 5 up, 6 in 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: pgmap v173: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:01.378 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:54:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: from='osd.5 [v2:172.21.15.44:6816/2666243870,v1:172.21.15.44:6817/2666243870]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:54:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: osdmap e84: 6 total, 5 up, 6 in 2024-04-22T23:54:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-22T23:54:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: pgmap v173: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 45/264 objects degraded (17.045%) 2024-04-22T23:54:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:02.381 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:02 smithi044 ceph-mon[82146]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:54:02.381 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:02 smithi044 ceph-mon[82146]: osd.5 [v2:172.21.15.44:6816/2666243870,v1:172.21.15.44:6817/2666243870] boot 2024-04-22T23:54:02.381 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:02 smithi044 ceph-mon[82146]: osdmap e85: 6 total, 6 up, 6 in 2024-04-22T23:54:02.381 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:54:02.381 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:02 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 45/264 objects degraded (17.045%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:54:02.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:02 smithi031 ceph-mon[111289]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:54:02.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:02 smithi031 ceph-mon[111289]: osd.5 [v2:172.21.15.44:6816/2666243870,v1:172.21.15.44:6817/2666243870] boot 2024-04-22T23:54:02.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:02 smithi031 ceph-mon[111289]: osdmap e85: 6 total, 6 up, 6 in 2024-04-22T23:54:02.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:54:02.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:02 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 45/264 objects degraded (17.045%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: osdmap e86: 6 total, 6 up, 6 in 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: pgmap v176: 65 pgs: 3 peering, 15 active+undersized, 12 active+undersized+degraded, 35 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 38/264 objects degraded (14.394%) 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all osd 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-22T23:54:03.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-22T23:54:03.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-22T23:54:03.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-22T23:54:03.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-22T23:54:03.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-22T23:54:03.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: osdmap e86: 6 total, 6 up, 6 in 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: pgmap v176: 65 pgs: 3 peering, 15 active+undersized, 12 active+undersized+degraded, 35 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 38/264 objects degraded (14.394%) 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all osd 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-22T23:54:03.461 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-22T23:54:03.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-22T23:54:03.462 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-22T23:54:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:04 smithi031 ceph-mon[111289]: Upgrade: Setting require_osd_release to 19 squid 2024-04-22T23:54:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:04 smithi031 ceph-mon[111289]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-22T23:54:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:04 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-22T23:54:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:04 smithi031 ceph-mon[111289]: osdmap e87: 6 total, 6 up, 6 in 2024-04-22T23:54:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:04 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:04.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:04 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-22T23:54:04.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:04 smithi044 ceph-mon[82146]: Upgrade: Setting require_osd_release to 19 squid 2024-04-22T23:54:04.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:04 smithi044 ceph-mon[82146]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-22T23:54:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:04 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-22T23:54:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:04 smithi044 ceph-mon[82146]: osdmap e87: 6 total, 6 up, 6 in 2024-04-22T23:54:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:04 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:04.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:04 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-22T23:54:05.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-22T23:54:05.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: osdmap e88: 6 total, 6 up, 6 in 2024-04-22T23:54:05.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: pgmap v179: 65 pgs: 3 peering, 11 active+undersized, 6 active+undersized+degraded, 45 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 18/264 objects degraded (6.818%) 2024-04-22T23:54:05.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-22T23:54:05.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 2 up:standby 2024-04-22T23:54:05.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:05.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:05.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:05.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:05 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-22T23:54:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: osdmap e88: 6 total, 6 up, 6 in 2024-04-22T23:54:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: pgmap v179: 65 pgs: 3 peering, 11 active+undersized, 6 active+undersized+degraded, 45 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 0 op/s; 18/264 objects degraded (6.818%) 2024-04-22T23:54:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-22T23:54:05.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 2 up:standby 2024-04-22T23:54:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:05.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:05 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:06.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.31:6828/367136352,v1:172.21.15.31:6829/367136352] up:boot 2024-04-22T23:54:06.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:54:06.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:54:06.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.31:6828/367136352,v1:172.21.15.31:6829/367136352] up:boot 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:54:06.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:54:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: pgmap v180: 65 pgs: 3 peering, 62 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 814 B/s rd, 1 op/s 2024-04-22T23:54:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 6 pgs degraded (PG_DEGRADED) 2024-04-22T23:54:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: Upgrade: Updating mds.cephfs.smithi031.ezuwxk 2024-04-22T23:54:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:07.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:54:07.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:07.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:07 smithi031 ceph-mon[111289]: Deploying daemon mds.cephfs.smithi031.ezuwxk on smithi031 2024-04-22T23:54:07.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: pgmap v180: 65 pgs: 3 peering, 62 active+clean; 255 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 814 B/s rd, 1 op/s 2024-04-22T23:54:07.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: Health check update: Degraded data redundancy: 18/264 objects degraded (6.818%), 6 pgs degraded (PG_DEGRADED) 2024-04-22T23:54:07.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: Upgrade: Updating mds.cephfs.smithi031.ezuwxk 2024-04-22T23:54:07.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.ezuwxk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:54:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:07.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:07 smithi044 ceph-mon[82146]: Deploying daemon mds.cephfs.smithi031.ezuwxk on smithi031 2024-04-22T23:54:08.330 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:08 smithi031 ceph-mon[111289]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 6 pgs degraded) 2024-04-22T23:54:08.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:08 smithi044 ceph-mon[82146]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/264 objects degraded (6.818%), 6 pgs degraded) 2024-04-22T23:54:09.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 -- 172.21.15.31:0/2531744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4100ea0 msgr2=0x7f14c4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 --2- 172.21.15.31:0/2531744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4100ea0 0x7f14c4101280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009910 tx=0x7f14b4009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:09.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 -- 172.21.15.31:0/2531744737 shutdown_connections 2024-04-22T23:54:09.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 --2- 172.21.15.31:0/2531744737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c4101850 0x7f14c41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 --2- 172.21.15.31:0/2531744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4100ea0 0x7f14c4101280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 -- 172.21.15.31:0/2531744737 >> 172.21.15.31:0/2531744737 conn(0x7f14c40fc750 msgr2=0x7f14c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 -- 172.21.15.31:0/2531744737 shutdown_connections 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.240+0000 7f14c96f9700 1 -- 172.21.15.31:0/2531744737 wait complete. 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c96f9700 1 Processor -- start 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c96f9700 1 -- start start 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c96f9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c4100ea0 0x7f14c4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c96f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 0x7f14c40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c96f9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4073b10 con 0x7f14c4101850 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c96f9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4073c50 con 0x7f14c4100ea0 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 0x7f14c40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 0x7f14c40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34992/0 (socket says 172.21.15.31:34992) 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.241+0000 7f14c27fc700 1 -- 172.21.15.31:0/392630306 learned_addr learned my addr 172.21.15.31:0/392630306 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c2ffd700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c4100ea0 0x7f14c4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c27fc700 1 -- 172.21.15.31:0/392630306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c4100ea0 msgr2=0x7f14c4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c27fc700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c4100ea0 0x7f14c4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c27fc700 1 -- 172.21.15.31:0/392630306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40095f0 con 0x7f14c4101850 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c27fc700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 0x7f14c40735d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c960 tx=0x7f14ac00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14ac007a70 con 0x7f14c4101850 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c41a2af0 con 0x7f14c4101850 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14ac007bd0 con 0x7f14c4101850 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14ac0186c0 con 0x7f14c4101850 2024-04-22T23:54:09.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.242+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c41a2e50 con 0x7f14c4101850 2024-04-22T23:54:09.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.243+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4109240 con 0x7f14c4101850 2024-04-22T23:54:09.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.244+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f14ac00faa0 con 0x7f14c4101850 2024-04-22T23:54:09.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.244+0000 7f14bbfff700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f14b006ecc0 0x7f14b0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.244+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f14ac08ef50 con 0x7f14c4101850 2024-04-22T23:54:09.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.244+0000 7f14c2ffd700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f14b006ecc0 0x7f14b0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.245+0000 7f14c2ffd700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f14b006ecc0 0x7f14b0071180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b5c0 tx=0x7f14b401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.249+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f14ac05b770 con 0x7f14c4101850 2024-04-22T23:54:09.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:09 smithi031 ceph-mon[111289]: pgmap v181: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 679 B/s rd, 1 op/s 2024-04-22T23:54:09.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:09 smithi044 ceph-mon[82146]: pgmap v181: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 679 B/s rd, 1 op/s 2024-04-22T23:54:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.479+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c4073e10 con 0x7f14b006ecc0 2024-04-22T23:54:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.481+0000 7f14bbfff700 1 -- 172.21.15.31:0/392630306 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f14c4073e10 con 0x7f14b006ecc0 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f14b006ecc0 msgr2=0x7f14b0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f14b006ecc0 0x7f14b0071180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f14b400b5c0 tx=0x7f14b401a040 comp rx=0 tx=0).stop 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 msgr2=0x7f14c40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 0x7f14c40735d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c960 tx=0x7f14ac00cd20 comp rx=0 tx=0).stop 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 shutdown_connections 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f14b006ecc0 0x7f14b0071180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14c4100ea0 0x7f14c4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 --2- 172.21.15.31:0/392630306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14c4101850 0x7f14c40735d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 >> 172.21.15.31:0/392630306 conn(0x7f14c40fc750 msgr2=0x7f14c40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:09.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 shutdown_connections 2024-04-22T23:54:09.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.483+0000 7f14c96f9700 1 -- 172.21.15.31:0/392630306 wait complete. 2024-04-22T23:54:09.484 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.594+0000 7fe068997700 1 -- 172.21.15.31:0/431304435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064100ea0 msgr2=0x7fe064101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.594+0000 7fe068997700 1 --2- 172.21.15.31:0/431304435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064100ea0 0x7fe064101280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe054009910 tx=0x7fe054009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.594+0000 7fe068997700 1 -- 172.21.15.31:0/431304435 shutdown_connections 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.594+0000 7fe068997700 1 --2- 172.21.15.31:0/431304435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe064101850 0x7fe0641058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.594+0000 7fe068997700 1 --2- 172.21.15.31:0/431304435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064100ea0 0x7fe064101280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.594+0000 7fe068997700 1 -- 172.21.15.31:0/431304435 >> 172.21.15.31:0/431304435 conn(0x7fe0640fc750 msgr2=0x7fe0640feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.595+0000 7fe068997700 1 -- 172.21.15.31:0/431304435 shutdown_connections 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.595+0000 7fe068997700 1 -- 172.21.15.31:0/431304435 wait complete. 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.595+0000 7fe068997700 1 Processor -- start 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.595+0000 7fe068997700 1 -- start start 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe068997700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe064100ea0 0x7fe064073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe068997700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 0x7fe0640735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe068997700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe064073b10 con 0x7fe064101850 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe068997700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe064073c50 con 0x7fe064100ea0 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 0x7fe0640735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 0x7fe0640735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35004/0 (socket says 172.21.15.31:35004) 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 -- 172.21.15.31:0/1609437157 learned_addr learned my addr 172.21.15.31:0/1609437157 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe06259c700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe064100ea0 0x7fe064073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 -- 172.21.15.31:0/1609437157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe064100ea0 msgr2=0x7fe064073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe064100ea0 0x7fe064073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 -- 172.21.15.31:0/1609437157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540095f0 con 0x7fe064101850 2024-04-22T23:54:09.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.596+0000 7fe061d9b700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 0x7fe0640735d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c960 tx=0x7fe04c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.597+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04c007a70 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.597+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0641a2af0 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.597+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0641a2e50 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.597+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe04c007bd0 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.597+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04c018630 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.598+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe044005320 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.598+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe04c00f450 con 0x7fe064101850 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.599+0000 7fe05b7fe700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe05006ecc0 0x7fe050071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.599+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fe04c08ed60 con 0x7fe064101850 2024-04-22T23:54:09.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.599+0000 7fe06259c700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe05006ecc0 0x7fe050071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.599+0000 7fe06259c700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe05006ecc0 0x7fe050071180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe054006010 tx=0x7fe05401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.604+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe04c057ad0 con 0x7fe064101850 2024-04-22T23:54:09.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.835+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe044000bf0 con 0x7fe05006ecc0 2024-04-22T23:54:09.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.837+0000 7fe05b7fe700 1 -- 172.21.15.31:0/1609437157 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe044000bf0 con 0x7fe05006ecc0 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe05006ecc0 msgr2=0x7fe050071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe05006ecc0 0x7fe050071180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe054006010 tx=0x7fe05401a040 comp rx=0 tx=0).stop 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 msgr2=0x7fe0640735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 0x7fe0640735d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c960 tx=0x7fe04c00cc70 comp rx=0 tx=0).stop 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 shutdown_connections 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe05006ecc0 0x7fe050071180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe064100ea0 0x7fe064073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 --2- 172.21.15.31:0/1609437157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064101850 0x7fe0640735d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 >> 172.21.15.31:0/1609437157 conn(0x7fe0640fc750 msgr2=0x7fe0640fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 shutdown_connections 2024-04-22T23:54:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.839+0000 7fe068997700 1 -- 172.21.15.31:0/1609437157 wait complete. 2024-04-22T23:54:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 -- 172.21.15.31:0/1022831841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 msgr2=0x7f95981070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 --2- 172.21.15.31:0/1022831841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f95981070b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f958800b0a0 tx=0x7f958800b3b0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 -- 172.21.15.31:0/1022831841 shutdown_connections 2024-04-22T23:54:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 --2- 172.21.15.31:0/1022831841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9598068580 0x7f95980689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 --2- 172.21.15.31:0/1022831841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f95981070b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 -- 172.21.15.31:0/1022831841 >> 172.21.15.31:0/1022831841 conn(0x7f9598075670 msgr2=0x7f9598075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 -- 172.21.15.31:0/1022831841 shutdown_connections 2024-04-22T23:54:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.946+0000 7f959ddf0700 1 -- 172.21.15.31:0/1022831841 wait complete. 2024-04-22T23:54:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f959ddf0700 1 Processor -- start 2024-04-22T23:54:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f959ddf0700 1 -- start start 2024-04-22T23:54:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f959ddf0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9598068580 0x7f95980ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f959ddf0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f9598100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f959ddf0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9598100770 con 0x7f9598106cd0 2024-04-22T23:54:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f959ddf0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981008b0 con 0x7f9598068580 2024-04-22T23:54:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f9596ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f9598100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f9596ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f9598100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35024/0 (socket says 172.21.15.31:35024) 2024-04-22T23:54:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f9596ffd700 1 -- 172.21.15.31:0/31221572 learned_addr learned my addr 172.21.15.31:0/31221572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.947+0000 7f95977fe700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9598068580 0x7f95980ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9596ffd700 1 -- 172.21.15.31:0/31221572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9598068580 msgr2=0x7f95980ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9596ffd700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9598068580 0x7f95980ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9596ffd700 1 -- 172.21.15.31:0/31221572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009d00 con 0x7f9598106cd0 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9596ffd700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f9598100230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f958000b4b0 tx=0x7f958000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9580011800 con 0x7f9598106cd0 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9580011e20 con 0x7f9598106cd0 2024-04-22T23:54:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95981a2b50 con 0x7f9598106cd0 2024-04-22T23:54:09.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958000f2e0 con 0x7f9598106cd0 2024-04-22T23:54:09.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.948+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95981a2f60 con 0x7f9598106cd0 2024-04-22T23:54:09.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.949+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95981091d0 con 0x7f9598106cd0 2024-04-22T23:54:09.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.950+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9580010380 con 0x7f9598106cd0 2024-04-22T23:54:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.950+0000 7f9594ff9700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f958406ecc0 0x7f9584071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.950+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f958008ea40 con 0x7f9598106cd0 2024-04-22T23:54:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.950+0000 7f95977fe700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f958406ecc0 0x7f9584071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.951+0000 7f95977fe700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f958406ecc0 0x7f9584071180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f80 tx=0x7f95880060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:09.955+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9580057730 con 0x7f9598106cd0 2024-04-22T23:54:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.178+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95980611d0 con 0x7f958406ecc0 2024-04-22T23:54:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.189+0000 7f9594ff9700 1 -- 172.21.15.31:0/31221572 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f95980611d0 con 0x7f958406ecc0 2024-04-22T23:54:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:54:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 113s ago 16m 25.2M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:54:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (17m) 113s ago 17m 8594k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:54:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (11m) 12s ago 16m 8572k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:54:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (3m) 113s ago 17m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:54:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (3m) 12s ago 16m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:54:10.182 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 113s ago 16m 80.1M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:54:10.182 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (11m) 113s ago 11m 16.9M - 18.2.1 5be31c24972a 2b5a0ca88270 2024-04-22T23:54:10.182 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (11m) 113s ago 11m 92.9M - 18.2.1 5be31c24972a 6449db8e16e7 2024-04-22T23:54:10.182 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (11m) 12s ago 11m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:54:10.182 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (11m) 12s ago 11m 21.3M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:54:10.182 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (8m) 113s ago 18m 508M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (7m) 12s ago 16m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (4m) 113s ago 18m 50.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (4m) 12s ago 15m 41.9M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 113s ago 17m 15.1M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 12s ago 16m 15.6M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 113s ago 15m 96.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:54:10.183 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 113s ago 14m 126M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:54:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (119s) 113s ago 14m 15.1M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:54:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (83s) 12s ago 13m 82.3M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:54:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (49s) 12s ago 13m 80.5M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:54:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (15s) 12s ago 13m 12.6M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 38dba54af5f6 2024-04-22T23:54:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 113s ago 16m 76.0M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:54:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f958406ecc0 msgr2=0x7f9584071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f958406ecc0 0x7f9584071180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9588000f80 tx=0x7f95880060c0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 msgr2=0x7f9598100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f9598100230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f958000b4b0 tx=0x7f958000b7c0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 shutdown_connections 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f958406ecc0 0x7f9584071180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9598068580 0x7f95980ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 --2- 172.21.15.31:0/31221572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9598106cd0 0x7f9598100230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 >> 172.21.15.31:0/31221572 conn(0x7f9598075670 msgr2=0x7f95980fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 shutdown_connections 2024-04-22T23:54:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.192+0000 7f959ddf0700 1 -- 172.21.15.31:0/31221572 wait complete. 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 -- 172.21.15.31:0/3595298065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 msgr2=0x7f5b4c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 --2- 172.21.15.31:0/3595298065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c105890 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009910 tx=0x7f5b40009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 -- 172.21.15.31:0/3595298065 shutdown_connections 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 --2- 172.21.15.31:0/3595298065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c105890 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 --2- 172.21.15.31:0/3595298065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c100e90 0x7f5b4c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 -- 172.21.15.31:0/3595298065 >> 172.21.15.31:0/3595298065 conn(0x7f5b4c0fc740 msgr2=0x7f5b4c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 -- 172.21.15.31:0/3595298065 shutdown_connections 2024-04-22T23:54:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.302+0000 7f5b52878700 1 -- 172.21.15.31:0/3595298065 wait complete. 2024-04-22T23:54:10.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b52878700 1 Processor -- start 2024-04-22T23:54:10.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b52878700 1 -- start start 2024-04-22T23:54:10.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b52878700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c100e90 0x7f5b4c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:10.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b52878700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b52878700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c19d350 con 0x7f5b4c101840 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b52878700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c196800 con 0x7f5b4c100e90 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35034/0 (socket says 172.21.15.31:35034) 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b4b7fe700 1 -- 172.21.15.31:0/3177945037 learned_addr learned my addr 172.21.15.31:0/3177945037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.303+0000 7f5b4bfff700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c100e90 0x7f5b4c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b4b7fe700 1 -- 172.21.15.31:0/3177945037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c100e90 msgr2=0x7f5b4c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b4b7fe700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c100e90 0x7f5b4c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b4b7fe700 1 -- 172.21.15.31:0/3177945037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400095f0 con 0x7f5b4c101840 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b4b7fe700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c19ccc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b40000c00 tx=0x7f5b4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b4001d070 con 0x7f5b4c101840 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b4000f460 con 0x7f5b4c101840 2024-04-22T23:54:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c196a80 con 0x7f5b4c101840 2024-04-22T23:54:10.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b40017610 con 0x7f5b4c101840 2024-04-22T23:54:10.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.304+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c196f70 con 0x7f5b4c101840 2024-04-22T23:54:10.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.305+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c1091e0 con 0x7f5b4c101840 2024-04-22T23:54:10.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.306+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5b4000fa00 con 0x7f5b4c101840 2024-04-22T23:54:10.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.306+0000 7f5b497fa700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5b3406ecc0 0x7f5b34071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:10.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.306+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f5b40090680 con 0x7f5b4c101840 2024-04-22T23:54:10.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.307+0000 7f5b4bfff700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5b3406ecc0 0x7f5b34071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:10.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.307+0000 7f5b4bfff700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5b3406ecc0 0x7f5b34071180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009a10 tx=0x7f5b3c009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:10.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.311+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5b4005cfd0 con 0x7f5b4c101840 2024-04-22T23:54:10.430 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:10 smithi031 ceph-mon[111289]: osdmap e89: 6 total, 6 up, 6 in 2024-04-22T23:54:10.430 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:10 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 2 up:standby 2024-04-22T23:54:10.430 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:10 smithi031 ceph-mon[111289]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:10.431 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:10 smithi031 ceph-mon[111289]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:10.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:10 smithi044 ceph-mon[82146]: osdmap e89: 6 total, 6 up, 6 in 2024-04-22T23:54:10.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:10 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 2 up:standby 2024-04-22T23:54:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:10 smithi044 ceph-mon[82146]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:10.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:10 smithi044 ceph-mon[82146]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.609+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5b4c04f2e0 con 0x7f5b4c101840 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.610+0000 7f5b497fa700 1 -- 172.21.15.31:0/3177945037 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7f5b4005cb60 con 0x7f5b4c101840 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:54:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 10 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:54:10.602 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5b3406ecc0 msgr2=0x7f5b34071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5b3406ecc0 0x7f5b34071180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009a10 tx=0x7f5b3c009190 comp rx=0 tx=0).stop 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 msgr2=0x7f5b4c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c19ccc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b40000c00 tx=0x7f5b4000bd70 comp rx=0 tx=0).stop 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 shutdown_connections 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5b3406ecc0 0x7f5b34071180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b4c100e90 0x7f5b4c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 --2- 172.21.15.31:0/3177945037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c101840 0x7f5b4c19ccc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 >> 172.21.15.31:0/3177945037 conn(0x7f5b4c0fc740 msgr2=0x7f5b4c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:10.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 shutdown_connections 2024-04-22T23:54:10.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.612+0000 7f5b52878700 1 -- 172.21.15.31:0/3177945037 wait complete. 2024-04-22T23:54:10.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 -- 172.21.15.31:0/703634991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 msgr2=0x7fb44c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 --2- 172.21.15.31:0/703634991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c1058a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb434009910 tx=0x7fb434009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 -- 172.21.15.31:0/703634991 shutdown_connections 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 --2- 172.21.15.31:0/703634991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c1058a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 --2- 172.21.15.31:0/703634991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb44c100ea0 0x7fb44c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 -- 172.21.15.31:0/703634991 >> 172.21.15.31:0/703634991 conn(0x7fb44c0fc750 msgr2=0x7fb44c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 -- 172.21.15.31:0/703634991 shutdown_connections 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.722+0000 7fb4513ba700 1 -- 172.21.15.31:0/703634991 wait complete. 2024-04-22T23:54:10.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb4513ba700 1 Processor -- start 2024-04-22T23:54:10.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb4513ba700 1 -- start start 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb4513ba700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb44c100ea0 0x7fb44c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb4513ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb4513ba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c19d240 con 0x7fb44c101850 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb4513ba700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c196780 con 0x7fb44c100ea0 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb44a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.723+0000 7fb44a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37412/0 (socket says 172.21.15.31:37412) 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb44a7fc700 1 -- 172.21.15.31:0/1130008223 learned_addr learned my addr 172.21.15.31:0/1130008223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb44affd700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb44c100ea0 0x7fb44c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb44a7fc700 1 -- 172.21.15.31:0/1130008223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb44c100ea0 msgr2=0x7fb44c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb44a7fc700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb44c100ea0 0x7fb44c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb44a7fc700 1 -- 172.21.15.31:0/1130008223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340095f0 con 0x7fb44c101850 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb44a7fc700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c19cc40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb43400bda0 tx=0x7fb43400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb43401d070 con 0x7fb44c101850 2024-04-22T23:54:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb434004fb0 con 0x7fb44c101850 2024-04-22T23:54:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c196a00 con 0x7fb44c101850 2024-04-22T23:54:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb43400f480 con 0x7fb44c101850 2024-04-22T23:54:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.724+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c196ef0 con 0x7fb44c101850 2024-04-22T23:54:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.725+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c1092a0 con 0x7fb44c101850 2024-04-22T23:54:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.726+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb434022870 con 0x7fb44c101850 2024-04-22T23:54:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.726+0000 7fb443fff700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb43806ecc0 0x7fb438071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.726+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb434090370 con 0x7fb44c101850 2024-04-22T23:54:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.727+0000 7fb44affd700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb43806ecc0 0x7fb438071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.727+0000 7fb44affd700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb43806ecc0 0x7fb438071180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb43c005ea0 tx=0x7fb43c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.731+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb43405ccc0 con 0x7fb44c101850 2024-04-22T23:54:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.988+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb44c04f2e0 con 0x7fb44c101850 2024-04-22T23:54:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.988+0000 7fb443fff700 1 -- 172.21.15.31:0/1130008223 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1707 (secure 0 0 0) 0x7fb4340276c0 con 0x7fb44c101850 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:e21 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 19 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:54:04.995710+0000 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:54:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:54:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24293 members: 24293 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:54:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/367136352,v1:172.21.15.31:6829/367136352] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:54:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb43806ecc0 msgr2=0x7fb438071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb43806ecc0 0x7fb438071180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb43c005ea0 tx=0x7fb43c005e30 comp rx=0 tx=0).stop 2024-04-22T23:54:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 msgr2=0x7fb44c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c19cc40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb43400bda0 tx=0x7fb43400be80 comp rx=0 tx=0).stop 2024-04-22T23:54:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 shutdown_connections 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb43806ecc0 0x7fb438071180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb44c100ea0 0x7fb44c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 --2- 172.21.15.31:0/1130008223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb44c101850 0x7fb44c19cc40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.991+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 >> 172.21.15.31:0/1130008223 conn(0x7fb44c0fc750 msgr2=0x7fb44c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.992+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 shutdown_connections 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:10.992+0000 7fb4513ba700 1 -- 172.21.15.31:0/1130008223 wait complete. 2024-04-22T23:54:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 -- 172.21.15.31:0/3158311873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 msgr2=0x7f8c3c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 --2- 172.21.15.31:0/3158311873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c1070b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009910 tx=0x7f8c2c009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 -- 172.21.15.31:0/3158311873 shutdown_connections 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 --2- 172.21.15.31:0/3158311873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c3c068580 0x7f8c3c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 --2- 172.21.15.31:0/3158311873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c1070b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 -- 172.21.15.31:0/3158311873 >> 172.21.15.31:0/3158311873 conn(0x7f8c3c075670 msgr2=0x7f8c3c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 -- 172.21.15.31:0/3158311873 shutdown_connections 2024-04-22T23:54:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.100+0000 7f8c410f1700 1 -- 172.21.15.31:0/3158311873 wait complete. 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.101+0000 7f8c410f1700 1 Processor -- start 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.101+0000 7f8c410f1700 1 -- start start 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c410f1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c3c068580 0x7f8c3c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c410f1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c410f1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c19d230 con 0x7f8c3c106cd0 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c410f1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1966e0 con 0x7f8c3c068580 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37432/0 (socket says 172.21.15.31:37432) 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 -- 172.21.15.31:0/2268782190 learned_addr learned my addr 172.21.15.31:0/2268782190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3ad9d700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c3c068580 0x7f8c3c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 -- 172.21.15.31:0/2268782190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c3c068580 msgr2=0x7f8c3c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c3c068580 0x7f8c3c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 -- 172.21.15.31:0/2268782190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0095f0 con 0x7f8c3c106cd0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.102+0000 7f8c3a59c700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c19cba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000d8d0 tx=0x7f8c3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.103+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c30009940 con 0x7f8c3c106cd0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.103+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c30010460 con 0x7f8c3c106cd0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.103+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3000f5f0 con 0x7f8c3c106cd0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.103+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c1969c0 con 0x7f8c3c106cd0 2024-04-22T23:54:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.103+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c196f10 con 0x7f8c3c106cd0 2024-04-22T23:54:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.104+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c1091d0 con 0x7f8c3c106cd0 2024-04-22T23:54:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.104+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f8c300105d0 con 0x7f8c3c106cd0 2024-04-22T23:54:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.105+0000 7f8c23fff700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f8c2406ecc0 0x7f8c24071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.105+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f8c3008ef40 con 0x7f8c3c106cd0 2024-04-22T23:54:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.105+0000 7f8c3ad9d700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f8c2406ecc0 0x7f8c24071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.105+0000 7f8c3ad9d700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f8c2406ecc0 0x7f8c24071180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006010 tx=0x7f8c2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.114+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f8c3005b760 con 0x7f8c3c106cd0 2024-04-22T23:54:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.358+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c3c197130 con 0x7f8c2406ecc0 2024-04-22T23:54:11.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:11 smithi031 ceph-mon[111289]: pgmap v183: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 166 B/s rd, 0 op/s 2024-04-22T23:54:11.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:11 smithi031 ceph-mon[111289]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:11.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:11 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3177945037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:11.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:11 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1130008223' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:54:11.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.360+0000 7f8c23fff700 1 -- 172.21.15.31:0/2268782190 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8c3c197130 con 0x7f8c2406ecc0 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-22T23:54:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "osd" 2024-04-22T23:54:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:54:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "12/23 daemons upgraded", 2024-04-22T23:54:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-04-22T23:54:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:54:11.352 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f8c2406ecc0 msgr2=0x7f8c24071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f8c2406ecc0 0x7f8c24071180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006010 tx=0x7f8c2c01a040 comp rx=0 tx=0).stop 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 msgr2=0x7f8c3c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c19cba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000d8d0 tx=0x7f8c3000dc90 comp rx=0 tx=0).stop 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 shutdown_connections 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f8c2406ecc0 0x7f8c24071180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c3c068580 0x7f8c3c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 --2- 172.21.15.31:0/2268782190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c3c106cd0 0x7f8c3c19cba0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 >> 172.21.15.31:0/2268782190 conn(0x7f8c3c075670 msgr2=0x7f8c3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 shutdown_connections 2024-04-22T23:54:11.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.362+0000 7f8c410f1700 1 -- 172.21.15.31:0/2268782190 wait complete. 2024-04-22T23:54:11.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:11 smithi044 ceph-mon[82146]: pgmap v183: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 166 B/s rd, 0 op/s 2024-04-22T23:54:11.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:11 smithi044 ceph-mon[82146]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:11.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:11 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3177945037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:11.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:11 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1130008223' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:54:11.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.471+0000 7f798a56c700 1 -- 172.21.15.31:0/83807357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 msgr2=0x7f7984101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.471+0000 7f798a56c700 1 --2- 172.21.15.31:0/83807357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984101260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7974009910 tx=0x7f7974009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 -- 172.21.15.31:0/83807357 shutdown_connections 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 --2- 172.21.15.31:0/83807357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 0x7f7984105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 --2- 172.21.15.31:0/83807357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984101260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 -- 172.21.15.31:0/83807357 >> 172.21.15.31:0/83807357 conn(0x7f79840fc710 msgr2=0x7f79840feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 -- 172.21.15.31:0/83807357 shutdown_connections 2024-04-22T23:54:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 -- 172.21.15.31:0/83807357 wait complete. 2024-04-22T23:54:11.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 Processor -- start 2024-04-22T23:54:11.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.472+0000 7f798a56c700 1 -- start start 2024-04-22T23:54:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f798a56c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f798a56c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 0x7f79840735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f798a56c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7984073b10 con 0x7f7984100e80 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f798a56c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7984073c50 con 0x7f7984101830 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37462/0 (socket says 172.21.15.31:37462) 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 -- 172.21.15.31:0/2550891192 learned_addr learned my addr 172.21.15.31:0/2550891192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 -- 172.21.15.31:0/2550891192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 msgr2=0x7f79840735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f79837fe700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 0x7f79840735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 0x7f79840735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 -- 172.21.15.31:0/2550891192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79740095f0 con 0x7f7984100e80 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f79837fe700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 0x7f79840735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f7983fff700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984073090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7974006010 tx=0x7f797400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.473+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f797401d070 con 0x7f7984100e80 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.474+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f797400f460 con 0x7f7984100e80 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.474+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7984073ed0 con 0x7f7984100e80 2024-04-22T23:54:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.474+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7974017630 con 0x7f7984100e80 2024-04-22T23:54:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.474+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79841a2d80 con 0x7f7984100e80 2024-04-22T23:54:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.475+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7984109220 con 0x7f7984100e80 2024-04-22T23:54:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.475+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f797400f5d0 con 0x7f7984100e80 2024-04-22T23:54:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.476+0000 7f79817fa700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f796c06ecc0 0x7f796c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.476+0000 7f79837fe700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f796c06ecc0 0x7f796c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:11.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.476+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f7974090320 con 0x7f7984100e80 2024-04-22T23:54:11.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.476+0000 7f79837fe700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f796c06ecc0 0x7f796c071180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7978005d90 tx=0x7f7978005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.481+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7974059090 con 0x7f7984100e80 2024-04-22T23:54:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.796+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7984108c00 con 0x7f7984100e80 2024-04-22T23:54:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.796+0000 7f79817fa700 1 -- 172.21.15.31:0/2550891192 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f797405c750 con 0x7f7984100e80 2024-04-22T23:54:11.788 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:54:11.788 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:54:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:54:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f796c06ecc0 msgr2=0x7f796c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f796c06ecc0 0x7f796c071180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7978005d90 tx=0x7f7978005d00 comp rx=0 tx=0).stop 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 msgr2=0x7f7984073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984073090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7974006010 tx=0x7f797400be20 comp rx=0 tx=0).stop 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 shutdown_connections 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f796c06ecc0 0x7f796c071180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7984100e80 0x7f7984073090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 --2- 172.21.15.31:0/2550891192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7984101830 0x7f79840735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.798+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 >> 172.21.15.31:0/2550891192 conn(0x7f79840fc710 msgr2=0x7f79840fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.799+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 shutdown_connections 2024-04-22T23:54:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:11.799+0000 7f798a56c700 1 -- 172.21.15.31:0/2550891192 wait complete. 2024-04-22T23:54:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:12 smithi031 ceph-mon[111289]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:12 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2550891192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:54:12.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:12 smithi044 ceph-mon[82146]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:12.460 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:12 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2550891192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:54:14.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: pgmap v184: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s 2024-04-22T23:54:14.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:boot 2024-04-22T23:54:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:54:14.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:13 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:54:14.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: pgmap v184: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s 2024-04-22T23:54:14.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:14.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:14.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:14.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:boot 2024-04-22T23:54:14.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:54:14.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:13 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:54:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:15 smithi031 ceph-mon[111289]: pgmap v185: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-22T23:54:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:15 smithi044 ceph-mon[82146]: pgmap v185: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-22T23:54:16.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:16.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:16.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:18.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:17 smithi031 ceph-mon[111289]: pgmap v186: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-22T23:54:18.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:17 smithi044 ceph-mon[82146]: pgmap v186: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-22T23:54:19.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:19 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:19.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:19 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:19.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:19 smithi031 ceph-mon[111289]: pgmap v187: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-22T23:54:19.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:19 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:19.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:19 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:19.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:19 smithi044 ceph-mon[82146]: pgmap v187: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-04-22T23:54:20.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:20 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:20.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:20 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:20.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:20 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:20.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:20 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:21.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:21 smithi031 ceph-mon[111289]: pgmap v188: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:21.666 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:21 smithi044 ceph-mon[82146]: pgmap v188: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: pgmap v189: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:23.762 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.763 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.763 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.763 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:23 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.830 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: pgmap v189: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:23.830 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:23.830 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:23.830 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:23.830 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:23.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:23.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:23.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:23.831 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:23 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:25.914 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:25 smithi031 ceph-mon[111289]: pgmap v190: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:25.914 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:25 smithi031 ceph-mon[111289]: Upgrade: Updating mds.cephfs.smithi031.usiyoa 2024-04-22T23:54:25.914 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:25.914 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:54:25.914 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:25 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:25.914 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:25 smithi031 ceph-mon[111289]: Deploying daemon mds.cephfs.smithi031.usiyoa on smithi031 2024-04-22T23:54:25.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:25 smithi044 ceph-mon[82146]: pgmap v190: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:25.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:25 smithi044 ceph-mon[82146]: Upgrade: Updating mds.cephfs.smithi031.usiyoa 2024-04-22T23:54:25.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:25.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.usiyoa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:54:25.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:25 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:25.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:25 smithi044 ceph-mon[82146]: Deploying daemon mds.cephfs.smithi031.usiyoa on smithi031 2024-04-22T23:54:27.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:27 smithi031 ceph-mon[111289]: pgmap v191: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:27.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:27 smithi044 ceph-mon[82146]: pgmap v191: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:28 smithi031 ceph-mon[111289]: osdmap e90: 6 total, 6 up, 6 in 2024-04-22T23:54:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:28 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 2 up:standby 2024-04-22T23:54:28.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:28 smithi044 ceph-mon[82146]: osdmap e90: 6 total, 6 up, 6 in 2024-04-22T23:54:28.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:28 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 2 up:standby 2024-04-22T23:54:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:29 smithi031 ceph-mon[111289]: pgmap v193: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:29.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:29 smithi044 ceph-mon[82146]: pgmap v193: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:30.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:30.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:31.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: pgmap v194: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:31.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.31:6828/1426404610,v1:172.21.15.31:6829/1426404610] up:boot 2024-04-22T23:54:31.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:54:31.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:54:31.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:31.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:31.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:31.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: pgmap v194: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:31.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.31:6828/1426404610,v1:172.21.15.31:6829/1426404610] up:boot 2024-04-22T23:54:31.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.ibkztg=up:active} 3 up:standby 2024-04-22T23:54:31.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:54:31.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:31.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:31.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:33.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:33 smithi031 ceph-mon[111289]: pgmap v195: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:33.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:33 smithi044 ceph-mon[82146]: pgmap v195: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:35.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:35 smithi031 ceph-mon[111289]: pgmap v196: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:35 smithi044 ceph-mon[82146]: pgmap v196: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:37.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:36 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:37.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:36 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:37.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:36 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:37.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:36 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:38.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:37 smithi031 ceph-mon[111289]: pgmap v197: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:38.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:38.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:37 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:38.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:37 smithi044 ceph-mon[82146]: pgmap v197: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:38.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:37 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:38.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:37 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:40.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:39 smithi031 ceph-mon[111289]: pgmap v198: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:40.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:39 smithi044 ceph-mon[82146]: pgmap v198: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: pgmap v199: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.820 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.821 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.821 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: pgmap v199: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.912 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.913 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.913 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 -- 172.21.15.31:0/704789785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 msgr2=0x7f445410d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 --2- 172.21.15.31:0/704789785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f445410d290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4440009910 tx=0x7f4440009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 -- 172.21.15.31:0/704789785 shutdown_connections 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 --2- 172.21.15.31:0/704789785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f445410d290 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 --2- 172.21.15.31:0/704789785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 0x7f4454068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 -- 172.21.15.31:0/704789785 >> 172.21.15.31:0/704789785 conn(0x7f4454075670 msgr2=0x7f4454075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 -- 172.21.15.31:0/704789785 shutdown_connections 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.919+0000 7f445ab24700 1 -- 172.21.15.31:0/704789785 wait complete. 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f445ab24700 1 Processor -- start 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f445ab24700 1 -- start start 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f445ab24700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 0x7f445410cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f445ab24700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f4454103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f445ab24700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445410d5f0 con 0x7f4454068ea0 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f445ab24700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454104380 con 0x7f4454068580 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f4453fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f4454103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f44588c0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 0x7f445410cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:41.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f4453fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f4454103e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59608/0 (socket says 172.21.15.31:59608) 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f4453fff700 1 -- 172.21.15.31:0/564862620 learned_addr learned my addr 172.21.15.31:0/564862620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f4453fff700 1 -- 172.21.15.31:0/564862620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 msgr2=0x7f445410cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f4453fff700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 0x7f445410cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f4453fff700 1 -- 172.21.15.31:0/564862620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400095f0 con 0x7f4454068ea0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.920+0000 7f44588c0700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 0x7f445410cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.921+0000 7f4453fff700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f4454103e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f444000f690 tx=0x7f444000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.921+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f444001d070 con 0x7f4454068ea0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.921+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44400050d0 con 0x7f4454068ea0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.921+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4454104600 con 0x7f4454068ea0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.921+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4440022470 con 0x7f4454068ea0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.921+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4454104af0 con 0x7f4454068ea0 2024-04-22T23:54:41.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.922+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445404f2e0 con 0x7f4454068ea0 2024-04-22T23:54:41.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.923+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4440003780 con 0x7f4454068ea0 2024-04-22T23:54:41.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.923+0000 7f4451ffb700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f444406ecc0 0x7f4444071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:41.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.923+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f44400903a0 con 0x7f4454068ea0 2024-04-22T23:54:41.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.924+0000 7f44588c0700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f444406ecc0 0x7f4444071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:41.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.924+0000 7f44588c0700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f444406ecc0 0x7f4444071180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4448011490 tx=0x7f4448009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:41.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:41.929+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f4440059140 con 0x7f4454068ea0 2024-04-22T23:54:42.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.160+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44541089b0 con 0x7f444406ecc0 2024-04-22T23:54:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.162+0000 7f4451ffb700 1 -- 172.21.15.31:0/564862620 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f44541089b0 con 0x7f444406ecc0 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f444406ecc0 msgr2=0x7f4444071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f444406ecc0 0x7f4444071180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4448011490 tx=0x7f4448009260 comp rx=0 tx=0).stop 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 msgr2=0x7f4454103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f4454103e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f444000f690 tx=0x7f444000f6c0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 shutdown_connections 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f444406ecc0 0x7f4444071180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4454068580 0x7f445410cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 --2- 172.21.15.31:0/564862620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4454068ea0 0x7f4454103e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 >> 172.21.15.31:0/564862620 conn(0x7f4454075670 msgr2=0x7f44540fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 shutdown_connections 2024-04-22T23:54:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.164+0000 7f445ab24700 1 -- 172.21.15.31:0/564862620 wait complete. 2024-04-22T23:54:42.166 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.272+0000 7f24611ae700 1 -- 172.21.15.31:0/2793814029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 msgr2=0x7f245c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.272+0000 7f24611ae700 1 --2- 172.21.15.31:0/2793814029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c101280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f244c009910 tx=0x7f244c009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.272+0000 7f24611ae700 1 -- 172.21.15.31:0/2793814029 shutdown_connections 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.272+0000 7f24611ae700 1 --2- 172.21.15.31:0/2793814029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 0x7f245c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.272+0000 7f24611ae700 1 --2- 172.21.15.31:0/2793814029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c101280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.272+0000 7f24611ae700 1 -- 172.21.15.31:0/2793814029 >> 172.21.15.31:0/2793814029 conn(0x7f245c0fc750 msgr2=0x7f245c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.273+0000 7f24611ae700 1 -- 172.21.15.31:0/2793814029 shutdown_connections 2024-04-22T23:54:42.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.273+0000 7f24611ae700 1 -- 172.21.15.31:0/2793814029 wait complete. 2024-04-22T23:54:42.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.273+0000 7f24611ae700 1 Processor -- start 2024-04-22T23:54:42.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f24611ae700 1 -- start start 2024-04-22T23:54:42.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f24611ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f24611ae700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 0x7f245c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f24611ae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19d230 con 0x7f245c100ea0 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f24611ae700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c196770 con 0x7f245c101850 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59624/0 (socket says 172.21.15.31:59624) 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245ad9d700 1 -- 172.21.15.31:0/2151600313 learned_addr learned my addr 172.21.15.31:0/2151600313 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245a59c700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 0x7f245c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245ad9d700 1 -- 172.21.15.31:0/2151600313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 msgr2=0x7f245c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245ad9d700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 0x7f245c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245ad9d700 1 -- 172.21.15.31:0/2151600313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0095f0 con 0x7f245c100ea0 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.274+0000 7f245a59c700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 0x7f245c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.275+0000 7f245ad9d700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c19c6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f244c006010 tx=0x7f244c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.275+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c01d070 con 0x7f245c100ea0 2024-04-22T23:54:42.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.275+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f244c004fb0 con 0x7f245c100ea0 2024-04-22T23:54:42.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.275+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c1969f0 con 0x7f245c100ea0 2024-04-22T23:54:42.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.275+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c022ea0 con 0x7f245c100ea0 2024-04-22T23:54:42.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.275+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c196ee0 con 0x7f245c100ea0 2024-04-22T23:54:42.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.276+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c109240 con 0x7f245c100ea0 2024-04-22T23:54:42.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.276+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f244c022470 con 0x7f245c100ea0 2024-04-22T23:54:42.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.277+0000 7f2443fff700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f244406ecc0 0x7f2444071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.277+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f244c0904c0 con 0x7f245c100ea0 2024-04-22T23:54:42.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.277+0000 7f245a59c700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f244406ecc0 0x7f2444071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.277+0000 7f245a59c700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f244406ecc0 0x7f2444071180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f2450005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:42.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.282+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f244c05cd90 con 0x7f245c100ea0 2024-04-22T23:54:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.512+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f245c197d60 con 0x7f244406ecc0 2024-04-22T23:54:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.513+0000 7f2443fff700 1 -- 172.21.15.31:0/2151600313 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f245c197d60 con 0x7f244406ecc0 2024-04-22T23:54:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f244406ecc0 msgr2=0x7f2444071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f244406ecc0 0x7f2444071180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2450005fd0 tx=0x7f2450005e30 comp rx=0 tx=0).stop 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 msgr2=0x7f245c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c19c6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f244c006010 tx=0x7f244c00be20 comp rx=0 tx=0).stop 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 shutdown_connections 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f244406ecc0 0x7f2444071180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.515+0000 7f24611ae700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c100ea0 0x7f245c19c6f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.516+0000 7f24611ae700 1 --2- 172.21.15.31:0/2151600313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f245c101850 0x7f245c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.516+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 >> 172.21.15.31:0/2151600313 conn(0x7f245c0fc750 msgr2=0x7f245c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.516+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 shutdown_connections 2024-04-22T23:54:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.516+0000 7f24611ae700 1 -- 172.21.15.31:0/2151600313 wait complete. 2024-04-22T23:54:42.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 -- 172.21.15.31:0/727625529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c100750 msgr2=0x7f2c1c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 --2- 172.21.15.31:0/727625529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c100750 0x7f2c1c1047a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009910 tx=0x7f2c0c009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 -- 172.21.15.31:0/727625529 shutdown_connections 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 --2- 172.21.15.31:0/727625529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c100750 0x7f2c1c1047a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 --2- 172.21.15.31:0/727625529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 -- 172.21.15.31:0/727625529 >> 172.21.15.31:0/727625529 conn(0x7f2c1c0fb650 msgr2=0x7f2c1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.624+0000 7f2c2474e700 1 -- 172.21.15.31:0/727625529 shutdown_connections 2024-04-22T23:54:42.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 -- 172.21.15.31:0/727625529 wait complete. 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 Processor -- start 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 -- start start 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c100750 0x7f2c1c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c19d240 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.625+0000 7f2c2474e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c1966f0 con 0x7f2c1c100750 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59642/0 (socket says 172.21.15.31:59642) 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 -- 172.21.15.31:0/3648364237 learned_addr learned my addr 172.21.15.31:0/3648364237 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c21ce9700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c100750 0x7f2c1c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 -- 172.21.15.31:0/3648364237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c100750 msgr2=0x7f2c1c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c100750 0x7f2c1c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 -- 172.21.15.31:0/3648364237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0095f0 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c224ea700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c19c670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800e920 tx=0x7f2c1800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c1800c9d0 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c18010430 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c1969d0 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c18018630 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.626+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1c196ef0 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.627+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1c1091b0 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.628+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2c180105a0 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.628+0000 7f2c137fe700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2c0806ecc0 0x7f2c08071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.628+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f2c18014070 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.628+0000 7f2c21ce9700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2c0806ecc0 0x7f2c08071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.629+0000 7f2c21ce9700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2c0806ecc0 0x7f2c08071180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0059d0 tx=0x7f2c0c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:42.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.633+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2c18054150 con 0x7f2c1c0ffda0 2024-04-22T23:54:42.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.851+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c1c197160 con 0x7f2c0806ecc0 2024-04-22T23:54:42.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.864+0000 7f2c137fe700 1 -- 172.21.15.31:0/3648364237 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2c1c197160 con 0x7f2c0806ecc0 2024-04-22T23:54:42.855 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:54:42.855 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 7s ago 17m 24.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:54:42.855 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (17m) 7s ago 17m 8980k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (11m) 44s ago 16m 8572k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (4m) 7s ago 17m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (4m) 44s ago 16m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 7s ago 17m 81.3M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (30s) 7s ago 12m 13.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a7e757804d66 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (12s) 7s ago 12m 14.8M - 19.0.0-3252-g43be0201 a0434699c1d7 961938a7288c 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (12m) 44s ago 12m 147M - 18.2.1 5be31c24972a 6d02e3ba1a7c 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (12m) 44s ago 12m 21.3M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (9m) 7s ago 19m 512M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (8m) 44s ago 16m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (5m) 7s ago 19m 55.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (4m) 44s ago 16m 41.9M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 7s ago 17m 15.3M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:54:42.856 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (7m) 44s ago 16m 15.6M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 7s ago 15m 101M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (3m) 7s ago 15m 128M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 7s ago 14m 85.9M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (115s) 44s ago 14m 82.3M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (82s) 44s ago 14m 80.5M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (48s) 44s ago 13m 12.6M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 38dba54af5f6 2024-04-22T23:54:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (7m) 7s ago 17m 76.7M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2c0806ecc0 msgr2=0x7f2c08071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2c0806ecc0 0x7f2c08071180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0059d0 tx=0x7f2c0c005940 comp rx=0 tx=0).stop 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 msgr2=0x7f2c1c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c19c670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800e920 tx=0x7f2c1800ec30 comp rx=0 tx=0).stop 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 shutdown_connections 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2c0806ecc0 0x7f2c08071180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c1c0ffda0 0x7f2c1c19c670 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 --2- 172.21.15.31:0/3648364237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c1c100750 0x7f2c1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 >> 172.21.15.31:0/3648364237 conn(0x7f2c1c0fb650 msgr2=0x7f2c1c069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 shutdown_connections 2024-04-22T23:54:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.867+0000 7f2c2474e700 1 -- 172.21.15.31:0/3648364237 wait complete. 2024-04-22T23:54:42.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.973+0000 7fdf934f0700 1 -- 172.21.15.31:0/2135758021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 msgr2=0x7fdf8c0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.973+0000 7fdf934f0700 1 --2- 172.21.15.31:0/2135758021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c0ff9e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009910 tx=0x7fdf88009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.973+0000 7fdf934f0700 1 -- 172.21.15.31:0/2135758021 shutdown_connections 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.973+0000 7fdf934f0700 1 --2- 172.21.15.31:0/2135758021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8c0fffb0 0x7fdf8c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.973+0000 7fdf934f0700 1 --2- 172.21.15.31:0/2135758021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c0ff9e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.973+0000 7fdf934f0700 1 -- 172.21.15.31:0/2135758021 >> 172.21.15.31:0/2135758021 conn(0x7fdf8c074df0 msgr2=0x7fdf8c075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 -- 172.21.15.31:0/2135758021 shutdown_connections 2024-04-22T23:54:42.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 -- 172.21.15.31:0/2135758021 wait complete. 2024-04-22T23:54:42.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 Processor -- start 2024-04-22T23:54:42.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 -- start start 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8c0fffb0 0x7fdf8c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c19d2c0 con 0x7fdf8c0ff600 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.974+0000 7fdf934f0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c196770 con 0x7fdf8c0fffb0 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59666/0 (socket says 172.21.15.31:59666) 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 -- 172.21.15.31:0/594150742 learned_addr learned my addr 172.21.15.31:0/594150742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf90a8b700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8c0fffb0 0x7fdf8c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 -- 172.21.15.31:0/594150742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8c0fffb0 msgr2=0x7fdf8c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8c0fffb0 0x7fdf8c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 -- 172.21.15.31:0/594150742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880095f0 con 0x7fdf8c0ff600 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf9128c700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c19c6f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800f690 tx=0x7fdf8800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf8801d070 con 0x7fdf8c0ff600 2024-04-22T23:54:42.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c1969f0 con 0x7fdf8c0ff600 2024-04-22T23:54:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf880050d0 con 0x7fdf8c0ff600 2024-04-22T23:54:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf88022470 con 0x7fdf8c0ff600 2024-04-22T23:54:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.975+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c196ee0 con 0x7fdf8c0ff600 2024-04-22T23:54:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.977+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c109270 con 0x7fdf8c0ff600 2024-04-22T23:54:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.977+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fdf88003780 con 0x7fdf8c0ff600 2024-04-22T23:54:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.978+0000 7fdf827fc700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdf7806ecc0 0x7fdf78071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.978+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fdf880903a0 con 0x7fdf8c0ff600 2024-04-22T23:54:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.978+0000 7fdf90a8b700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdf7806ecc0 0x7fdf78071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:42.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.978+0000 7fdf90a8b700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdf7806ecc0 0x7fdf78071180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c197be0 tx=0x7fdf7c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:42.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:42.982+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fdf8805cc70 con 0x7fdf8c0ff600 2024-04-22T23:54:43.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.282+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf8c04f2e0 con 0x7fdf8c0ff600 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.283+0000 7fdf827fc700 1 -- 172.21.15.31:0/594150742 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fdf8805c800 con 0x7fdf8c0ff600 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:54:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 12 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:54:43.275 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdf7806ecc0 msgr2=0x7fdf78071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdf7806ecc0 0x7fdf78071180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c197be0 tx=0x7fdf7c009260 comp rx=0 tx=0).stop 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 msgr2=0x7fdf8c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c19c6f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800f690 tx=0x7fdf8800f6c0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 shutdown_connections 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fdf7806ecc0 0x7fdf78071180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf8c0ff600 0x7fdf8c19c6f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 --2- 172.21.15.31:0/594150742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf8c0fffb0 0x7fdf8c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 >> 172.21.15.31:0/594150742 conn(0x7fdf8c074df0 msgr2=0x7fdf8c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 shutdown_connections 2024-04-22T23:54:43.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.285+0000 7fdf934f0700 1 -- 172.21.15.31:0/594150742 wait complete. 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 -- 172.21.15.31:0/1763533735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0100ea0 msgr2=0x7fb1c0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/1763533735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0100ea0 0x7fb1c0101280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000b0a0 tx=0x7fb1b000b3b0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 -- 172.21.15.31:0/1763533735 shutdown_connections 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/1763533735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1c0101850 0x7fb1c01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/1763533735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0100ea0 0x7fb1c0101280 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 -- 172.21.15.31:0/1763533735 >> 172.21.15.31:0/1763533735 conn(0x7fb1c00fc750 msgr2=0x7fb1c00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 -- 172.21.15.31:0/1763533735 shutdown_connections 2024-04-22T23:54:43.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.392+0000 7fb1c83a7700 1 -- 172.21.15.31:0/1763533735 wait complete. 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c83a7700 1 Processor -- start 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c83a7700 1 -- start start 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c83a7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1c0100ea0 0x7fb1c0196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c83a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 0x7fb1c0196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c83a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01974a0 con 0x7fb1c0101850 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c83a7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0197610 con 0x7fb1c0100ea0 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c5942700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 0x7fb1c0196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c6143700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1c0100ea0 0x7fb1c0196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c5942700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 0x7fb1c0196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59692/0 (socket says 172.21.15.31:59692) 2024-04-22T23:54:43.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c5942700 1 -- 172.21.15.31:0/983570784 learned_addr learned my addr 172.21.15.31:0/983570784 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c5942700 1 -- 172.21.15.31:0/983570784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1c0100ea0 msgr2=0x7fb1c0196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c5942700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1c0100ea0 0x7fb1c0196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.393+0000 7fb1c5942700 1 -- 172.21.15.31:0/983570784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0009d00 con 0x7fb1c0101850 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.394+0000 7fb1c5942700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 0x7fb1c0196ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c960 tx=0x7fb1b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.394+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b4007a70 con 0x7fb1c0101850 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.394+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c019b490 con 0x7fb1c0101850 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.394+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b4007bd0 con 0x7fb1c0101850 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.394+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b4018630 con 0x7fb1c0101850 2024-04-22T23:54:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.394+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c019b9e0 con 0x7fb1c0101850 2024-04-22T23:54:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.395+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c0109240 con 0x7fb1c0101850 2024-04-22T23:54:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.395+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb1b4018790 con 0x7fb1c0101850 2024-04-22T23:54:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.396+0000 7fb1bf7fe700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb1a806ecc0 0x7fb1a8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.396+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fb1b408eea0 con 0x7fb1c0101850 2024-04-22T23:54:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.396+0000 7fb1c6143700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb1a806ecc0 0x7fb1a8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.396+0000 7fb1c6143700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb1a806ecc0 0x7fb1a8071180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0000f80 tx=0x7fb1b00097e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:43.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.401+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb1b405b730 con 0x7fb1c0101850 2024-04-22T23:54:43.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.658+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb1c004f2e0 con 0x7fb1c0101850 2024-04-22T23:54:43.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.659+0000 7fb1bf7fe700 1 -- 172.21.15.31:0/983570784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1887 (secure 0 0 0) 0x7fb1b405b2c0 con 0x7fb1c0101850 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:e24 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 19 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:54:43.650 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:54:04.995710+0000 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24293} 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:54:43.651 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24293 members: 24293 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{0:24293} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.44:6824/1404295013,v1:172.21.15.44:6825/1404295013] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:24309} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{-1:34406} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:54:43.652 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/1426404610,v1:172.21.15.31:6829/1426404610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb1a806ecc0 msgr2=0x7fb1a8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb1a806ecc0 0x7fb1a8071180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0000f80 tx=0x7fb1b00097e0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 msgr2=0x7fb1c0196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 0x7fb1c0196ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c960 tx=0x7fb1b400cc70 comp rx=0 tx=0).stop 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 shutdown_connections 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb1a806ecc0 0x7fb1a8071180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1c0100ea0 0x7fb1c0196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 --2- 172.21.15.31:0/983570784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1c0101850 0x7fb1c0196ea0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 >> 172.21.15.31:0/983570784 conn(0x7fb1c00fc750 msgr2=0x7fb1c00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 shutdown_connections 2024-04-22T23:54:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.661+0000 7fb1c83a7700 1 -- 172.21.15.31:0/983570784 wait complete. 2024-04-22T23:54:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2024-04-22T23:54:43.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: pgmap v200: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: Upgrade: Updating mds.cephfs.smithi044.ibkztg 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: Deploying daemon mds.cephfs.smithi044.ibkztg on smithi044 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:43.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:43 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/594150742' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 -- 172.21.15.31:0/1590432933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58101850 msgr2=0x7fca581058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 --2- 172.21.15.31:0/1590432933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58101850 0x7fca581058a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009910 tx=0x7fca4c009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 -- 172.21.15.31:0/1590432933 shutdown_connections 2024-04-22T23:54:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 --2- 172.21.15.31:0/1590432933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58101850 0x7fca581058a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 --2- 172.21.15.31:0/1590432933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca58100ea0 0x7fca58101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 -- 172.21.15.31:0/1590432933 >> 172.21.15.31:0/1590432933 conn(0x7fca580fc750 msgr2=0x7fca580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 -- 172.21.15.31:0/1590432933 shutdown_connections 2024-04-22T23:54:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.769+0000 7fca5d860700 1 -- 172.21.15.31:0/1590432933 wait complete. 2024-04-22T23:54:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca5d860700 1 Processor -- start 2024-04-22T23:54:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca5d860700 1 -- start start 2024-04-22T23:54:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca5d860700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 0x7fca5819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca5d860700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca58101850 0x7fca5819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca5d860700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5819d210 con 0x7fca58100ea0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca5d860700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581966c0 con 0x7fca58101850 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca56ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 0x7fca5819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca56ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 0x7fca5819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59720/0 (socket says 172.21.15.31:59720) 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca56ffd700 1 -- 172.21.15.31:0/505570825 learned_addr learned my addr 172.21.15.31:0/505570825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca567fc700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca58101850 0x7fca5819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca56ffd700 1 -- 172.21.15.31:0/505570825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca58101850 msgr2=0x7fca5819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca56ffd700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca58101850 0x7fca5819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.770+0000 7fca56ffd700 1 -- 172.21.15.31:0/505570825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0095f0 con 0x7fca58100ea0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.771+0000 7fca56ffd700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 0x7fca5819c640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca4800d8d0 tx=0x7fca4800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.771+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca480098a0 con 0x7fca58100ea0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.771+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca48010460 con 0x7fca58100ea0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.771+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca581969a0 con 0x7fca58100ea0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.771+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4800f5f0 con 0x7fca58100ea0 2024-04-22T23:54:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.771+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca58196ec0 con 0x7fca58100ea0 2024-04-22T23:54:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.772+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca581092a0 con 0x7fca58100ea0 2024-04-22T23:54:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.772+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fca48009a00 con 0x7fca58100ea0 2024-04-22T23:54:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.773+0000 7fca5c85e700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fca4006ecc0 0x7fca40071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.773+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7fca4808edf0 con 0x7fca58100ea0 2024-04-22T23:54:43.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.773+0000 7fca567fc700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fca4006ecc0 0x7fca40071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:43.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.774+0000 7fca567fc700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fca4006ecc0 0x7fca40071180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006010 tx=0x7fca4c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:43.778+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fca4805b610 con 0x7fca58100ea0 2024-04-22T23:54:43.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: pgmap v200: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:43.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:43.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:43.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: Upgrade: Updating mds.cephfs.smithi044.ibkztg 2024-04-22T23:54:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.ibkztg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:54:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:54:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: Deploying daemon mds.cephfs.smithi044.ibkztg on smithi044 2024-04-22T23:54:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:43.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:43 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/594150742' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:54:43.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.006+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca581970e0 con 0x7fca4006ecc0 2024-04-22T23:54:43.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.008+0000 7fca5c85e700 1 -- 172.21.15.31:0/505570825 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fca581970e0 con 0x7fca4006ecc0 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "osd" 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "14/23 daemons upgraded", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:54:43.999 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fca4006ecc0 msgr2=0x7fca40071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fca4006ecc0 0x7fca40071180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006010 tx=0x7fca4c01a040 comp rx=0 tx=0).stop 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 msgr2=0x7fca5819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 0x7fca5819c640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca4800d8d0 tx=0x7fca4800dbe0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 shutdown_connections 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fca4006ecc0 0x7fca40071180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca58100ea0 0x7fca5819c640 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 --2- 172.21.15.31:0/505570825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca58101850 0x7fca5819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 >> 172.21.15.31:0/505570825 conn(0x7fca580fc750 msgr2=0x7fca580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 shutdown_connections 2024-04-22T23:54:44.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.010+0000 7fca5d860700 1 -- 172.21.15.31:0/505570825 wait complete. 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 -- 172.21.15.31:0/664020594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 msgr2=0x7f2a7c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 --2- 172.21.15.31:0/664020594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c1058a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009910 tx=0x7f2a70009c20 comp rx=0 tx=0).stop 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 -- 172.21.15.31:0/664020594 shutdown_connections 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 --2- 172.21.15.31:0/664020594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c1058a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 --2- 172.21.15.31:0/664020594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c100ea0 0x7f2a7c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 -- 172.21.15.31:0/664020594 >> 172.21.15.31:0/664020594 conn(0x7f2a7c0fc750 msgr2=0x7f2a7c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 -- 172.21.15.31:0/664020594 shutdown_connections 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.117+0000 7f2a80aef700 1 -- 172.21.15.31:0/664020594 wait complete. 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a80aef700 1 Processor -- start 2024-04-22T23:54:44.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a80aef700 1 -- start start 2024-04-22T23:54:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a80aef700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c100ea0 0x7f2a7c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a80aef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a80aef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c19f4a0 con 0x7f2a7c101850 2024-04-22T23:54:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a80aef700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c198950 con 0x7f2a7c100ea0 2024-04-22T23:54:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a79d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:44.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a79d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59738/0 (socket says 172.21.15.31:59738) 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a79d9b700 1 -- 172.21.15.31:0/825993417 learned_addr learned my addr 172.21.15.31:0/825993417 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.118+0000 7f2a7a59c700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c100ea0 0x7f2a7c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a79d9b700 1 -- 172.21.15.31:0/825993417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c100ea0 msgr2=0x7f2a7c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a79d9b700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c100ea0 0x7f2a7c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a79d9b700 1 -- 172.21.15.31:0/825993417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700095f0 con 0x7f2a7c101850 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a79d9b700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c19ee10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a700092e0 tx=0x7f2a7000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7001d070 con 0x7f2a7c101850 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a70005dc0 con 0x7f2a7c101850 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c198bd0 con 0x7f2a7c101850 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a70017400 con 0x7f2a7c101850 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.119+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c1990c0 con 0x7f2a7c101850 2024-04-22T23:54:44.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.120+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c105020 con 0x7f2a7c101850 2024-04-22T23:54:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.121+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2a7000f6f0 con 0x7f2a7c101850 2024-04-22T23:54:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.121+0000 7f2a6b7fe700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2a6406ecc0 0x7f2a64071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:54:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.121+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6704+0+0 (secure 0 0 0) 0x7f2a70090230 con 0x7f2a7c101850 2024-04-22T23:54:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.121+0000 7f2a7a59c700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2a6406ecc0 0x7f2a64071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:54:44.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.122+0000 7f2a7a59c700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2a6406ecc0 0x7f2a64071180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005950 tx=0x7f2a6c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:54:44.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.126+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2a7005cad0 con 0x7f2a7c101850 2024-04-22T23:54:44.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.426+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2a7c04f2e0 con 0x7f2a7c101850 2024-04-22T23:54:44.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.427+0000 7f2a6b7fe700 1 -- 172.21.15.31:0/825993417 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2a7005c660 con 0x7f2a7c101850 2024-04-22T23:54:44.418 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:54:44.418 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:54:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:54:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2a6406ecc0 msgr2=0x7f2a64071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2a6406ecc0 0x7f2a64071180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005950 tx=0x7f2a6c00b410 comp rx=0 tx=0).stop 2024-04-22T23:54:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 msgr2=0x7f2a7c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:54:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c19ee10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a700092e0 tx=0x7f2a7000bd70 comp rx=0 tx=0).stop 2024-04-22T23:54:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 shutdown_connections 2024-04-22T23:54:44.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2a6406ecc0 0x7f2a64071180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c100ea0 0x7f2a7c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 --2- 172.21.15.31:0/825993417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a7c101850 0x7f2a7c19ee10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:54:44.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 >> 172.21.15.31:0/825993417 conn(0x7f2a7c0fc750 msgr2=0x7f2a7c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:54:44.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 shutdown_connections 2024-04-22T23:54:44.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:54:44.429+0000 7f2a80aef700 1 -- 172.21.15.31:0/825993417 wait complete. 2024-04-22T23:54:44.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:44 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/983570784' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:54:44.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:44 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/825993417' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:54:44.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:44 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[111285]: 2024-04-22T23:54:44.570+0000 7f7cd7098700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:54:44.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:44 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/983570784' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:54:44.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:44 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/825993417' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:54:45.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: pgmap v201: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: osdmap e91: 6 total, 6 up, 6 in 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: Standby daemon mds.cephfs.smithi044.onzoro assigned to filesystem cephfs as rank 0 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:replay} 2 up:standby 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:45.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:54:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: pgmap v201: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail 2024-04-22T23:54:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:54:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:54:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: osdmap e91: 6 total, 6 up, 6 in 2024-04-22T23:54:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: Standby daemon mds.cephfs.smithi044.onzoro assigned to filesystem cephfs as rank 0 2024-04-22T23:54:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:54:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:54:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:replay} 2 up:standby 2024-04-22T23:54:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:45.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:54:46.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:46 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:54:46.953 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:46 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:54:47.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:47 smithi031 ceph-mon[111289]: pgmap v203: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-04-22T23:54:47.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:47 smithi044 ceph-mon[82146]: pgmap v203: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-04-22T23:54:49.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:49 smithi031 ceph-mon[111289]: pgmap v204: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 1 op/s 2024-04-22T23:54:49.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:49 smithi044 ceph-mon[82146]: pgmap v204: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.8 MiB/s rd, 1 op/s 2024-04-22T23:54:51.661 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:51 smithi044 ceph-mon[82146]: pgmap v205: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-22T23:54:51.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:51 smithi031 ceph-mon[111289]: pgmap v205: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-22T23:54:53.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:53 smithi031 ceph-mon[111289]: pgmap v206: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-22T23:54:53.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:53 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:53.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:53 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:53.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:53 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:53.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:53 smithi044 ceph-mon[82146]: pgmap v206: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-22T23:54:53.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:53 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:53 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:54:53.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:53 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:54:54.745 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:54 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.44:6824/4021256007,v1:172.21.15.44:6825/4021256007] up:boot 2024-04-22T23:54:54.745 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:54 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:replay} 3 up:standby 2024-04-22T23:54:54.745 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:54 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:54:54.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:54 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.44:6824/4021256007,v1:172.21.15.44:6825/4021256007] up:boot 2024-04-22T23:54:54.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:54 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:replay} 3 up:standby 2024-04-22T23:54:54.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:54 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:54:55.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:55 smithi031 ceph-mon[111289]: pgmap v207: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-22T23:54:55.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:55 smithi031 ceph-mon[111289]: reconnect by client.24345 192.168.0.1:0/3446383703 after 0.000999984 2024-04-22T23:54:55.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:55 smithi031 ceph-mon[111289]: reconnect by client.14542 192.168.0.1:0/1415787653 after 0.00199997 2024-04-22T23:54:55.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:55 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:reconnect 2024-04-22T23:54:55.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:55 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:reconnect} 3 up:standby 2024-04-22T23:54:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:55 smithi044 ceph-mon[82146]: pgmap v207: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-22T23:54:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:55 smithi044 ceph-mon[82146]: reconnect by client.24345 192.168.0.1:0/3446383703 after 0.000999984 2024-04-22T23:54:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:55 smithi044 ceph-mon[82146]: reconnect by client.14542 192.168.0.1:0/1415787653 after 0.00199997 2024-04-22T23:54:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:55 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:reconnect 2024-04-22T23:54:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:55 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:reconnect} 3 up:standby 2024-04-22T23:54:56.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:56 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:rejoin 2024-04-22T23:54:56.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:56 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:rejoin} 3 up:standby 2024-04-22T23:54:56.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:56 smithi031 ceph-mon[111289]: daemon mds.cephfs.smithi044.onzoro is now active in filesystem cephfs as rank 0 2024-04-22T23:54:56.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:56 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:rejoin 2024-04-22T23:54:56.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:56 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi044.onzoro=up:rejoin} 3 up:standby 2024-04-22T23:54:56.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:56 smithi044 ceph-mon[82146]: daemon mds.cephfs.smithi044.onzoro is now active in filesystem cephfs as rank 0 2024-04-22T23:54:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:57 smithi031 ceph-mon[111289]: pgmap v208: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-22T23:54:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:57 smithi031 ceph-mon[111289]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:54:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:57 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:active 2024-04-22T23:54:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:57 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi044.onzoro=up:active} 3 up:standby 2024-04-22T23:54:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:57 smithi044 ceph-mon[82146]: pgmap v208: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-22T23:54:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:57 smithi044 ceph-mon[82146]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:54:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:57 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.44:6826/3065727368,v1:172.21.15.44:6827/3065727368] up:active 2024-04-22T23:54:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:57 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi044.onzoro=up:active} 3 up:standby 2024-04-22T23:54:59.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:54:59 smithi031 ceph-mon[111289]: pgmap v209: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-04-22T23:54:59.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:54:59 smithi044 ceph-mon[82146]: pgmap v209: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-04-22T23:55:01.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:01 smithi031 ceph-mon[111289]: pgmap v210: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T23:55:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:01.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:01.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:01.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:01.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:01 smithi044 ceph-mon[82146]: pgmap v210: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T23:55:01.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:01.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:03.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:03 smithi031 ceph-mon[111289]: pgmap v211: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T23:55:03.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:03.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:03 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:03.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:03 smithi044 ceph-mon[82146]: pgmap v211: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T23:55:03.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:03.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:03 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:05.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:05 smithi031 ceph-mon[111289]: pgmap v212: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T23:55:05.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:05 smithi044 ceph-mon[82146]: pgmap v212: 65 pgs: 65 active+clean; 255 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 4.7 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T23:55:06.934 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:06.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:06 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:07.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:07.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:06 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:08.190 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:07 smithi044 ceph-mon[82146]: pgmap v213: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:55:08.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:08.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:55:08.191 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:07 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:08.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:07 smithi031 ceph-mon[111289]: pgmap v213: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:55:08.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:08.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi044.onzoro", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:55:08.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:07 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:09.146 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:08 smithi044 ceph-mon[82146]: Upgrade: Updating mds.cephfs.smithi044.onzoro 2024-04-22T23:55:09.146 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:08 smithi044 ceph-mon[82146]: Deploying daemon mds.cephfs.smithi044.onzoro on smithi044 2024-04-22T23:55:09.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:08 smithi031 ceph-mon[111289]: Upgrade: Updating mds.cephfs.smithi044.onzoro 2024-04-22T23:55:09.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:08 smithi031 ceph-mon[111289]: Deploying daemon mds.cephfs.smithi044.onzoro on smithi044 2024-04-22T23:55:10.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[111285]: 2024-04-22T23:55:09.702+0000 7f7cd7098700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: pgmap v214: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: osdmap e92: 6 total, 6 up, 6 in 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: Standby daemon mds.cephfs.smithi031.ezuwxk assigned to filesystem cephfs as rank 0 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:55:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:09 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:replay} 2 up:standby 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: pgmap v214: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: osdmap e92: 6 total, 6 up, 6 in 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: Standby daemon mds.cephfs.smithi031.ezuwxk assigned to filesystem cephfs as rank 0 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:55:10.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:09 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:replay} 2 up:standby 2024-04-22T23:55:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:11 smithi031 ceph-mon[111289]: pgmap v216: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-22T23:55:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:11 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:55:12.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:11 smithi044 ceph-mon[82146]: pgmap v216: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-22T23:55:12.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:11 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:55:14.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:13 smithi031 ceph-mon[111289]: pgmap v217: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 5.0 KiB/s wr, 3 op/s 2024-04-22T23:55:14.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:13 smithi044 ceph-mon[82146]: pgmap v217: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 5.0 KiB/s wr, 3 op/s 2024-04-22T23:55:14.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 -- 172.21.15.31:0/2952040394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0101de0 msgr2=0x7f23d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 --2- 172.21.15.31:0/2952040394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0101de0 0x7f23d01021c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009910 tx=0x7f23c0009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 -- 172.21.15.31:0/2952040394 shutdown_connections 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 --2- 172.21.15.31:0/2952040394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d0102700 0x7f23d010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 --2- 172.21.15.31:0/2952040394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0101de0 0x7f23d01021c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 -- 172.21.15.31:0/2952040394 >> 172.21.15.31:0/2952040394 conn(0x7f23d00fb610 msgr2=0x7f23d00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 -- 172.21.15.31:0/2952040394 shutdown_connections 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.542+0000 7f23d637c700 1 -- 172.21.15.31:0/2952040394 wait complete. 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.543+0000 7f23d637c700 1 Processor -- start 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.543+0000 7f23d637c700 1 -- start start 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.543+0000 7f23d637c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d0101de0 0x7f23d019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.543+0000 7f23d637c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 0x7f23d019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.543+0000 7f23d637c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d019f550 con 0x7f23d0102700 2024-04-22T23:55:14.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.543+0000 7f23d637c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0198a00 con 0x7f23d0101de0 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 0x7f23d019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 0x7f23d019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59060/0 (socket says 172.21.15.31:59060) 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 -- 172.21.15.31:0/3141013361 learned_addr learned my addr 172.21.15.31:0/3141013361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cffff700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d0101de0 0x7f23d019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 -- 172.21.15.31:0/3141013361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d0101de0 msgr2=0x7f23d019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d0101de0 0x7f23d019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 -- 172.21.15.31:0/3141013361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00095f0 con 0x7f23d0102700 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cf7fe700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 0x7f23d019eec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b3d0 tx=0x7f23c400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c4011800 con 0x7f23d0102700 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23c4011e20 con 0x7f23d0102700 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d0198ce0 con 0x7f23d0102700 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.544+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c4010ea0 con 0x7f23d0102700 2024-04-22T23:55:14.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.545+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d0199230 con 0x7f23d0102700 2024-04-22T23:55:14.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.545+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d010a750 con 0x7f23d0102700 2024-04-22T23:55:14.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.546+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f23c4010380 con 0x7f23d0102700 2024-04-22T23:55:14.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.547+0000 7f23cd7fa700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ecc0 0x7f23b8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:14.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.547+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f23c408ea90 con 0x7f23d0102700 2024-04-22T23:55:14.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.547+0000 7f23cffff700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ecc0 0x7f23b8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:14.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.547+0000 7f23cffff700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ecc0 0x7f23b8071180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23c000b5c0 tx=0x7f23c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:14.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.552+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f23c405b200 con 0x7f23d0102700 2024-04-22T23:55:14.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.782+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d01994b0 con 0x7f23b806ecc0 2024-04-22T23:55:14.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.784+0000 7f23cd7fa700 1 -- 172.21.15.31:0/3141013361 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f23d01994b0 con 0x7f23b806ecc0 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ecc0 msgr2=0x7f23b8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ecc0 0x7f23b8071180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23c000b5c0 tx=0x7f23c001a040 comp rx=0 tx=0).stop 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 msgr2=0x7f23d019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 0x7f23d019eec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b3d0 tx=0x7f23c400b790 comp rx=0 tx=0).stop 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 shutdown_connections 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f23b806ecc0 0x7f23b8071180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23d0101de0 0x7f23d019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 --2- 172.21.15.31:0/3141013361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d0102700 0x7f23d019eec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 >> 172.21.15.31:0/3141013361 conn(0x7f23d00fb610 msgr2=0x7f23d0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:14.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 shutdown_connections 2024-04-22T23:55:14.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.786+0000 7f23d637c700 1 -- 172.21.15.31:0/3141013361 wait complete. 2024-04-22T23:55:14.790 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:55:14.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 -- 172.21.15.31:0/1346837657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948106cd0 msgr2=0x7f39481070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:14.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 --2- 172.21.15.31:0/1346837657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948106cd0 0x7f39481070b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3938009910 tx=0x7f3938009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:14.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 -- 172.21.15.31:0/1346837657 shutdown_connections 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 --2- 172.21.15.31:0/1346837657 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3948068580 0x7f39480689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 --2- 172.21.15.31:0/1346837657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948106cd0 0x7f39481070b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 -- 172.21.15.31:0/1346837657 >> 172.21.15.31:0/1346837657 conn(0x7f3948075670 msgr2=0x7f3948075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 -- 172.21.15.31:0/1346837657 shutdown_connections 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.895+0000 7f394e9bb700 1 -- 172.21.15.31:0/1346837657 wait complete. 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f394e9bb700 1 Processor -- start 2024-04-22T23:55:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f394e9bb700 1 -- start start 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f394e9bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 0x7f394819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f394e9bb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3948106cd0 0x7f394819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f394e9bb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394819d230 con 0x7f3948068580 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f394e9bb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39481966e0 con 0x7f3948106cd0 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f3947fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 0x7f394819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f3947fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 0x7f394819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59088/0 (socket says 172.21.15.31:59088) 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f3947fff700 1 -- 172.21.15.31:0/933932828 learned_addr learned my addr 172.21.15.31:0/933932828 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.896+0000 7f39477fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3948106cd0 0x7f394819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.897+0000 7f3947fff700 1 -- 172.21.15.31:0/933932828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3948106cd0 msgr2=0x7f394819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.897+0000 7f3947fff700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3948106cd0 0x7f394819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.897+0000 7f3947fff700 1 -- 172.21.15.31:0/933932828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39380095f0 con 0x7f3948068580 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.897+0000 7f3947fff700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 0x7f394819c660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39380092e0 tx=0x7f393800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.898+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f393801d070 con 0x7f3948068580 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.898+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3938005e00 con 0x7f3948068580 2024-04-22T23:55:14.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.898+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3948196960 con 0x7f3948068580 2024-04-22T23:55:14.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.898+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3938017400 con 0x7f3948068580 2024-04-22T23:55:14.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.899+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3948196e00 con 0x7f3948068580 2024-04-22T23:55:14.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.900+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39481091f0 con 0x7f3948068580 2024-04-22T23:55:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.900+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f39380176c0 con 0x7f3948068580 2024-04-22T23:55:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.901+0000 7f39457fa700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f393006ecc0 0x7f3930071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.901+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3938090590 con 0x7f3948068580 2024-04-22T23:55:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.901+0000 7f39477fe700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f393006ecc0 0x7f3930071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:14.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.901+0000 7f39477fe700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f393006ecc0 0x7f3930071180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3948197b50 tx=0x7f393c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:14.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:14.906+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f393805dc30 con 0x7f3948068580 2024-04-22T23:55:15.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.148+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3948197070 con 0x7f393006ecc0 2024-04-22T23:55:15.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.150+0000 7f39457fa700 1 -- 172.21.15.31:0/933932828 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3948197070 con 0x7f393006ecc0 2024-04-22T23:55:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.153+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f393006ecc0 msgr2=0x7f3930071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.153+0000 7f394e9bb700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f393006ecc0 0x7f3930071180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3948197b50 tx=0x7f393c00b410 comp rx=0 tx=0).stop 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.153+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 msgr2=0x7f394819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.153+0000 7f394e9bb700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 0x7f394819c660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f39380092e0 tx=0x7f393800bd70 comp rx=0 tx=0).stop 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.153+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 shutdown_connections 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.154+0000 7f394e9bb700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f393006ecc0 0x7f3930071180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.154+0000 7f394e9bb700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3948068580 0x7f394819c660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.154+0000 7f394e9bb700 1 --2- 172.21.15.31:0/933932828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3948106cd0 0x7f394819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.154+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 >> 172.21.15.31:0/933932828 conn(0x7f3948075670 msgr2=0x7f39480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.154+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 shutdown_connections 2024-04-22T23:55:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.154+0000 7f394e9bb700 1 -- 172.21.15.31:0/933932828 wait complete. 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 -- 172.21.15.31:0/3673211415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588106cd0 msgr2=0x7f05881070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 --2- 172.21.15.31:0/3673211415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588106cd0 0x7f05881070b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0584009910 tx=0x7f0584009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 -- 172.21.15.31:0/3673211415 shutdown_connections 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 --2- 172.21.15.31:0/3673211415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0588068580 0x7f05880689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 --2- 172.21.15.31:0/3673211415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588106cd0 0x7f05881070b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 -- 172.21.15.31:0/3673211415 >> 172.21.15.31:0/3673211415 conn(0x7f0588075670 msgr2=0x7f0588075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 -- 172.21.15.31:0/3673211415 shutdown_connections 2024-04-22T23:55:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.267+0000 7f058ff6e700 1 -- 172.21.15.31:0/3673211415 wait complete. 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058ff6e700 1 Processor -- start 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058ff6e700 1 -- start start 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058ff6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 0x7f0588196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058ff6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0588106cd0 0x7f0588196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058ff6e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0588197550 con 0x7f0588068580 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058ff6e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058819b360 con 0x7f0588106cd0 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058dd0a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 0x7f0588196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058dd0a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 0x7f0588196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59106/0 (socket says 172.21.15.31:59106) 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.268+0000 7f058dd0a700 1 -- 172.21.15.31:0/2979567150 learned_addr learned my addr 172.21.15.31:0/2979567150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058d509700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0588106cd0 0x7f0588196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058dd0a700 1 -- 172.21.15.31:0/2979567150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0588106cd0 msgr2=0x7f0588196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058dd0a700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0588106cd0 0x7f0588196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058dd0a700 1 -- 172.21.15.31:0/2979567150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05840095f0 con 0x7f0588068580 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058dd0a700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 0x7f0588196980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0584006010 tx=0x7f058400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058401d070 con 0x7f0588068580 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0584004fb0 con 0x7f0588068580 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058819b5e0 con 0x7f0588068580 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058400f690 con 0x7f0588068580 2024-04-22T23:55:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.269+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058819bad0 con 0x7f0588068580 2024-04-22T23:55:15.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.271+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05881091f0 con 0x7f0588068580 2024-04-22T23:55:15.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.271+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f05840228a0 con 0x7f0588068580 2024-04-22T23:55:15.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.271+0000 7f057effd700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f057406ecc0 0x7f0574071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:15.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.271+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f0584090890 con 0x7f0588068580 2024-04-22T23:55:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.271+0000 7f058d509700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f057406ecc0 0x7f0574071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.272+0000 7f058d509700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f057406ecc0 0x7f0574071180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0578006fd0 tx=0x7f0578008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:15.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.276+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f058405d000 con 0x7f0588068580 2024-04-22T23:55:15.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.515+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0588197d50 con 0x7f057406ecc0 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.527+0000 7f057effd700 1 -- 172.21.15.31:0/2979567150 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f0588197d50 con 0x7f057406ecc0 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 39s ago 18m 24.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (18m) 39s ago 18m 8980k - 18.2.1 5be31c24972a 77e01c072230 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (12m) 14s ago 17m 8887k - 18.2.1 5be31c24972a 7a7ec676d3b6 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (4m) 39s ago 18m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (4m) 14s ago 17m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:55:15.518 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 39s ago 17m 81.3M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (63s) 39s ago 12m 13.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a7e757804d66 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (45s) 39s ago 12m 14.8M - 19.0.0-3252-g43be0201 a0434699c1d7 961938a7288c 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (22s) 14s ago 12m 14.9M - 19.0.0-3252-g43be0201 a0434699c1d7 053ad6c26779 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (12m) 14s ago 12m 172M - 18.2.1 5be31c24972a 736716b51b58 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (9m) 39s ago 19m 512M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (8m) 14s ago 17m 453M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (5m) 39s ago 19m 55.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (5m) 14s ago 17m 43.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (8m) 39s ago 18m 15.3M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 14s ago 17m 16.2M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 39s ago 16m 101M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (3m) 39s ago 15m 128M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (3m) 39s ago 15m 85.9M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (2m) 14s ago 14m 90.8M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:55:15.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (114s) 14s ago 14m 85.0M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:55:15.520 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (81s) 14s ago 14m 88.2M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 38dba54af5f6 2024-04-22T23:55:15.520 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (7m) 39s ago 17m 76.7M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:55:15.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f057406ecc0 msgr2=0x7f0574071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f057406ecc0 0x7f0574071180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0578006fd0 tx=0x7f0578008040 comp rx=0 tx=0).stop 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 msgr2=0x7f0588196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 0x7f0588196980 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0584006010 tx=0x7f058400be20 comp rx=0 tx=0).stop 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 shutdown_connections 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f057406ecc0 0x7f0574071180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0588068580 0x7f0588196980 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 --2- 172.21.15.31:0/2979567150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0588106cd0 0x7f0588196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 >> 172.21.15.31:0/2979567150 conn(0x7f0588075670 msgr2=0x7f05880fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 shutdown_connections 2024-04-22T23:55:15.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.529+0000 7f058ff6e700 1 -- 172.21.15.31:0/2979567150 wait complete. 2024-04-22T23:55:15.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 -- 172.21.15.31:0/932830165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80ff5a0 msgr2=0x7f09e80ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/932830165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80ff5a0 0x7f09e80ff980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009910 tx=0x7f09d8009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 -- 172.21.15.31:0/932830165 shutdown_connections 2024-04-22T23:55:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/932830165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80fff50 0x7f09e8105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/932830165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80ff5a0 0x7f09e80ff980 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 -- 172.21.15.31:0/932830165 >> 172.21.15.31:0/932830165 conn(0x7f09e8074e30 msgr2=0x7f09e8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 -- 172.21.15.31:0/932830165 shutdown_connections 2024-04-22T23:55:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.636+0000 7f09ee5dc700 1 -- 172.21.15.31:0/932830165 wait complete. 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.637+0000 7f09ee5dc700 1 Processor -- start 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.637+0000 7f09ee5dc700 1 -- start start 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09ee5dc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80ff5a0 0x7f09e819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09ee5dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 0x7f09e819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09ee5dc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e819d360 con 0x7f09e80fff50 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09ee5dc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8196810 con 0x7f09e80ff5a0 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 0x7f09e819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 0x7f09e819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59132/0 (socket says 172.21.15.31:59132) 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 -- 172.21.15.31:0/4009052107 learned_addr learned my addr 172.21.15.31:0/4009052107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e7fff700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80ff5a0 0x7f09e819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:15.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 -- 172.21.15.31:0/4009052107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80ff5a0 msgr2=0x7f09e819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80ff5a0 0x7f09e819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 -- 172.21.15.31:0/4009052107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80095f0 con 0x7f09e80fff50 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e7fff700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80ff5a0 0x7f09e819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.638+0000 7f09e77fe700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 0x7f09e819ccd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00b480 tx=0x7f09dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.639+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09dc0118c0 con 0x7f09e80fff50 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.639+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09dc010380 con 0x7f09e80fff50 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.639+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e8196af0 con 0x7f09e80fff50 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.639+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09dc00f4f0 con 0x7f09e80fff50 2024-04-22T23:55:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.639+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e8197040 con 0x7f09e80fff50 2024-04-22T23:55:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.640+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e8109270 con 0x7f09e80fff50 2024-04-22T23:55:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.640+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f09dc0104f0 con 0x7f09e80fff50 2024-04-22T23:55:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.641+0000 7f09e57fa700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f09d006ecc0 0x7f09d0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.641+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f09dc08f020 con 0x7f09e80fff50 2024-04-22T23:55:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.641+0000 7f09e7fff700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f09d006ecc0 0x7f09d0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.642+0000 7f09e7fff700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f09d006ecc0 0x7f09d0071180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005100 tx=0x7f09d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:15.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.649+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f09dc05b810 con 0x7f09e80fff50 2024-04-22T23:55:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.950+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09e804f2e0 con 0x7f09e80fff50 2024-04-22T23:55:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.950+0000 7f09e57fa700 1 -- 172.21.15.31:0/4009052107 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f09dc05b3a0 con 0x7f09e80fff50 2024-04-22T23:55:15.941 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:55:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:55:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 3 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 13 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:55:15.942 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f09d006ecc0 msgr2=0x7f09d0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f09d006ecc0 0x7f09d0071180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005100 tx=0x7f09d801a040 comp rx=0 tx=0).stop 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 msgr2=0x7f09e819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 0x7f09e819ccd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00b480 tx=0x7f09dc00b840 comp rx=0 tx=0).stop 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 shutdown_connections 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f09d006ecc0 0x7f09d0071180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e80ff5a0 0x7f09e819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 --2- 172.21.15.31:0/4009052107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09e80fff50 0x7f09e819ccd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:15.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 >> 172.21.15.31:0/4009052107 conn(0x7f09e8074e30 msgr2=0x7f09e80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:15.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 shutdown_connections 2024-04-22T23:55:15.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:15.954+0000 7f09ee5dc700 1 -- 172.21.15.31:0/4009052107 wait complete. 2024-04-22T23:55:16.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 -- 172.21.15.31:0/2276973482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20100ea0 msgr2=0x7fec20101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 --2- 172.21.15.31:0/2276973482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20100ea0 0x7fec20101280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec1c009910 tx=0x7fec1c009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 -- 172.21.15.31:0/2276973482 shutdown_connections 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 --2- 172.21.15.31:0/2276973482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20101850 0x7fec201058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 --2- 172.21.15.31:0/2276973482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20100ea0 0x7fec20101280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 -- 172.21.15.31:0/2276973482 >> 172.21.15.31:0/2276973482 conn(0x7fec200fc750 msgr2=0x7fec200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 -- 172.21.15.31:0/2276973482 shutdown_connections 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.067+0000 7fec27d76700 1 -- 172.21.15.31:0/2276973482 wait complete. 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec27d76700 1 Processor -- start 2024-04-22T23:55:16.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec27d76700 1 -- start start 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec27d76700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20100ea0 0x7fec20073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec27d76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 0x7fec200735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec27d76700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20073b10 con 0x7fec20101850 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec27d76700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20073c50 con 0x7fec20100ea0 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec25311700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 0x7fec200735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec25311700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 0x7fec200735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59154/0 (socket says 172.21.15.31:59154) 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec25311700 1 -- 172.21.15.31:0/2018856840 learned_addr learned my addr 172.21.15.31:0/2018856840 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.068+0000 7fec25b12700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20100ea0 0x7fec20073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec25311700 1 -- 172.21.15.31:0/2018856840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20100ea0 msgr2=0x7fec20073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec25311700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20100ea0 0x7fec20073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec25311700 1 -- 172.21.15.31:0/2018856840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0095f0 con 0x7fec20101850 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec25311700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 0x7fec200735d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d900 tx=0x7fec1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec10009900 con 0x7fec20101850 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec10010460 con 0x7fec20101850 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec201a2b10 con 0x7fec20101850 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec1000f5f0 con 0x7fec20101850 2024-04-22T23:55:16.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.069+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec201a2f20 con 0x7fec20101850 2024-04-22T23:55:16.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.070+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec20109240 con 0x7fec20101850 2024-04-22T23:55:16.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.071+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fec100105d0 con 0x7fec20101850 2024-04-22T23:55:16.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.071+0000 7fec16ffd700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fec0c06ecc0 0x7fec0c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.071+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fec1008ec40 con 0x7fec20101850 2024-04-22T23:55:16.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.071+0000 7fec25b12700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fec0c06ecc0 0x7fec0c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.072+0000 7fec25b12700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fec0c06ecc0 0x7fec0c071180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006010 tx=0x7fec1c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:16.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.081+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fec100577e0 con 0x7fec20101850 2024-04-22T23:55:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:15 smithi031 ceph-mon[111289]: pgmap v218: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-22T23:55:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:15 smithi031 ceph-mon[111289]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:15 smithi044 ceph-mon[82146]: pgmap v218: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-22T23:55:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:15 smithi044 ceph-mon[82146]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.340+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fec2004f2e0 con 0x7fec20101850 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.341+0000 7fec16ffd700 1 -- 172.21.15.31:0/2018856840 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1748 (secure 0 0 0) 0x7fec100176f0 con 0x7fec20101850 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:e32 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 32 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:55:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:55:09.707761+0000 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 92 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34406} 2024-04-22T23:55:16.333 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{0:34406} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:16.334 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/1426404610,v1:172.21.15.31:6829/1426404610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:16.335 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/4021256007,v1:172.21.15.44:6825/4021256007] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.344+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fec0c06ecc0 msgr2=0x7fec0c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.344+0000 7fec27d76700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fec0c06ecc0 0x7fec0c071180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fec1c006010 tx=0x7fec1c01a040 comp rx=0 tx=0).stop 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 msgr2=0x7fec200735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 0x7fec200735d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d900 tx=0x7fec1000dc10 comp rx=0 tx=0).stop 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 shutdown_connections 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fec0c06ecc0 0x7fec0c071180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec20100ea0 0x7fec20073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 --2- 172.21.15.31:0/2018856840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec20101850 0x7fec200735d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 >> 172.21.15.31:0/2018856840 conn(0x7fec200fc750 msgr2=0x7fec200fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:16.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 shutdown_connections 2024-04-22T23:55:16.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.345+0000 7fec27d76700 1 -- 172.21.15.31:0/2018856840 wait complete. 2024-04-22T23:55:16.337 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 32 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 -- 172.21.15.31:0/108760234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 msgr2=0x7f46441058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 --2- 172.21.15.31:0/108760234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f46441058a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4638009910 tx=0x7f4638009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 -- 172.21.15.31:0/108760234 shutdown_connections 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 --2- 172.21.15.31:0/108760234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f46441058a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 --2- 172.21.15.31:0/108760234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4644100ea0 0x7f4644101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 -- 172.21.15.31:0/108760234 >> 172.21.15.31:0/108760234 conn(0x7f46440fc750 msgr2=0x7f46440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 -- 172.21.15.31:0/108760234 shutdown_connections 2024-04-22T23:55:16.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.461+0000 7f4648f81700 1 -- 172.21.15.31:0/108760234 wait complete. 2024-04-22T23:55:16.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4648f81700 1 Processor -- start 2024-04-22T23:55:16.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4648f81700 1 -- start start 2024-04-22T23:55:16.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4648f81700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4644100ea0 0x7f464419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4648f81700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f464419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4648f81700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464419d350 con 0x7f4644101850 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4648f81700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644196800 con 0x7f4644100ea0 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4641d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f464419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4641d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f464419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59160/0 (socket says 172.21.15.31:59160) 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4641d9b700 1 -- 172.21.15.31:0/1928379942 learned_addr learned my addr 172.21.15.31:0/1928379942 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f464259c700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4644100ea0 0x7f464419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4641d9b700 1 -- 172.21.15.31:0/1928379942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4644100ea0 msgr2=0x7f464419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4641d9b700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4644100ea0 0x7f464419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.462+0000 7f4641d9b700 1 -- 172.21.15.31:0/1928379942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380095f0 con 0x7f4644101850 2024-04-22T23:55:16.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.463+0000 7f4641d9b700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f464419ccc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f463800bda0 tx=0x7f463800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.463+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f463801d070 con 0x7f4644101850 2024-04-22T23:55:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.463+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463800f460 con 0x7f4644101850 2024-04-22T23:55:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.463+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4644196a80 con 0x7f4644101850 2024-04-22T23:55:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.463+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4638017630 con 0x7f4644101850 2024-04-22T23:55:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.463+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644196f70 con 0x7f4644101850 2024-04-22T23:55:16.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.464+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46441091f0 con 0x7f4644101850 2024-04-22T23:55:16.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.465+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f463800fab0 con 0x7f4644101850 2024-04-22T23:55:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.466+0000 7f46337fe700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f462c06ecc0 0x7f462c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.466+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f46380904a0 con 0x7f4644101850 2024-04-22T23:55:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.466+0000 7f464259c700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f462c06ecc0 0x7f462c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.467+0000 7f464259c700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f462c06ecc0 0x7f462c071180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4634009a10 tx=0x7f4634009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:16.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.475+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f463805cc10 con 0x7f4644101850 2024-04-22T23:55:16.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.706+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4644197250 con 0x7f462c06ecc0 2024-04-22T23:55:16.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.708+0000 7f46337fe700 1 -- 172.21.15.31:0/1928379942 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4644197250 con 0x7f462c06ecc0 2024-04-22T23:55:16.699 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:55:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-22T23:55:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd" 2024-04-22T23:55:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:55:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "15/23 daemons upgraded", 2024-04-22T23:55:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-04-22T23:55:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:55:16.701 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:55:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f462c06ecc0 msgr2=0x7f462c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f462c06ecc0 0x7f462c071180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4634009a10 tx=0x7f4634009190 comp rx=0 tx=0).stop 2024-04-22T23:55:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 msgr2=0x7f464419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f464419ccc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f463800bda0 tx=0x7f463800bdd0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 shutdown_connections 2024-04-22T23:55:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f462c06ecc0 0x7f462c071180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4644100ea0 0x7f464419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 --2- 172.21.15.31:0/1928379942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4644101850 0x7f464419ccc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 >> 172.21.15.31:0/1928379942 conn(0x7f46440fc750 msgr2=0x7f46440fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 shutdown_connections 2024-04-22T23:55:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.711+0000 7f4648f81700 1 -- 172.21.15.31:0/1928379942 wait complete. 2024-04-22T23:55:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.820+0000 7f3e04703700 1 -- 172.21.15.31:0/1151880452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 msgr2=0x7f3dfc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.820+0000 7f3e04703700 1 --2- 172.21.15.31:0/1151880452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0689e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009910 tx=0x7f3dec009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.820+0000 7f3e04703700 1 -- 172.21.15.31:0/1151880452 shutdown_connections 2024-04-22T23:55:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.820+0000 7f3e04703700 1 --2- 172.21.15.31:0/1151880452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0689e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.820+0000 7f3e04703700 1 --2- 172.21.15.31:0/1151880452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc106cd0 0x7f3dfc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.820+0000 7f3e04703700 1 -- 172.21.15.31:0/1151880452 >> 172.21.15.31:0/1151880452 conn(0x7f3dfc075670 msgr2=0x7f3dfc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 -- 172.21.15.31:0/1151880452 shutdown_connections 2024-04-22T23:55:16.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 -- 172.21.15.31:0/1151880452 wait complete. 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 Processor -- start 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 -- start start 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc106cd0 0x7f3dfc100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc100770 con 0x7f3dfc068580 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.821+0000 7f3e04703700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dfc1008b0 con 0x7f3dfc106cd0 2024-04-22T23:55:16.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59188/0 (socket says 172.21.15.31:59188) 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 -- 172.21.15.31:0/2824256050 learned_addr learned my addr 172.21.15.31:0/2824256050 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e01c9e700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc106cd0 0x7f3dfc100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 -- 172.21.15.31:0/2824256050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc106cd0 msgr2=0x7f3dfc100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc106cd0 0x7f3dfc100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 -- 172.21.15.31:0/2824256050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec0095f0 con 0x7f3dfc068580 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e0249f700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0ffcf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3df800b4b0 tx=0x7f3df800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df8011800 con 0x7f3dfc068580 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc1a2b50 con 0x7f3dfc068580 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df8011e20 con 0x7f3dfc068580 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df800f2e0 con 0x7f3dfc068580 2024-04-22T23:55:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.822+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc1a2f60 con 0x7f3dfc068580 2024-04-22T23:55:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.823+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dfc1091d0 con 0x7f3dfc068580 2024-04-22T23:55:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.824+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3df8010380 con 0x7f3dfc068580 2024-04-22T23:55:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.825+0000 7f3df37fe700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3de806ecc0 0x7f3de8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.825+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3df808eb10 con 0x7f3dfc068580 2024-04-22T23:55:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.825+0000 7f3e01c9e700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3de806ecc0 0x7f3de8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.825+0000 7f3e01c9e700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3de806ecc0 0x7f3de8071180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc1014e0 tx=0x7f3dec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:16.830+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f3df8057750 con 0x7f3dfc068580 2024-04-22T23:55:16.934 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:16 smithi044 ceph-mon[82146]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:16.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:16 smithi044 ceph-mon[82146]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:16.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:16 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4009052107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:16.935 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:16 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2018856840' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:55:17.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.130+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3dfc04f2e0 con 0x7f3dfc068580 2024-04-22T23:55:17.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:16 smithi031 ceph-mon[111289]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:17.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:16 smithi031 ceph-mon[111289]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:17.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:16 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4009052107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:17.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:16 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2018856840' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:55:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.130+0000 7f3df37fe700 1 -- 172.21.15.31:0/2824256050 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f3df805ae10 con 0x7f3dfc068580 2024-04-22T23:55:17.122 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-22T23:55:17.122 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-22T23:55:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs is degraded 2024-04-22T23:55:17.122 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:55:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:55:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3de806ecc0 msgr2=0x7f3de8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3de806ecc0 0x7f3de8071180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc1014e0 tx=0x7f3dec01a040 comp rx=0 tx=0).stop 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 msgr2=0x7f3dfc0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0ffcf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3df800b4b0 tx=0x7f3df800b7c0 comp rx=0 tx=0).stop 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 shutdown_connections 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3de806ecc0 0x7f3de8071180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dfc068580 0x7f3dfc0ffcf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 --2- 172.21.15.31:0/2824256050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc106cd0 0x7f3dfc100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.132+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 >> 172.21.15.31:0/2824256050 conn(0x7f3dfc075670 msgr2=0x7f3dfc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.133+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 shutdown_connections 2024-04-22T23:55:17.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:17.133+0000 7f3e04703700 1 -- 172.21.15.31:0/2824256050 wait complete. 2024-04-22T23:55:18.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:17 smithi031 ceph-mon[111289]: pgmap v219: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-22T23:55:18.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:17 smithi031 ceph-mon[111289]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:18.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:17 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2824256050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:55:18.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:17 smithi044 ceph-mon[82146]: pgmap v219: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-22T23:55:18.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:17 smithi044 ceph-mon[82146]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:18.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:17 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2824256050' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:55:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: pgmap v220: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-22T23:55:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.44:6826/2995756226,v1:172.21.15.44:6827/2995756226] up:boot 2024-04-22T23:55:19.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:replay} 3 up:standby 2024-04-22T23:55:19.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:19 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: pgmap v220: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.44:6826/2995756226,v1:172.21.15.44:6827/2995756226] up:boot 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:replay} 3 up:standby 2024-04-22T23:55:19.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:19 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:55:21.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:20 smithi031 ceph-mon[111289]: reconnect by client.14542 192.168.0.1:0/1415787653 after 0.000999984 2024-04-22T23:55:21.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:20 smithi031 ceph-mon[111289]: reconnect by client.24345 192.168.0.1:0/3446383703 after 0.000999984 2024-04-22T23:55:21.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:20 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:reconnect 2024-04-22T23:55:21.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:20 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:reconnect} 3 up:standby 2024-04-22T23:55:21.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:20 smithi044 ceph-mon[82146]: reconnect by client.14542 192.168.0.1:0/1415787653 after 0.000999984 2024-04-22T23:55:21.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:20 smithi044 ceph-mon[82146]: reconnect by client.24345 192.168.0.1:0/3446383703 after 0.000999984 2024-04-22T23:55:21.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:20 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:reconnect 2024-04-22T23:55:21.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:20 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:reconnect} 3 up:standby 2024-04-22T23:55:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:21 smithi031 ceph-mon[111289]: pgmap v221: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-22T23:55:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:21 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:rejoin 2024-04-22T23:55:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:21 smithi031 ceph-mon[111289]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:rejoin} 3 up:standby 2024-04-22T23:55:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:21 smithi031 ceph-mon[111289]: daemon mds.cephfs.smithi031.ezuwxk is now active in filesystem cephfs as rank 0 2024-04-22T23:55:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:22.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:21 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:21 smithi044 ceph-mon[82146]: pgmap v221: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-22T23:55:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:21 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:rejoin 2024-04-22T23:55:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:21 smithi044 ceph-mon[82146]: fsmap cephfs:1/1 {0=cephfs.smithi031.ezuwxk=up:rejoin} 3 up:standby 2024-04-22T23:55:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:21 smithi044 ceph-mon[82146]: daemon mds.cephfs.smithi031.ezuwxk is now active in filesystem cephfs as rank 0 2024-04-22T23:55:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:21 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:23.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:22 smithi031 ceph-mon[111289]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:55:23.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:22 smithi031 ceph-mon[111289]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:active 2024-04-22T23:55:23.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:22 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 3 up:standby 2024-04-22T23:55:23.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:22 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:23.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:22 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:23.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:22 smithi044 ceph-mon[82146]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:55:23.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:22 smithi044 ceph-mon[82146]: mds.? [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] up:active 2024-04-22T23:55:23.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:22 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 3 up:standby 2024-04-22T23:55:23.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:22 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:23.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:22 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:24.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:23 smithi031 ceph-mon[111289]: pgmap v222: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-22T23:55:24.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:23 smithi044 ceph-mon[82146]: pgmap v222: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-22T23:55:26.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:25 smithi031 ceph-mon[111289]: pgmap v223: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T23:55:26.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:25 smithi044 ceph-mon[82146]: pgmap v223: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-22T23:55:27.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: pgmap v224: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:55:27.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all mds 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.ezuwxk"}]': finished 2024-04-22T23:55:27.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.usiyoa"}]': finished 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.ibkztg"}]': finished 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.onzoro"}]': finished 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: Upgrade: Setting filesystem cephfs Joinable 2024-04-22T23:55:27.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:27 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-22T23:55:27.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: pgmap v224: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:55:27.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:27.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:27.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all mds 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.ezuwxk"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.ezuwxk"}]': finished 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.usiyoa"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.usiyoa"}]': finished 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.ibkztg"}]: dispatch 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.ibkztg"}]': finished 2024-04-22T23:55:27.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.onzoro"}]: dispatch 2024-04-22T23:55:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi044.onzoro"}]': finished 2024-04-22T23:55:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: Upgrade: Setting filesystem cephfs Joinable 2024-04-22T23:55:27.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:27 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-22T23:55:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:28 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-22T23:55:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:28 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 3 up:standby 2024-04-22T23:55:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:28 smithi031 ceph-mon[111289]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-22T23:55:28.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:28 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-22T23:55:28.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:28 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-22T23:55:28.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:28 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 3 up:standby 2024-04-22T23:55:28.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:28 smithi044 ceph-mon[82146]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-22T23:55:28.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:28 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-22T23:55:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: pgmap v225: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 6.7 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-22T23:55:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-22T23:55:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 3 up:standby 2024-04-22T23:55:29.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all rgw 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all rbd-mirror 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:29.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:29 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:29.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: pgmap v225: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 6.7 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-22T23:55:29.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-22T23:55:29.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 3 up:standby 2024-04-22T23:55:29.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:55:29.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all rgw 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all rbd-mirror 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:29.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:29 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:31 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:31.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:31 smithi031 ceph-mon[111289]: pgmap v226: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 7.0 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:55:31.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:31.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:31 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:31.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:31 smithi044 ceph-mon[82146]: pgmap v226: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 7.0 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:55:33.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:33 smithi031 ceph-mon[111289]: pgmap v227: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:55:33.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:33 smithi044 ceph-mon[82146]: pgmap v227: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 8.7 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:55:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:35 smithi044 ceph-mon[82146]: pgmap v228: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 9.7 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-22T23:55:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:35 smithi044 ceph-mon[82146]: Upgrade: Updating ceph-exporter.smithi031 (1/2) 2024-04-22T23:55:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:35 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:35 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:55:35.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:35 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:35.960 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:35 smithi044 ceph-mon[82146]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:55:36.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:35 smithi031 ceph-mon[111289]: pgmap v228: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 9.7 MiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-22T23:55:36.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:35 smithi031 ceph-mon[111289]: Upgrade: Updating ceph-exporter.smithi031 (1/2) 2024-04-22T23:55:36.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:36.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:55:36.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:35 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:36.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:35 smithi031 ceph-mon[111289]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-04-22T23:55:37.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:37 smithi044 ceph-mon[82146]: pgmap v229: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-22T23:55:38.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:37 smithi031 ceph-mon[111289]: pgmap v229: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-22T23:55:40.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:39 smithi044 ceph-mon[82146]: pgmap v230: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-04-22T23:55:40.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:39 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:40.087 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:39 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:40.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:39 smithi031 ceph-mon[111289]: pgmap v230: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-04-22T23:55:40.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:39 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:40.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:39 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:41.952 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:41 smithi044 ceph-mon[82146]: pgmap v231: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-04-22T23:55:41.952 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:41.952 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:55:41.952 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:41 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:42.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:41 smithi031 ceph-mon[111289]: pgmap v231: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-04-22T23:55:42.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:42.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi044", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-22T23:55:42.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:41 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:43.101 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:42 smithi044 ceph-mon[82146]: Upgrade: Updating ceph-exporter.smithi044 (2/2) 2024-04-22T23:55:43.102 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:42 smithi044 ceph-mon[82146]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:55:43.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:42 smithi031 ceph-mon[111289]: Upgrade: Updating ceph-exporter.smithi044 (2/2) 2024-04-22T23:55:43.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:42 smithi031 ceph-mon[111289]: Deploying daemon ceph-exporter.smithi044 on smithi044 2024-04-22T23:55:43.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:43 smithi044 ceph-mon[82146]: pgmap v232: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 4 op/s 2024-04-22T23:55:44.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:43 smithi031 ceph-mon[111289]: pgmap v232: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 4 op/s 2024-04-22T23:55:46.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:45 smithi044 ceph-mon[82146]: pgmap v233: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-04-22T23:55:46.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:46.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:46.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:46.062 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:45 smithi031 ceph-mon[111289]: pgmap v233: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-04-22T23:55:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:55:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.256+0000 7fcb31aac700 1 -- 172.21.15.31:0/3173008282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c101850 msgr2=0x7fcb2c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.256+0000 7fcb31aac700 1 --2- 172.21.15.31:0/3173008282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c101850 0x7fcb2c1058a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009910 tx=0x7fcb20009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.256+0000 7fcb31aac700 1 -- 172.21.15.31:0/3173008282 shutdown_connections 2024-04-22T23:55:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.256+0000 7fcb31aac700 1 --2- 172.21.15.31:0/3173008282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c101850 0x7fcb2c1058a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.257+0000 7fcb31aac700 1 --2- 172.21.15.31:0/3173008282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.257+0000 7fcb31aac700 1 -- 172.21.15.31:0/3173008282 >> 172.21.15.31:0/3173008282 conn(0x7fcb2c0fc750 msgr2=0x7fcb2c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.257+0000 7fcb31aac700 1 -- 172.21.15.31:0/3173008282 shutdown_connections 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.257+0000 7fcb31aac700 1 -- 172.21.15.31:0/3173008282 wait complete. 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.257+0000 7fcb31aac700 1 Processor -- start 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.257+0000 7fcb31aac700 1 -- start start 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb31aac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb31aac700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb2c101850 0x7fcb2c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb31aac700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19d2d0 con 0x7fcb2c100ea0 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb31aac700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c196780 con 0x7fcb2c101850 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56702/0 (socket says 172.21.15.31:56702) 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 -- 172.21.15.31:0/1493713656 learned_addr learned my addr 172.21.15.31:0/1493713656 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2affd700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb2c101850 0x7fcb2c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 -- 172.21.15.31:0/1493713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb2c101850 msgr2=0x7fcb2c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb2c101850 0x7fcb2c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 -- 172.21.15.31:0/1493713656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200095f0 con 0x7fcb2c100ea0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb2b7fe700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c19c700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e920 tx=0x7fcb1c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1c00c9d0 con 0x7fcb2c100ea0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb1c010430 con 0x7fcb2c100ea0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c196a60 con 0x7fcb2c100ea0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.258+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c196f80 con 0x7fcb2c100ea0 2024-04-22T23:55:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.259+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1c018630 con 0x7fcb2c100ea0 2024-04-22T23:55:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.259+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c1092a0 con 0x7fcb2c100ea0 2024-04-22T23:55:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.260+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fcb1c0105a0 con 0x7fcb2c100ea0 2024-04-22T23:55:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.260+0000 7fcb28ff9700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fcb1406ecc0 0x7fcb14071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.260+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fcb1c014070 con 0x7fcb2c100ea0 2024-04-22T23:55:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.260+0000 7fcb2affd700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fcb1406ecc0 0x7fcb14071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.261+0000 7fcb2affd700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fcb1406ecc0 0x7fcb14071180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb20005b40 tx=0x7fcb20005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:47.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.265+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fcb1c054150 con 0x7fcb2c100ea0 2024-04-22T23:55:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.504+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb2c197da0 con 0x7fcb1406ecc0 2024-04-22T23:55:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.505+0000 7fcb28ff9700 1 -- 172.21.15.31:0/1493713656 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fcb2c197da0 con 0x7fcb1406ecc0 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fcb1406ecc0 msgr2=0x7fcb14071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fcb1406ecc0 0x7fcb14071180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcb20005b40 tx=0x7fcb20005a90 comp rx=0 tx=0).stop 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 msgr2=0x7fcb2c19c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c19c700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00e920 tx=0x7fcb1c00ec30 comp rx=0 tx=0).stop 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 shutdown_connections 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fcb1406ecc0 0x7fcb14071180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c100ea0 0x7fcb2c19c700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 --2- 172.21.15.31:0/1493713656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb2c101850 0x7fcb2c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 >> 172.21.15.31:0/1493713656 conn(0x7fcb2c0fc750 msgr2=0x7fcb2c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 shutdown_connections 2024-04-22T23:55:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.507+0000 7fcb31aac700 1 -- 172.21.15.31:0/1493713656 wait complete. 2024-04-22T23:55:47.509 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2111614332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 msgr2=0x7fba141047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2111614332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba141047a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fba04009910 tx=0x7fba04009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2111614332 shutdown_connections 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2111614332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba141047a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2111614332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140ffda0 0x7fba14100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2111614332 >> 172.21.15.31:0/2111614332 conn(0x7fba140fb650 msgr2=0x7fba140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2111614332 shutdown_connections 2024-04-22T23:55:47.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.616+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2111614332 wait complete. 2024-04-22T23:55:47.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1c3d9700 1 Processor -- start 2024-04-22T23:55:47.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1c3d9700 1 -- start start 2024-04-22T23:55:47.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1c3d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140ffda0 0x7fba141977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1c3d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba14197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1c3d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba14198300 con 0x7fba14100750 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1c3d9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba14198470 con 0x7fba140ffda0 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba19974700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba14197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba19974700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba14197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56712/0 (socket says 172.21.15.31:56712) 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba19974700 1 -- 172.21.15.31:0/2467196570 learned_addr learned my addr 172.21.15.31:0/2467196570 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.617+0000 7fba1a175700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140ffda0 0x7fba141977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba19974700 1 -- 172.21.15.31:0/2467196570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140ffda0 msgr2=0x7fba141977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba19974700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140ffda0 0x7fba141977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba19974700 1 -- 172.21.15.31:0/2467196570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040095f0 con 0x7fba14100750 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba19974700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba14197ce0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fba040059d0 tx=0x7fba0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0401d070 con 0x7fba14100750 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba0400b610 con 0x7fba14100750 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba141a0810 con 0x7fba14100750 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba04022470 con 0x7fba14100750 2024-04-22T23:55:47.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.618+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba141a0c80 con 0x7fba14100750 2024-04-22T23:55:47.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.619+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba141081a0 con 0x7fba14100750 2024-04-22T23:55:47.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.619+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fba04003710 con 0x7fba14100750 2024-04-22T23:55:47.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.620+0000 7fba0b7fe700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fba0006ecc0 0x7fba00071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.620+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fba04090450 con 0x7fba14100750 2024-04-22T23:55:47.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.620+0000 7fba1a175700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fba0006ecc0 0x7fba00071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.620+0000 7fba1a175700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fba0006ecc0 0x7fba00071180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba10009c60 tx=0x7fba10009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:47.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.625+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fba04059090 con 0x7fba14100750 2024-04-22T23:55:47.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.859+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba140611d0 con 0x7fba0006ecc0 2024-04-22T23:55:47.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.860+0000 7fba0b7fe700 1 -- 172.21.15.31:0/2467196570 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fba140611d0 con 0x7fba0006ecc0 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fba0006ecc0 msgr2=0x7fba00071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fba0006ecc0 0x7fba00071180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba10009c60 tx=0x7fba10009260 comp rx=0 tx=0).stop 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 msgr2=0x7fba14197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba14197ce0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fba040059d0 tx=0x7fba0400f690 comp rx=0 tx=0).stop 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 shutdown_connections 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fba0006ecc0 0x7fba00071180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba140ffda0 0x7fba141977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 --2- 172.21.15.31:0/2467196570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba14100750 0x7fba14197ce0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 >> 172.21.15.31:0/2467196570 conn(0x7fba140fb650 msgr2=0x7fba140fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 shutdown_connections 2024-04-22T23:55:47.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.863+0000 7fba1c3d9700 1 -- 172.21.15.31:0/2467196570 wait complete. 2024-04-22T23:55:47.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.973+0000 7f9bdf176700 1 -- 172.21.15.31:0/1291487250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 msgr2=0x7f9bd81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.973+0000 7f9bdf176700 1 --2- 172.21.15.31:0/1291487250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd81070b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009910 tx=0x7f9bcc009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:47.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.973+0000 7f9bdf176700 1 -- 172.21.15.31:0/1291487250 shutdown_connections 2024-04-22T23:55:47.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.973+0000 7f9bdf176700 1 --2- 172.21.15.31:0/1291487250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8068580 0x7f9bd80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.973+0000 7f9bdf176700 1 --2- 172.21.15.31:0/1291487250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd81070b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 -- 172.21.15.31:0/1291487250 >> 172.21.15.31:0/1291487250 conn(0x7f9bd8075670 msgr2=0x7f9bd8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 -- 172.21.15.31:0/1291487250 shutdown_connections 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 -- 172.21.15.31:0/1291487250 wait complete. 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 Processor -- start 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 -- start start 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8068580 0x7f9bd819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd819d230 con 0x7f9bd8106cd0 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.974+0000 7f9bdf176700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81966e0 con 0x7f9bd8068580 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56730/0 (socket says 172.21.15.31:56730) 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 -- 172.21.15.31:0/2886284566 learned_addr learned my addr 172.21.15.31:0/2886284566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bdcf12700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8068580 0x7f9bd819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 -- 172.21.15.31:0/2886284566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8068580 msgr2=0x7f9bd819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8068580 0x7f9bd819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:47.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 -- 172.21.15.31:0/2886284566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0095f0 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd7fff700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd819cba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400e920 tx=0x7f9bc400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc400c9d0 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc4010430 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc4018630 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.975+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd81969c0 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.976+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd8196f10 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.977+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd81091d0 con 0x7f9bd8106cd0 2024-04-22T23:55:47.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.977+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9bc40105a0 con 0x7f9bd8106cd0 2024-04-22T23:55:47.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.978+0000 7f9bd5ffb700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9bc806ecc0 0x7f9bc8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:47.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.978+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9bc4014070 con 0x7f9bd8106cd0 2024-04-22T23:55:47.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.978+0000 7f9bdcf12700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9bc806ecc0 0x7f9bc8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:47.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.978+0000 7f9bdcf12700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9bc806ecc0 0x7f9bc8071180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00b760 tx=0x7f9bcc005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:47.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:47.984+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9bc4054150 con 0x7f9bd8106cd0 2024-04-22T23:55:48.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:47 smithi031 ceph-mon[111289]: pgmap v234: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-04-22T23:55:48.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:48.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:47 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:48.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:47 smithi044 ceph-mon[82146]: pgmap v234: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-04-22T23:55:48.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:48.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:47 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.246+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bd8197180 con 0x7f9bc806ecc0 2024-04-22T23:55:48.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.259+0000 7f9bd5ffb700 1 -- 172.21.15.31:0/2886284566 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9bd8197180 con 0x7f9bc806ecc0 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 72s ago 18m 24.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 starting - - - - 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (3s) 0s ago 17m 6916k - 19.0.0-3252-g43be0201 a0434699c1d7 93233952bee9 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 72s ago 18m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (5m) 0s ago 17m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 72s ago 18m 81.3M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (95s) 72s ago 13m 13.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a7e757804d66 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (78s) 72s ago 13m 14.8M - 19.0.0-3252-g43be0201 a0434699c1d7 961938a7288c 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (55s) 0s ago 13m 16.2M - 19.0.0-3252-g43be0201 a0434699c1d7 053ad6c26779 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (30s) 0s ago 13m 12.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a5acb0d92e12 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (10m) 72s ago 20m 512M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (9m) 0s ago 17m 453M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:55:48.250 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (6m) 72s ago 20m 55.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (6m) 0s ago 17m 46.8M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 72s ago 18m 15.3M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (8m) 0s ago 17m 15.0M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 72s ago 16m 101M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 72s ago 16m 128M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (3m) 72s ago 15m 85.9M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (3m) 0s ago 15m 93.5M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (2m) 0s ago 15m 88.8M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (113s) 0s ago 14m 91.6M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 38dba54af5f6 2024-04-22T23:55:48.251 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (8m) 72s ago 18m 76.7M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9bc806ecc0 msgr2=0x7f9bc8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9bc806ecc0 0x7f9bc8071180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00b760 tx=0x7f9bcc005a90 comp rx=0 tx=0).stop 2024-04-22T23:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 msgr2=0x7f9bd819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd819cba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400e920 tx=0x7f9bc400ec30 comp rx=0 tx=0).stop 2024-04-22T23:55:48.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 shutdown_connections 2024-04-22T23:55:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9bc806ecc0 0x7f9bc8071180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8068580 0x7f9bd819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 --2- 172.21.15.31:0/2886284566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bd8106cd0 0x7f9bd819cba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 >> 172.21.15.31:0/2886284566 conn(0x7f9bd8075670 msgr2=0x7f9bd80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 shutdown_connections 2024-04-22T23:55:48.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.261+0000 7f9bdf176700 1 -- 172.21.15.31:0/2886284566 wait complete. 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 -- 172.21.15.31:0/1983896000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 msgr2=0x7f80fc1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 --2- 172.21.15.31:0/1983896000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc1021e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009910 tx=0x7f80ec009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 -- 172.21.15.31:0/1983896000 shutdown_connections 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 --2- 172.21.15.31:0/1983896000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80fc102720 0x7f80fc10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 --2- 172.21.15.31:0/1983896000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc1021e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 -- 172.21.15.31:0/1983896000 >> 172.21.15.31:0/1983896000 conn(0x7f80fc0fb650 msgr2=0x7f80fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 -- 172.21.15.31:0/1983896000 shutdown_connections 2024-04-22T23:55:48.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.370+0000 7f8102740700 1 -- 172.21.15.31:0/1983896000 wait complete. 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f8102740700 1 Processor -- start 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f8102740700 1 -- start start 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f8102740700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f8102740700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80fc102720 0x7f80fc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f8102740700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc19d260 con 0x7f80fc101e00 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f8102740700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc196710 con 0x7f80fc102720 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56746/0 (socket says 172.21.15.31:56746) 2024-04-22T23:55:48.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 -- 172.21.15.31:0/31399808 learned_addr learned my addr 172.21.15.31:0/31399808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80f3fff700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80fc102720 0x7f80fc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 -- 172.21.15.31:0/31399808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80fc102720 msgr2=0x7f80fc19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80fc102720 0x7f80fc19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 -- 172.21.15.31:0/31399808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec0095f0 con 0x7f80fc101e00 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.371+0000 7f80fbfff700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc19c690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80ec006010 tx=0x7f80ec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.372+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80ec01d070 con 0x7f80fc101e00 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.372+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80ec005dc0 con 0x7f80fc101e00 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.372+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc196990 con 0x7f80fc101e00 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.372+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80ec017400 con 0x7f80fc101e00 2024-04-22T23:55:48.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.372+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc196e80 con 0x7f80fc101e00 2024-04-22T23:55:48.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.373+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc108310 con 0x7f80fc101e00 2024-04-22T23:55:48.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.373+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f80ec003a60 con 0x7f80fc101e00 2024-04-22T23:55:48.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.374+0000 7f80f9ffb700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80e806ecc0 0x7f80e8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:48.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.374+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f80ec090370 con 0x7f80fc101e00 2024-04-22T23:55:48.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.374+0000 7f80f3fff700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80e806ecc0 0x7f80e8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:48.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.374+0000 7f80f3fff700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80e806ecc0 0x7f80e8071180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80fc197b80 tx=0x7f80e4007400 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:48.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.379+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f80ec05cb60 con 0x7f80fc101e00 2024-04-22T23:55:48.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.685+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80fc197cc0 con 0x7f80fc101e00 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.686+0000 7f80f9ffb700 1 -- 172.21.15.31:0/31399808 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f80ec05c6f0 con 0x7f80fc101e00 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:55:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:55:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:55:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:55:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:55:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:55:48.678 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:55:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80e806ecc0 msgr2=0x7f80e8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80e806ecc0 0x7f80e8071180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f80fc197b80 tx=0x7f80e4007400 comp rx=0 tx=0).stop 2024-04-22T23:55:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 msgr2=0x7f80fc19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc19c690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f80ec006010 tx=0x7f80ec00bd70 comp rx=0 tx=0).stop 2024-04-22T23:55:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 shutdown_connections 2024-04-22T23:55:48.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80e806ecc0 0x7f80e8071180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc101e00 0x7f80fc19c690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 --2- 172.21.15.31:0/31399808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80fc102720 0x7f80fc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 >> 172.21.15.31:0/31399808 conn(0x7f80fc0fb650 msgr2=0x7f80fc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 shutdown_connections 2024-04-22T23:55:48.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.688+0000 7f8102740700 1 -- 172.21.15.31:0/31399808 wait complete. 2024-04-22T23:55:48.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 -- 172.21.15.31:0/815235054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00101850 msgr2=0x7f2e001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 --2- 172.21.15.31:0/815235054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00101850 0x7f2e001058a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009910 tx=0x7f2df0009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 -- 172.21.15.31:0/815235054 shutdown_connections 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 --2- 172.21.15.31:0/815235054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00101850 0x7f2e001058a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 --2- 172.21.15.31:0/815235054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e00100ea0 0x7f2e00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 -- 172.21.15.31:0/815235054 >> 172.21.15.31:0/815235054 conn(0x7f2e000fc750 msgr2=0x7f2e000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 -- 172.21.15.31:0/815235054 shutdown_connections 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 -- 172.21.15.31:0/815235054 wait complete. 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.799+0000 7f2e05579700 1 Processor -- start 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2e05579700 1 -- start start 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2e05579700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 0x7f2e0019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2e05579700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e00101850 0x7f2e0019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2e05579700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0019d2d0 con 0x7f2e00100ea0 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2e05579700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e00196780 con 0x7f2e00101850 2024-04-22T23:55:48.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 0x7f2e0019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 0x7f2e0019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56756/0 (socket says 172.21.15.31:56756) 2024-04-22T23:55:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 -- 172.21.15.31:0/4003615405 learned_addr learned my addr 172.21.15.31:0/4003615405 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:48.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfe7fc700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e00101850 0x7f2e0019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 -- 172.21.15.31:0/4003615405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e00101850 msgr2=0x7f2e0019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e00101850 0x7f2e0019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 -- 172.21.15.31:0/4003615405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00095f0 con 0x7f2e00100ea0 2024-04-22T23:55:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.800+0000 7f2dfeffd700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 0x7f2e0019c700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d900 tx=0x7f2df400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.801+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2df40041f0 con 0x7f2e00100ea0 2024-04-22T23:55:48.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.801+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2df4004cf0 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.801+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e00196a60 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.801+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2df400b750 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.801+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e00196fb0 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.802+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e001092a0 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.802+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f2df4004750 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.802+0000 7f2de7fff700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2de806ecc0 0x7f2de8071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.803+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f2df408eed0 con 0x7f2e00100ea0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.803+0000 7f2dfe7fc700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2de806ecc0 0x7f2de8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:48.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.803+0000 7f2dfe7fc700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2de806ecc0 0x7f2de8071180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2e00197bf0 tx=0x7f2df001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:48.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:48.807+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2df405b640 con 0x7f2e00100ea0 2024-04-22T23:55:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:48 smithi031 ceph-mon[111289]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:48 smithi031 ceph-mon[111289]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:48 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:48 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:48.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:48 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/31399808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.082+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e0004f2e0 con 0x7f2e00100ea0 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.082+0000 7f2de7fff700 1 -- 172.21.15.31:0/4003615405 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1962 (secure 0 0 0) 0x7f2df405b1d0 con 0x7f2e00100ea0 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:e39 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 39 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:55:49.073 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:55:28.588901+0000 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 92 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34406} 2024-04-22T23:55:49.074 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34406 members: 34406 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/1426404610,v1:172.21.15.31:6829/1426404610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:49.075 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:55:49.076 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:55:49.076 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/4021256007,v1:172.21.15.44:6825/4021256007] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:49.076 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:44345} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6826/2995756226,v1:172.21.15.44:6827/2995756226] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:55:49.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2de806ecc0 msgr2=0x7f2de8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2de806ecc0 0x7f2de8071180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2e00197bf0 tx=0x7f2df001a040 comp rx=0 tx=0).stop 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 msgr2=0x7f2e0019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 0x7f2e0019c700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d900 tx=0x7f2df400dc10 comp rx=0 tx=0).stop 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 shutdown_connections 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f2de806ecc0 0x7f2de8071180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e00100ea0 0x7f2e0019c700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 --2- 172.21.15.31:0/4003615405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e00101850 0x7f2e0019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 >> 172.21.15.31:0/4003615405 conn(0x7f2e000fc750 msgr2=0x7f2e000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 shutdown_connections 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.084+0000 7f2e05579700 1 -- 172.21.15.31:0/4003615405 wait complete. 2024-04-22T23:55:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 39 2024-04-22T23:55:49.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:48 smithi044 ceph-mon[82146]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:49.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:48 smithi044 ceph-mon[82146]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:49.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:48 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:49.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:48 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:49.124 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:48 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/31399808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/2098174385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 msgr2=0x7f6b18105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/2098174385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18105fb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009910 tx=0x7f6b08009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/2098174385 shutdown_connections 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/2098174385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b180ffc20 0x7f6b18100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/2098174385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18105fb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/2098174385 >> 172.21.15.31:0/2098174385 conn(0x7f6b18075670 msgr2=0x7f6b18075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/2098174385 shutdown_connections 2024-04-22T23:55:49.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.194+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/2098174385 wait complete. 2024-04-22T23:55:49.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1cbcc700 1 Processor -- start 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1cbcc700 1 -- start start 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1cbcc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b180ffc20 0x7f6b18101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1cbcc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1cbcc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b181022c0 con 0x7f6b18105bd0 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1cbcc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18102400 con 0x7f6b180ffc20 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b0ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56768/0 (socket says 172.21.15.31:56768) 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b0ffff700 1 -- 172.21.15.31:0/3625124303 learned_addr learned my addr 172.21.15.31:0/3625124303 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b1659c700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b180ffc20 0x7f6b18101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b0ffff700 1 -- 172.21.15.31:0/3625124303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b180ffc20 msgr2=0x7f6b18101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.195+0000 7f6b0ffff700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b180ffc20 0x7f6b18101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b0ffff700 1 -- 172.21.15.31:0/3625124303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080095f0 con 0x7f6b18105bd0 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b0ffff700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18101d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000d8d0 tx=0x7f6b0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b000098a0 con 0x7f6b18105bd0 2024-04-22T23:55:49.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b00010460 con 0x7f6b18105bd0 2024-04-22T23:55:49.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b181a2a20 con 0x7f6b18105bd0 2024-04-22T23:55:49.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b0000f5f0 con 0x7f6b18105bd0 2024-04-22T23:55:49.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.196+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b181a2e30 con 0x7f6b18105bd0 2024-04-22T23:55:49.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.197+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1804f2e0 con 0x7f6b18105bd0 2024-04-22T23:55:49.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.197+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6b00009a00 con 0x7f6b18105bd0 2024-04-22T23:55:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.198+0000 7f6b0f7fe700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6b0406ecc0 0x7f6b04071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.198+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6b0008ef20 con 0x7f6b18105bd0 2024-04-22T23:55:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.198+0000 7f6b1659c700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6b0406ecc0 0x7f6b04071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:49.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.198+0000 7f6b1659c700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6b0406ecc0 0x7f6b04071180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006010 tx=0x7f6b0801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:49.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.203+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6b00057ac0 con 0x7f6b18105bd0 2024-04-22T23:55:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.451+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b18068d90 con 0x7f6b0406ecc0 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.452+0000 7f6b0f7fe700 1 -- 172.21.15.31:0/3625124303 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f6b18068d90 con 0x7f6b0406ecc0 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "mds", 2024-04-22T23:55:49.443 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "17/23 daemons upgraded", 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:55:49.444 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6b0406ecc0 msgr2=0x7f6b04071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6b0406ecc0 0x7f6b04071180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b08006010 tx=0x7f6b0801a040 comp rx=0 tx=0).stop 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 msgr2=0x7f6b18101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18101d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000d8d0 tx=0x7f6b0000dbe0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 shutdown_connections 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6b0406ecc0 0x7f6b04071180 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b180ffc20 0x7f6b18101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 --2- 172.21.15.31:0/3625124303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b18105bd0 0x7f6b18101d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 >> 172.21.15.31:0/3625124303 conn(0x7f6b18075670 msgr2=0x7f6b18068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 shutdown_connections 2024-04-22T23:55:49.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.455+0000 7f6b1cbcc700 1 -- 172.21.15.31:0/3625124303 wait complete. 2024-04-22T23:55:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 -- 172.21.15.31:0/3971754923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9950105bd0 msgr2=0x7f9950105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3971754923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9950105bd0 0x7f9950105fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9940009910 tx=0x7f9940009c20 comp rx=0 tx=0).stop 2024-04-22T23:55:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 -- 172.21.15.31:0/3971754923 shutdown_connections 2024-04-22T23:55:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3971754923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99500ffc20 0x7f9950100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3971754923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9950105bd0 0x7f9950105fb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 -- 172.21.15.31:0/3971754923 >> 172.21.15.31:0/3971754923 conn(0x7f9950075670 msgr2=0x7f9950075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 -- 172.21.15.31:0/3971754923 shutdown_connections 2024-04-22T23:55:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.575+0000 7f9955fc8700 1 -- 172.21.15.31:0/3971754923 wait complete. 2024-04-22T23:55:49.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f9955fc8700 1 Processor -- start 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f9955fc8700 1 -- start start 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f9955fc8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 0x7f995019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f9955fc8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950105bd0 0x7f995019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f9955fc8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995019d1f0 con 0x7f99500ffc20 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f9955fc8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99501966a0 con 0x7f9950105bd0 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f994f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 0x7f995019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f994f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 0x7f995019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56786/0 (socket says 172.21.15.31:56786) 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f994f7fe700 1 -- 172.21.15.31:0/3639232902 learned_addr learned my addr 172.21.15.31:0/3639232902 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.576+0000 7f994effd700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950105bd0 0x7f995019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994f7fe700 1 -- 172.21.15.31:0/3639232902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950105bd0 msgr2=0x7f995019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994f7fe700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950105bd0 0x7f995019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994f7fe700 1 -- 172.21.15.31:0/3639232902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400095f0 con 0x7f99500ffc20 2024-04-22T23:55:49.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994f7fe700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 0x7f995019c620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9940006010 tx=0x7f994000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994001d070 con 0x7f99500ffc20 2024-04-22T23:55:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f994000f460 con 0x7f99500ffc20 2024-04-22T23:55:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940017610 con 0x7f99500ffc20 2024-04-22T23:55:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9950196920 con 0x7f99500ffc20 2024-04-22T23:55:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.577+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9950196dc0 con 0x7f99500ffc20 2024-04-22T23:55:49.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.578+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995004f2e0 con 0x7f99500ffc20 2024-04-22T23:55:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.578+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f994000f5d0 con 0x7f99500ffc20 2024-04-22T23:55:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.579+0000 7f994cff9700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f993c06ecc0 0x7f993c071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:55:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.579+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9940090520 con 0x7f99500ffc20 2024-04-22T23:55:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.579+0000 7f994effd700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f993c06ecc0 0x7f993c071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:55:49.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.579+0000 7f994effd700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f993c06ecc0 0x7f993c071180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9950075bf0 tx=0x7f993800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:55:49.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.584+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f994005cc90 con 0x7f99500ffc20 2024-04-22T23:55:49.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.892+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9950197030 con 0x7f99500ffc20 2024-04-22T23:55:49.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.893+0000 7f994cff9700 1 -- 172.21.15.31:0/3639232902 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f994005c820 con 0x7f99500ffc20 2024-04-22T23:55:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:55:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:55:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f993c06ecc0 msgr2=0x7f993c071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f993c06ecc0 0x7f993c071180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9950075bf0 tx=0x7f993800a300 comp rx=0 tx=0).stop 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 msgr2=0x7f995019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 0x7f995019c620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9940006010 tx=0x7f994000bd70 comp rx=0 tx=0).stop 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 shutdown_connections 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f993c06ecc0 0x7f993c071180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99500ffc20 0x7f995019c620 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 --2- 172.21.15.31:0/3639232902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9950105bd0 0x7f995019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:55:49.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 >> 172.21.15.31:0/3639232902 conn(0x7f9950075670 msgr2=0x7f99500686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:55:49.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 shutdown_connections 2024-04-22T23:55:49.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:55:49.895+0000 7f9955fc8700 1 -- 172.21.15.31:0/3639232902 wait complete. 2024-04-22T23:55:50.150 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:49 smithi044 ceph-mon[82146]: pgmap v235: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 5.3 MiB/s rd, 2 op/s 2024-04-22T23:55:50.150 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:49 smithi044 ceph-mon[82146]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:50.150 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:49 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4003615405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:55:50.150 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:50.150 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:49 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:50.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:49 smithi031 ceph-mon[111289]: pgmap v235: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 5.3 MiB/s rd, 2 op/s 2024-04-22T23:55:50.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:49 smithi031 ceph-mon[111289]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:50.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:49 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4003615405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:55:50.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:50.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:49 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:51.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:50 smithi031 ceph-mon[111289]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:51.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:50 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3639232902' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:55:51.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:50 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:51.146 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:50 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:51.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:50 smithi044 ceph-mon[82146]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:55:51.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:50 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3639232902' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:55:51.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:50 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:51.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:50 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:51.891 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:51 smithi044 ceph-mon[82146]: pgmap v236: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2 op/s 2024-04-22T23:55:52.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:51 smithi031 ceph-mon[111289]: pgmap v236: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2 op/s 2024-04-22T23:55:53.396 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:52 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:53.396 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:52 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:53.396 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:52 smithi044 ceph-mon[82146]: pgmap v237: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1 op/s 2024-04-22T23:55:53.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:52 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:53.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:52 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:53.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:52 smithi031 ceph-mon[111289]: pgmap v237: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1 op/s 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: pgmap v238: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: Upgrade: Setting filesystem cephfs Joinable 2024-04-22T23:55:55.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:55 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: pgmap v238: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:55.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: Upgrade: Setting filesystem cephfs Joinable 2024-04-22T23:55:55.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:55 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-22T23:55:56.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all ceph-exporter 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"}]: dispatch 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"}]': finished 2024-04-22T23:55:56.694 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]': finished 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all iscsi 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all nfs 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: Upgrade: Setting container_image for all nvmeof 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.695 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: Upgrade: Finalizing container_image settings 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-22T23:55:56.696 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.697 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: Upgrade: Complete! 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:56.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: fsmap cephfs:1 {0=cephfs.smithi031.ezuwxk=up:active} 1 up:standby-replay 2 up:standby 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all ceph-exporter 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"}]': finished 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi044"}]': finished 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all iscsi 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all nfs 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: Upgrade: Setting container_image for all nvmeof 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.711 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: Upgrade: Finalizing container_image settings 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-22T23:55:56.712 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.713 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: Upgrade: Complete! 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:55:56.714 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:55:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:57 smithi031 ceph-mon[111289]: pgmap v239: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:55:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:57 smithi044 ceph-mon[82146]: pgmap v239: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:55:59.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:55:59 smithi031 ceph-mon[111289]: pgmap v240: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:55:59.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:55:59 smithi044 ceph-mon[82146]: pgmap v240: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:01.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:01 smithi031 ceph-mon[111289]: pgmap v241: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:01.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:01 smithi044 ceph-mon[82146]: pgmap v241: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:03.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:03 smithi031 ceph-mon[111289]: pgmap v242: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:03.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:03 smithi044 ceph-mon[82146]: pgmap v242: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:05.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:05 smithi031 ceph-mon[111289]: pgmap v243: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:05.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:05 smithi044 ceph-mon[82146]: pgmap v243: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:07.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:07 smithi031 ceph-mon[111289]: pgmap v244: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:07.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:07 smithi044 ceph-mon[82146]: pgmap v244: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:09.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:09 smithi031 ceph-mon[111289]: pgmap v245: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:09.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:09 smithi044 ceph-mon[82146]: pgmap v245: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:11.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:11 smithi031 ceph-mon[111289]: pgmap v246: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:11.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:11 smithi044 ceph-mon[82146]: pgmap v246: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:13.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:13 smithi031 ceph-mon[111289]: pgmap v247: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:13.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:13 smithi044 ceph-mon[82146]: pgmap v247: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:15.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:15 smithi031 ceph-mon[111289]: pgmap v248: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:15.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:15.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:15 smithi044 ceph-mon[82146]: pgmap v248: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:15.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:17.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:17 smithi031 ceph-mon[111289]: pgmap v249: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:17.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:17 smithi044 ceph-mon[82146]: pgmap v249: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:19.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:19 smithi044 ceph-mon[82146]: pgmap v250: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:20.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:19 smithi031 ceph-mon[111289]: pgmap v250: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:20.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/1839748719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 msgr2=0x7f5a5410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/1839748719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a5410ac10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a44009910 tx=0x7f5a44009c20 comp rx=0 tx=0).stop 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/1839748719 shutdown_connections 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/1839748719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a5410ac10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/1839748719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a541021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/1839748719 >> 172.21.15.31:0/1839748719 conn(0x7f5a540fb650 msgr2=0x7f5a540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/1839748719 shutdown_connections 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/1839748719 wait complete. 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.014+0000 7f5a5a4b1700 1 Processor -- start 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a5a4b1700 1 -- start start 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a5a4b1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a540ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a5a4b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a540fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:20.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a5a4b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a541004e0 con 0x7f5a54102720 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a5a4b1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a54100620 con 0x7f5a54101e00 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a540fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a540fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38862/0 (socket says 172.21.15.31:38862) 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a537fe700 1 -- 172.21.15.31:0/2671071985 learned_addr learned my addr 172.21.15.31:0/2671071985 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a53fff700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a540ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a537fe700 1 -- 172.21.15.31:0/2671071985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 msgr2=0x7f5a540ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a537fe700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a540ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a537fe700 1 -- 172.21.15.31:0/2671071985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440095f0 con 0x7f5a54102720 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.015+0000 7f5a53fff700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a540ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.016+0000 7f5a537fe700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a540fffa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400bd70 tx=0x7f5a4400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.016+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4401d070 con 0x7f5a54102720 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.016+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a4400f460 con 0x7f5a54102720 2024-04-22T23:56:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.016+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a541008a0 con 0x7f5a54102720 2024-04-22T23:56:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.016+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a44017630 con 0x7f5a54102720 2024-04-22T23:56:20.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.016+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a541a2d70 con 0x7f5a54102720 2024-04-22T23:56:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.017+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a54108310 con 0x7f5a54102720 2024-04-22T23:56:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.017+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5a44003a60 con 0x7f5a54102720 2024-04-22T23:56:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.018+0000 7f5a517fa700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5a4006ecc0 0x7f5a40071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.018+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5a44090470 con 0x7f5a54102720 2024-04-22T23:56:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.018+0000 7f5a53fff700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5a4006ecc0 0x7f5a40071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:20.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.018+0000 7f5a53fff700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5a4006ecc0 0x7f5a40071180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005fd0 tx=0x7f5a3c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:20.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.023+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5a44059130 con 0x7f5a54102720 2024-04-22T23:56:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.253+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a540611d0 con 0x7f5a4006ecc0 2024-04-22T23:56:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.254+0000 7f5a517fa700 1 -- 172.21.15.31:0/2671071985 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f5a540611d0 con 0x7f5a4006ecc0 2024-04-22T23:56:20.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5a4006ecc0 msgr2=0x7f5a40071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:20.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5a4006ecc0 0x7f5a40071180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c005fd0 tx=0x7f5a3c005e30 comp rx=0 tx=0).stop 2024-04-22T23:56:20.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 msgr2=0x7f5a540fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:20.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a540fffa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400bd70 tx=0x7f5a4400be50 comp rx=0 tx=0).stop 2024-04-22T23:56:20.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 shutdown_connections 2024-04-22T23:56:20.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5a4006ecc0 0x7f5a40071180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a54101e00 0x7f5a540ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 --2- 172.21.15.31:0/2671071985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a54102720 0x7f5a540fffa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 >> 172.21.15.31:0/2671071985 conn(0x7f5a540fb650 msgr2=0x7f5a54105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 shutdown_connections 2024-04-22T23:56:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:20.256+0000 7f5a5a4b1700 1 -- 172.21.15.31:0/2671071985 wait complete. 2024-04-22T23:56:20.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch ps' 2024-04-22T23:56:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:56:21.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:21 smithi031 ceph-mon[111289]: pgmap v251: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:21.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:21 smithi031 ceph-mon[111289]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:56:21.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:21 smithi044 ceph-mon[82146]: pgmap v251: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:21.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:21 smithi044 ceph-mon[82146]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:56:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 -- 172.21.15.31:0/4007324653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 msgr2=0x7f30a8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:22.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 --2- 172.21.15.31:0/4007324653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a8105890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3098009910 tx=0x7f3098009c20 comp rx=0 tx=0).stop 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 -- 172.21.15.31:0/4007324653 shutdown_connections 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 --2- 172.21.15.31:0/4007324653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a8105890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 --2- 172.21.15.31:0/4007324653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30a8100e90 0x7f30a8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 -- 172.21.15.31:0/4007324653 >> 172.21.15.31:0/4007324653 conn(0x7f30a80fc740 msgr2=0x7f30a80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 -- 172.21.15.31:0/4007324653 shutdown_connections 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.839+0000 7f30af7a0700 1 -- 172.21.15.31:0/4007324653 wait complete. 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30af7a0700 1 Processor -- start 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30af7a0700 1 -- start start 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30af7a0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30a8100e90 0x7f30a819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30af7a0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:22.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30af7a0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a819d2e0 con 0x7f30a8101840 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30af7a0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a8196790 con 0x7f30a8100e90 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41476/0 (socket says 172.21.15.31:41476) 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 -- 172.21.15.31:0/2654472099 learned_addr learned my addr 172.21.15.31:0/2654472099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30ad53c700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30a8100e90 0x7f30a819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 -- 172.21.15.31:0/2654472099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30a8100e90 msgr2=0x7f30a819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30a8100e90 0x7f30a819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 -- 172.21.15.31:0/2654472099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980095f0 con 0x7f30a8101840 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.840+0000 7f30acd3b700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a819cc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30980092e0 tx=0x7f309800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.841+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309801d070 con 0x7f30a8101840 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.841+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f309800f460 con 0x7f30a8101840 2024-04-22T23:56:22.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.841+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a8196a10 con 0x7f30a8101840 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.841+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30980175f0 con 0x7f30a8101840 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.841+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a8196f00 con 0x7f30a8101840 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.842+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a8109290 con 0x7f30a8101840 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.842+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f309800f5d0 con 0x7f30a8101840 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.843+0000 7f309e7fc700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f309406ecc0 0x7f3094071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.843+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f30980904f0 con 0x7f30a8101840 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.843+0000 7f30ad53c700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f309406ecc0 0x7f3094071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:22.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.843+0000 7f30ad53c700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f309406ecc0 0x7f3094071180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005fd0 tx=0x7f30a400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:22.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:22.850+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f309805cc40 con 0x7f30a8101840 2024-04-22T23:56:23.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.067+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a8197170 con 0x7f309406ecc0 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.078+0000 7f309e7fc700 1 -- 172.21.15.31:0/2654472099 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f30a8197170 con 0x7f309406ecc0 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 33s ago 19m 24.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (44s) 33s ago 19m 7390k - 19.0.0-3252-g43be0201 a0434699c1d7 185cd63b46ed 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (38s) 35s ago 18m 6916k - 19.0.0-3252-g43be0201 a0434699c1d7 93233952bee9 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 33s ago 19m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (5m) 35s ago 18m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (8m) 33s ago 18m 80.0M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:56:23.069 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (2m) 33s ago 13m 86.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a7e757804d66 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (112s) 33s ago 13m 192M - 19.0.0-3252-g43be0201 a0434699c1d7 961938a7288c 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (90s) 35s ago 13m 16.2M - 19.0.0-3252-g43be0201 a0434699c1d7 053ad6c26779 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (65s) 35s ago 13m 12.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a5acb0d92e12 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (10m) 33s ago 20m 514M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (9m) 35s ago 18m 453M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (7m) 33s ago 20m 59.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (6m) 35s ago 18m 46.8M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 33s ago 19m 15.3M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (9m) 35s ago 18m 15.0M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 33s ago 17m 105M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 33s ago 16m 114M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 33s ago 16m 95.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:56:23.070 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (3m) 35s ago 16m 93.5M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:56:23.071 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (3m) 35s ago 15m 88.8M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:56:23.071 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (2m) 35s ago 15m 91.6M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 38dba54af5f6 2024-04-22T23:56:23.071 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (8m) 33s ago 18m 78.3M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:56:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f309406ecc0 msgr2=0x7f3094071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f309406ecc0 0x7f3094071180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005fd0 tx=0x7f30a400a560 comp rx=0 tx=0).stop 2024-04-22T23:56:23.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 msgr2=0x7f30a819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a819cc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f30980092e0 tx=0x7f309800be20 comp rx=0 tx=0).stop 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 shutdown_connections 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f309406ecc0 0x7f3094071180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f30a8100e90 0x7f30a819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 --2- 172.21.15.31:0/2654472099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30a8101840 0x7f30a819cc50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 >> 172.21.15.31:0/2654472099 conn(0x7f30a80fc740 msgr2=0x7f30a80fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 shutdown_connections 2024-04-22T23:56:23.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:23.080+0000 7f30af7a0700 1 -- 172.21.15.31:0/2654472099 wait complete. 2024-04-22T23:56:23.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch upgrade status' 2024-04-22T23:56:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:56:23.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:23 smithi031 ceph-mon[111289]: pgmap v252: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:23.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:23 smithi044 ceph-mon[82146]: pgmap v252: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:24.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:24 smithi044 ceph-mon[82146]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:56:25.080 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:24 smithi031 ceph-mon[111289]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:56:25.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 -- 172.21.15.31:0/1869376455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8101850 msgr2=0x7ff4f81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:25.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 --2- 172.21.15.31:0/1869376455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8101850 0x7ff4f81058a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009910 tx=0x7ff4ec009c20 comp rx=0 tx=0).stop 2024-04-22T23:56:25.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 -- 172.21.15.31:0/1869376455 shutdown_connections 2024-04-22T23:56:25.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 --2- 172.21.15.31:0/1869376455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8101850 0x7ff4f81058a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 --2- 172.21.15.31:0/1869376455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 -- 172.21.15.31:0/1869376455 >> 172.21.15.31:0/1869376455 conn(0x7ff4f80fc750 msgr2=0x7ff4f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 -- 172.21.15.31:0/1869376455 shutdown_connections 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.610+0000 7ff4ff561700 1 -- 172.21.15.31:0/1869376455 wait complete. 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4ff561700 1 Processor -- start 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4ff561700 1 -- start start 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4ff561700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4ff561700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4f8101850 0x7ff4f80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4ff561700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f8073b10 con 0x7ff4f8100ea0 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4ff561700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f8073c50 con 0x7ff4f8101850 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.611+0000 7ff4fd2fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fd2fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41494/0 (socket says 172.21.15.31:41494) 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fd2fd700 1 -- 172.21.15.31:0/2255828721 learned_addr learned my addr 172.21.15.31:0/2255828721 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fcafc700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4f8101850 0x7ff4f80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:25.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fd2fd700 1 -- 172.21.15.31:0/2255828721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4f8101850 msgr2=0x7ff4f80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fd2fd700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4f8101850 0x7ff4f80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fd2fd700 1 -- 172.21.15.31:0/2255828721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0095f0 con 0x7ff4f8100ea0 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4fd2fd700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8073090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400b4b0 tx=0x7ff4f400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f4011800 con 0x7ff4f8100ea0 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f81a2b90 con 0x7ff4f8100ea0 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f4011e20 con 0x7ff4f8100ea0 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f400f2e0 con 0x7ff4f8100ea0 2024-04-22T23:56:25.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.612+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f81a2fa0 con 0x7ff4f8100ea0 2024-04-22T23:56:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.613+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f81091d0 con 0x7ff4f8100ea0 2024-04-22T23:56:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.614+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff4f4010380 con 0x7ff4f8100ea0 2024-04-22T23:56:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.614+0000 7ff4ea7fc700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff4e406ecc0 0x7ff4e4071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.614+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff4f408eb70 con 0x7ff4f8100ea0 2024-04-22T23:56:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.614+0000 7ff4fcafc700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff4e406ecc0 0x7ff4e4071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:25.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.615+0000 7ff4fcafc700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff4e406ecc0 0x7ff4e4071180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8074880 tx=0x7ff4ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:25.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.619+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff4f40577b0 con 0x7ff4f8100ea0 2024-04-22T23:56:25.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.849+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4f80611d0 con 0x7ff4e406ecc0 2024-04-22T23:56:25.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.850+0000 7ff4ea7fc700 1 -- 172.21.15.31:0/2255828721 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff4f80611d0 con 0x7ff4e406ecc0 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": null, 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": false, 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "", 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": null, 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-22T23:56:25.841 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff4e406ecc0 msgr2=0x7ff4e4071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff4e406ecc0 0x7ff4e4071180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8074880 tx=0x7ff4ec01a040 comp rx=0 tx=0).stop 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 msgr2=0x7ff4f8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8073090 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400b4b0 tx=0x7ff4f400b870 comp rx=0 tx=0).stop 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 shutdown_connections 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff4e406ecc0 0x7ff4e4071180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4f8100ea0 0x7ff4f8073090 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 --2- 172.21.15.31:0/2255828721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4f8101850 0x7ff4f80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 >> 172.21.15.31:0/2255828721 conn(0x7ff4f80fc750 msgr2=0x7ff4f80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 shutdown_connections 2024-04-22T23:56:25.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:25.852+0000 7ff4ff561700 1 -- 172.21.15.31:0/2255828721 wait complete. 2024-04-22T23:56:26.141 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:25 smithi031 ceph-mon[111289]: pgmap v253: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:26.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:25 smithi044 ceph-mon[82146]: pgmap v253: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:26.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph health detail' 2024-04-22T23:56:26.764 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:56:26.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:26 smithi031 ceph-mon[111289]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:56:27.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:26 smithi044 ceph-mon[82146]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:56:27.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:27 smithi031 ceph-mon[111289]: pgmap v254: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:28.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:27 smithi044 ceph-mon[82146]: pgmap v254: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 -- 172.21.15.31:0/23712845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364101850 msgr2=0x7ff3641058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 --2- 172.21.15.31:0/23712845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364101850 0x7ff3641058a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff354009910 tx=0x7ff354009c20 comp rx=0 tx=0).stop 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 -- 172.21.15.31:0/23712845 shutdown_connections 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 --2- 172.21.15.31:0/23712845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364101850 0x7ff3641058a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 --2- 172.21.15.31:0/23712845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff364100ea0 0x7ff364101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 -- 172.21.15.31:0/23712845 >> 172.21.15.31:0/23712845 conn(0x7ff3640fc750 msgr2=0x7ff3640feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 -- 172.21.15.31:0/23712845 shutdown_connections 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.437+0000 7ff36be8c700 1 -- 172.21.15.31:0/23712845 wait complete. 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff36be8c700 1 Processor -- start 2024-04-22T23:56:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff36be8c700 1 -- start start 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff36be8c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 0x7ff36419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff36be8c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff364101850 0x7ff36419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff36be8c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36419d2d0 con 0x7ff364100ea0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff36be8c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364196780 con 0x7ff364101850 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369c28700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 0x7ff36419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369c28700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 0x7ff36419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41502/0 (socket says 172.21.15.31:41502) 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369c28700 1 -- 172.21.15.31:0/2808718298 learned_addr learned my addr 172.21.15.31:0/2808718298 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369427700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff364101850 0x7ff36419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369c28700 1 -- 172.21.15.31:0/2808718298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff364101850 msgr2=0x7ff36419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369c28700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff364101850 0x7ff36419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.438+0000 7ff369c28700 1 -- 172.21.15.31:0/2808718298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540095f0 con 0x7ff364100ea0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.439+0000 7ff369c28700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 0x7ff36419c700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c8a0 tx=0x7ff36000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.439+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff36000cea0 con 0x7ff364100ea0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.439+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff360004830 con 0x7ff364100ea0 2024-04-22T23:56:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.439+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff364196a60 con 0x7ff364100ea0 2024-04-22T23:56:28.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.439+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3600055b0 con 0x7ff364100ea0 2024-04-22T23:56:28.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.439+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff364196fb0 con 0x7ff364100ea0 2024-04-22T23:56:28.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.440+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3641092a0 con 0x7ff364100ea0 2024-04-22T23:56:28.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.440+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff3600049a0 con 0x7ff364100ea0 2024-04-22T23:56:28.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.441+0000 7ff35affd700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff35006ecc0 0x7ff350071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:28.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.441+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff36008ebc0 con 0x7ff364100ea0 2024-04-22T23:56:28.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.441+0000 7ff369427700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff35006ecc0 0x7ff350071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:28.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.441+0000 7ff369427700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff35006ecc0 0x7ff350071180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff364197bf0 tx=0x7ff35401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.446+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff36005b3b0 con 0x7ff364100ea0 2024-04-22T23:56:28.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.745+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff364197280 con 0x7ff364100ea0 2024-04-22T23:56:28.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.745+0000 7ff35affd700 1 -- 172.21.15.31:0/2808718298 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ff36001d090 con 0x7ff364100ea0 2024-04-22T23:56:28.737 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-22T23:56:28.737 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-22T23:56:28.737 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff35006ecc0 msgr2=0x7ff350071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff35006ecc0 0x7ff350071180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff364197bf0 tx=0x7ff35401a040 comp rx=0 tx=0).stop 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 msgr2=0x7ff36419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 0x7ff36419c700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c8a0 tx=0x7ff36000cc60 comp rx=0 tx=0).stop 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 shutdown_connections 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff35006ecc0 0x7ff350071180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364100ea0 0x7ff36419c700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 --2- 172.21.15.31:0/2808718298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff364101850 0x7ff36419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 >> 172.21.15.31:0/2808718298 conn(0x7ff3640fc750 msgr2=0x7ff3640fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 shutdown_connections 2024-04-22T23:56:28.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:28.748+0000 7ff36be8c700 1 -- 172.21.15.31:0/2808718298 wait complete. 2024-04-22T23:56:29.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions' 2024-04-22T23:56:29.649 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:56:29.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:29 smithi031 ceph-mon[111289]: pgmap v255: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:29.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:29 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2808718298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:56:30.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:29 smithi044 ceph-mon[82146]: pgmap v255: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:30.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:29 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2808718298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:56:30.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:31.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 -- 172.21.15.31:0/2098205443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 msgr2=0x7f26901047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:31.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 --2- 172.21.15.31:0/2098205443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f26901047a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2684009910 tx=0x7f2684009c20 comp rx=0 tx=0).stop 2024-04-22T23:56:31.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 -- 172.21.15.31:0/2098205443 shutdown_connections 2024-04-22T23:56:31.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 --2- 172.21.15.31:0/2098205443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f26901047a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:31.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 --2- 172.21.15.31:0/2098205443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26900ffda0 0x7f2690100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:31.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 -- 172.21.15.31:0/2098205443 >> 172.21.15.31:0/2098205443 conn(0x7f26900fb650 msgr2=0x7f26900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 -- 172.21.15.31:0/2098205443 shutdown_connections 2024-04-22T23:56:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.187+0000 7f2696776700 1 -- 172.21.15.31:0/2098205443 wait complete. 2024-04-22T23:56:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f2696776700 1 Processor -- start 2024-04-22T23:56:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f2696776700 1 -- start start 2024-04-22T23:56:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f2696776700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26900ffda0 0x7f2690101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f2696776700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f2690101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:31.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f2696776700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26901022c0 con 0x7f2690100750 2024-04-22T23:56:31.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f2696776700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690102400 con 0x7f26900ffda0 2024-04-22T23:56:31.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f268f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f2690101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:31.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f268f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f2690101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40742/0 (socket says 172.21.15.31:40742) 2024-04-22T23:56:31.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f268f7fe700 1 -- 172.21.15.31:0/3269697312 learned_addr learned my addr 172.21.15.31:0/3269697312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f268ffff700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26900ffda0 0x7f2690101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f268f7fe700 1 -- 172.21.15.31:0/3269697312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26900ffda0 msgr2=0x7f2690101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.188+0000 7f268f7fe700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26900ffda0 0x7f2690101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f268f7fe700 1 -- 172.21.15.31:0/3269697312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840095f0 con 0x7f2690100750 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f268f7fe700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f2690101d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26840092e0 tx=0x7f268400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268401d070 con 0x7f2690100750 2024-04-22T23:56:31.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2690102680 con 0x7f2690100750 2024-04-22T23:56:31.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2684005dc0 con 0x7f2690100750 2024-04-22T23:56:31.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2684017400 con 0x7f2690100750 2024-04-22T23:56:31.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.189+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26901a2da0 con 0x7f2690100750 2024-04-22T23:56:31.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.190+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26901091b0 con 0x7f2690100750 2024-04-22T23:56:31.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.190+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f268400f6f0 con 0x7f2690100750 2024-04-22T23:56:31.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.191+0000 7f268d7fa700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f267806ecc0 0x7f2678071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:56:31.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.191+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f26840903c0 con 0x7f2690100750 2024-04-22T23:56:31.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.191+0000 7f268ffff700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f267806ecc0 0x7f2678071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:56:31.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.191+0000 7f268ffff700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f267806ecc0 0x7f2678071180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2680005950 tx=0x7f268000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:56:31.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.196+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2684059000 con 0x7f2690100750 2024-04-22T23:56:31.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.497+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2690108b90 con 0x7f2690100750 2024-04-22T23:56:31.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.498+0000 7f268d7fa700 1 -- 172.21.15.31:0/3269697312 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f268405c6c0 con 0x7f2690100750 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:56:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:56:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:56:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:56:31.491 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:56:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f267806ecc0 msgr2=0x7f2678071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f267806ecc0 0x7f2678071180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2680005950 tx=0x7f268000b410 comp rx=0 tx=0).stop 2024-04-22T23:56:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 msgr2=0x7f2690101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:56:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f2690101d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26840092e0 tx=0x7f268400bd70 comp rx=0 tx=0).stop 2024-04-22T23:56:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 shutdown_connections 2024-04-22T23:56:31.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f267806ecc0 0x7f2678071180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26900ffda0 0x7f2690101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 --2- 172.21.15.31:0/3269697312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100750 0x7f2690101d80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:56:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 >> 172.21.15.31:0/3269697312 conn(0x7f26900fb650 msgr2=0x7f2690069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:56:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 shutdown_connections 2024-04-22T23:56:31.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:56:31.500+0000 7f2696776700 1 -- 172.21.15.31:0/3269697312 wait complete. 2024-04-22T23:56:31.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:31 smithi031 ceph-mon[111289]: pgmap v256: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:31.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:31 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3269697312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:56:32.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:31 smithi044 ceph-mon[82146]: pgmap v256: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:32.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:31 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3269697312' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:56:32.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-22T23:56:32.450 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:56:33.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:33 smithi031 ceph-mon[111289]: pgmap v257: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:34.139 INFO:teuthology.orchestra.run.smithi031.stdout:wait for servicemap items w/ changing names to refresh 2024-04-22T23:56:34.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:33 smithi044 ceph-mon[82146]: pgmap v257: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:34.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'sleep 60' 2024-04-22T23:56:35.013 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:56:35.992 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:35 smithi031 ceph-mon[111289]: pgmap v258: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:36.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:35 smithi044 ceph-mon[82146]: pgmap v258: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:38.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:37 smithi031 ceph-mon[111289]: pgmap v259: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:38.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:37 smithi044 ceph-mon[82146]: pgmap v259: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:39.994 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:39 smithi031 ceph-mon[111289]: pgmap v260: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:40.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:39 smithi044 ceph-mon[82146]: pgmap v260: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:42.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:41 smithi031 ceph-mon[111289]: pgmap v261: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:42.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:41 smithi044 ceph-mon[82146]: pgmap v261: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:44.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:43 smithi031 ceph-mon[111289]: pgmap v262: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:44.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:43 smithi044 ceph-mon[82146]: pgmap v262: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:45 smithi031 ceph-mon[111289]: pgmap v263: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:46.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:46.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:45 smithi044 ceph-mon[82146]: pgmap v263: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:46.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:56:48.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:47 smithi031 ceph-mon[111289]: pgmap v264: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:48.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:47 smithi044 ceph-mon[82146]: pgmap v264: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:50.116 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:49 smithi031 ceph-mon[111289]: pgmap v265: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:50.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:49 smithi044 ceph-mon[82146]: pgmap v265: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:52.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:51 smithi031 ceph-mon[111289]: pgmap v266: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:52.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:51 smithi044 ceph-mon[82146]: pgmap v266: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:54.011 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:53 smithi044 ceph-mon[82146]: pgmap v267: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:54.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:53 smithi031 ceph-mon[111289]: pgmap v267: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:56:56.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:55 smithi031 ceph-mon[111289]: pgmap v268: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:56.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:55 smithi044 ceph-mon[82146]: pgmap v268: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:56:57.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:56:57.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:57.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:56 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:57.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:56:57.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:57.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:56 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:57 smithi031 ceph-mon[111289]: pgmap v269: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:56:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:56:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:56:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:57 smithi044 ceph-mon[82146]: pgmap v269: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:56:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:56:58.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:56:58.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:56:59 smithi031 ceph-mon[111289]: pgmap v270: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:00.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:56:59 smithi044 ceph-mon[82146]: pgmap v270: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:01.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:02.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:01 smithi031 ceph-mon[111289]: pgmap v271: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:02.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:01 smithi044 ceph-mon[82146]: pgmap v271: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:04.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:03 smithi031 ceph-mon[111289]: pgmap v272: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:04.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:03 smithi044 ceph-mon[82146]: pgmap v272: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:05.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:05 smithi044 ceph-mon[82146]: pgmap v273: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:06.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:05 smithi031 ceph-mon[111289]: pgmap v273: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:08.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:07 smithi031 ceph-mon[111289]: pgmap v274: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:08.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:07 smithi044 ceph-mon[82146]: pgmap v274: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:10.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:09 smithi031 ceph-mon[111289]: pgmap v275: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:10.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:09 smithi044 ceph-mon[82146]: pgmap v275: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:11 smithi031 ceph-mon[111289]: pgmap v276: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:12.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:11 smithi044 ceph-mon[82146]: pgmap v276: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:14.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:13 smithi031 ceph-mon[111289]: pgmap v277: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:14.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:13 smithi044 ceph-mon[82146]: pgmap v277: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:15.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:15 smithi044 ceph-mon[82146]: pgmap v278: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:15.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:15 smithi031 ceph-mon[111289]: pgmap v278: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:18.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:17 smithi031 ceph-mon[111289]: pgmap v279: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:18.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:17 smithi044 ceph-mon[82146]: pgmap v279: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:20.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:19 smithi031 ceph-mon[111289]: pgmap v280: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:20.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:19 smithi044 ceph-mon[82146]: pgmap v280: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:21 smithi031 ceph-mon[111289]: pgmap v281: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:21 smithi044 ceph-mon[82146]: pgmap v281: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:24.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:23 smithi031 ceph-mon[111289]: pgmap v282: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:24.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:23 smithi044 ceph-mon[82146]: pgmap v282: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:26.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:25 smithi031 ceph-mon[111289]: pgmap v283: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:26.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:25 smithi044 ceph-mon[82146]: pgmap v283: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:28.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:27 smithi031 ceph-mon[111289]: pgmap v284: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:28.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:27 smithi044 ceph-mon[82146]: pgmap v284: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:30.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:29 smithi031 ceph-mon[111289]: pgmap v285: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:30.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:29 smithi044 ceph-mon[82146]: pgmap v285: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:30.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:31.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:32.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:31 smithi031 ceph-mon[111289]: pgmap v286: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:32.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:31 smithi044 ceph-mon[82146]: pgmap v286: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:34.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:33 smithi031 ceph-mon[111289]: pgmap v287: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:34.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:33 smithi044 ceph-mon[82146]: pgmap v287: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:35.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:35 smithi031 ceph-mon[111289]: pgmap v288: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:36.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:35 smithi044 ceph-mon[82146]: pgmap v288: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:37.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch ps' 2024-04-22T23:57:37.335 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:57:38.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:37 smithi031 ceph-mon[111289]: pgmap v289: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:38.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:37 smithi044 ceph-mon[82146]: pgmap v289: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 -- 172.21.15.31:0/3870839816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 msgr2=0x7f88740ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 --2- 172.21.15.31:0/3870839816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f88740ffd70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8868009910 tx=0x7f8868009c20 comp rx=0 tx=0).stop 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 -- 172.21.15.31:0/3870839816 shutdown_connections 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 --2- 172.21.15.31:0/3870839816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8874100340 0x7f887410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 --2- 172.21.15.31:0/3870839816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f88740ffd70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 -- 172.21.15.31:0/3870839816 >> 172.21.15.31:0/3870839816 conn(0x7f88740fb650 msgr2=0x7f88740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 -- 172.21.15.31:0/3870839816 shutdown_connections 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.990+0000 7f887b020700 1 -- 172.21.15.31:0/3870839816 wait complete. 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f887b020700 1 Processor -- start 2024-04-22T23:57:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f887b020700 1 -- start start 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f887b020700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f887419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f887b020700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8874100340 0x7f887419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f887b020700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887419f520 con 0x7f88740ff990 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f887b020700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88741989d0 con 0x7f8874100340 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f887419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f887419e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48900/0 (socket says 172.21.15.31:48900) 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 -- 172.21.15.31:0/3770054920 learned_addr learned my addr 172.21.15.31:0/3770054920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8873fff700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8874100340 0x7f887419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 -- 172.21.15.31:0/3770054920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8874100340 msgr2=0x7f887419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8874100340 0x7f887419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 -- 172.21.15.31:0/3770054920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88680095f0 con 0x7f88740ff990 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.991+0000 7f8878dbc700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f887419e950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8868000c00 tx=0x7f88680056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.992+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f886801d070 con 0x7f88740ff990 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.992+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f886800f460 con 0x7f88740ff990 2024-04-22T23:57:38.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.992+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8874198c50 con 0x7f88740ff990 2024-04-22T23:57:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.992+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8868021640 con 0x7f88740ff990 2024-04-22T23:57:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.992+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88741991a0 con 0x7f88740ff990 2024-04-22T23:57:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.993+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887410a770 con 0x7f88740ff990 2024-04-22T23:57:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.993+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f886800f5d0 con 0x7f88740ff990 2024-04-22T23:57:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.994+0000 7f8871ffb700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f886406ecc0 0x7f8864071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.994+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f88680902e0 con 0x7f88740ff990 2024-04-22T23:57:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.994+0000 7f8873fff700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f886406ecc0 0x7f8864071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.994+0000 7f8873fff700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f886406ecc0 0x7f8864071180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8860006fd0 tx=0x7f8860008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:38.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:38.999+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f886805ca30 con 0x7f88740ff990 2024-04-22T23:57:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.219+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 --> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8874199f30 con 0x7f886406ecc0 2024-04-22T23:57:39.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.230+0000 7f8871ffb700 1 -- 172.21.15.31:0/3770054920 <== mgr.34104 v2:172.21.15.31:6800/2602450284 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8874199f30 con 0x7f886406ecc0 2024-04-22T23:57:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:57:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (9m) 109s ago 20m 24.7M - 0.25.0 c8568f914cd2 146fbbc39e2f 2024-04-22T23:57:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (2m) 109s ago 20m 7390k - 19.0.0-3252-g43be0201 a0434699c1d7 185cd63b46ed 2024-04-22T23:57:39.221 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi044 smithi044 running (114s) 111s ago 19m 6916k - 19.0.0-3252-g43be0201 a0434699c1d7 93233952bee9 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 109s ago 20m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 72754665a070 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi044 smithi044 running (7m) 111s ago 19m 6665k - 19.0.0-3252-g43be0201 a0434699c1d7 b829f27c77bb 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (9m) 109s ago 20m 80.0M - 9.4.12 2bacad6d85d8 cc50b68250d3 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.ezuwxk smithi031 running (3m) 109s ago 15m 86.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a7e757804d66 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.usiyoa smithi031 running (3m) 109s ago 15m 192M - 19.0.0-3252-g43be0201 a0434699c1d7 961938a7288c 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.ibkztg smithi044 running (2m) 111s ago 15m 16.2M - 19.0.0-3252-g43be0201 a0434699c1d7 053ad6c26779 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi044.onzoro smithi044 running (2m) 111s ago 14m 12.6M - 19.0.0-3252-g43be0201 a0434699c1d7 a5acb0d92e12 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.fazdcv smithi031 *:8443,9283,8765 running (12m) 109s ago 21m 514M - 19.0.0-3252-g43be0201 a0434699c1d7 e51608bde021 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi044.iglcti smithi044 *:8443,9283,8765 running (11m) 111s ago 19m 453M - 19.0.0-3252-g43be0201 a0434699c1d7 8073cc411125 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 109s ago 22m 59.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 82d8b2207da3 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi044 smithi044 running (7m) 111s ago 19m 46.8M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 7af6a8053287 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (10m) 109s ago 20m 15.3M - 1.5.0 0da6a335fe13 d425212b963d 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi044 smithi044 *:9100 running (10m) 111s ago 19m 15.0M - 1.5.0 0da6a335fe13 fd21adcee2be 2024-04-22T23:57:39.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 109s ago 18m 105M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 95e2a9c1a50a 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 109s ago 18m 114M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 fff5512a8954 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 109s ago 17m 95.4M 1217M 19.0.0-3252-g43be0201 a0434699c1d7 2cb66af4fede 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi044 running (4m) 111s ago 17m 93.5M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 bec8ae29bf31 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi044 running (4m) 111s ago 16m 88.8M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 26f6a1ebe5e5 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi044 running (3m) 111s ago 16m 91.6M 2241M 19.0.0-3252-g43be0201 a0434699c1d7 38dba54af5f6 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (10m) 109s ago 20m 78.3M - 2.43.0 a07b618ecd1d 6845c1285a33 2024-04-22T23:57:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f886406ecc0 msgr2=0x7f8864071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f886406ecc0 0x7f8864071180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8860006fd0 tx=0x7f8860008040 comp rx=0 tx=0).stop 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 msgr2=0x7f887419e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f887419e950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8868000c00 tx=0x7f88680056a0 comp rx=0 tx=0).stop 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 shutdown_connections 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f886406ecc0 0x7f8864071180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88740ff990 0x7f887419e950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 --2- 172.21.15.31:0/3770054920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8874100340 0x7f887419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 >> 172.21.15.31:0/3770054920 conn(0x7f88740fb650 msgr2=0x7f88740fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 shutdown_connections 2024-04-22T23:57:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:39.233+0000 7f887b020700 1 -- 172.21.15.31:0/3770054920 wait complete. 2024-04-22T23:57:39.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions' 2024-04-22T23:57:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:57:40.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:39 smithi031 ceph-mon[111289]: pgmap v290: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:40.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:39 smithi044 ceph-mon[82146]: pgmap v290: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:41.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:40 smithi031 ceph-mon[111289]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:57:41.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:40 smithi044 ceph-mon[82146]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:57:42.037 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:41 smithi031 ceph-mon[111289]: pgmap v291: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 -- 172.21.15.31:0/2166570819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70400ff410 msgr2=0x7f70400ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 --2- 172.21.15.31:0/2166570819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70400ff410 0x7f70400ff890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7034009910 tx=0x7f7034009c20 comp rx=0 tx=0).stop 2024-04-22T23:57:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 -- 172.21.15.31:0/2166570819 shutdown_connections 2024-04-22T23:57:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 --2- 172.21.15.31:0/2166570819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70400ff410 0x7f70400ff890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 --2- 172.21.15.31:0/2166570819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7040106db0 0x7f7040107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 -- 172.21.15.31:0/2166570819 >> 172.21.15.31:0/2166570819 conn(0x7f7040074e30 msgr2=0x7f7040075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 -- 172.21.15.31:0/2166570819 shutdown_connections 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 -- 172.21.15.31:0/2166570819 wait complete. 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.020+0000 7f7044fa3700 1 Processor -- start 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f7044fa3700 1 -- start start 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f7044fa3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70400ff410 0x7f70401005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f7044fa3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 0x7f7040100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f7044fa3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040101100 con 0x7f7040106db0 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f7044fa3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040101270 con 0x7f70400ff410 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 0x7f7040100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 0x7f7040100b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52206/0 (socket says 172.21.15.31:52206) 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 -- 172.21.15.31:0/365882913 learned_addr learned my addr 172.21.15.31:0/365882913 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703e59c700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70400ff410 0x7f70401005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 -- 172.21.15.31:0/365882913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70400ff410 msgr2=0x7f70401005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70400ff410 0x7f70401005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 -- 172.21.15.31:0/365882913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70340095f0 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.021+0000 7f703dd9b700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 0x7f7040100b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f703400b5c0 tx=0x7f703400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.022+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703401d070 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.022+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70401a2b10 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.022+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7034005e00 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.022+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7034017400 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.022+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70401a2ec0 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.023+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70401092b0 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.023+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f703400f460 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.024+0000 7f702f7fe700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f702806ecc0 0x7f7028071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.024+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f7034090530 con 0x7f7040106db0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.024+0000 7f703e59c700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f702806ecc0 0x7f7028071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.024+0000 7f703e59c700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f702806ecc0 0x7f7028071180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7030005950 tx=0x7f70300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:42.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.029+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7034059170 con 0x7f7040106db0 2024-04-22T23:57:42.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:41 smithi044 ceph-mon[82146]: pgmap v291: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:42.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.327+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f704004f2e0 con 0x7f7040106db0 2024-04-22T23:57:42.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.328+0000 7f702f7fe700 1 -- 172.21.15.31:0/365882913 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f703405c830 con 0x7f7040106db0 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:57:42.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f702806ecc0 msgr2=0x7f7028071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f702806ecc0 0x7f7028071180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7030005950 tx=0x7f70300058e0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 msgr2=0x7f7040100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 0x7f7040100b30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f703400b5c0 tx=0x7f703400be20 comp rx=0 tx=0).stop 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 shutdown_connections 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f702806ecc0 0x7f7028071180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70400ff410 0x7f70401005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 --2- 172.21.15.31:0/365882913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7040106db0 0x7f7040100b30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 >> 172.21.15.31:0/365882913 conn(0x7f7040074e30 msgr2=0x7f70400fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 shutdown_connections 2024-04-22T23:57:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:42.330+0000 7f7044fa3700 1 -- 172.21.15.31:0/365882913 wait complete. 2024-04-22T23:57:42.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:42 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/365882913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:57:42.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-22T23:57:43.124 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:57:43.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:42 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/365882913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:57:44.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:43 smithi031 ceph-mon[111289]: pgmap v292: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:44.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:43 smithi044 ceph-mon[82146]: pgmap v292: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.855+0000 7f20fda6f700 1 -- 172.21.15.31:0/2554683959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 msgr2=0x7f20f8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.855+0000 7f20fda6f700 1 --2- 172.21.15.31:0/2554683959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8105fd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009910 tx=0x7f20e8009c20 comp rx=0 tx=0).stop 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.855+0000 7f20fda6f700 1 -- 172.21.15.31:0/2554683959 shutdown_connections 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.855+0000 7f20fda6f700 1 --2- 172.21.15.31:0/2554683959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 0x7f20f80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.856+0000 7f20fda6f700 1 --2- 172.21.15.31:0/2554683959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8105fd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.856+0000 7f20fda6f700 1 -- 172.21.15.31:0/2554683959 >> 172.21.15.31:0/2554683959 conn(0x7f20f80fb650 msgr2=0x7f20f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.856+0000 7f20fda6f700 1 -- 172.21.15.31:0/2554683959 shutdown_connections 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.856+0000 7f20fda6f700 1 -- 172.21.15.31:0/2554683959 wait complete. 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.856+0000 7f20fda6f700 1 Processor -- start 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.856+0000 7f20fda6f700 1 -- start start 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20fda6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 0x7f20f8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20fda6f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:44.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20fda6f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81982e0 con 0x7f20f8105bf0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20fda6f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8198420 con 0x7f20f80ffb40 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52222/0 (socket says 172.21.15.31:52222) 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 -- 172.21.15.31:0/935159145 learned_addr learned my addr 172.21.15.31:0/935159145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f77fe700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 0x7f20f8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 -- 172.21.15.31:0/935159145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 msgr2=0x7f20f8197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 0x7f20f8197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 -- 172.21.15.31:0/935159145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80095f0 con 0x7f20f8105bf0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f77fe700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 0x7f20f8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.857+0000 7f20f6ffd700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8197cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00b4b0 tx=0x7f20ec00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.858+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20ec011800 con 0x7f20f8105bf0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.858+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81a07f0 con 0x7f20f8105bf0 2024-04-22T23:57:44.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.858+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20ec011e20 con 0x7f20f8105bf0 2024-04-22T23:57:44.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.858+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20ec00f2e0 con 0x7f20f8105bf0 2024-04-22T23:57:44.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.858+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f81a0cb0 con 0x7f20f8105bf0 2024-04-22T23:57:44.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.859+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f8108020 con 0x7f20f8105bf0 2024-04-22T23:57:44.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.859+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f20ec010380 con 0x7f20f8105bf0 2024-04-22T23:57:44.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.860+0000 7f20f4ff9700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f20e006ecc0 0x7f20e0071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:44.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.860+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f20ec08ebc0 con 0x7f20f8105bf0 2024-04-22T23:57:44.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.860+0000 7f20f77fe700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f20e006ecc0 0x7f20e0071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:44.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.860+0000 7f20f77fe700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f20e006ecc0 0x7f20e0071180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005100 tx=0x7f20e801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:44.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:44.865+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f20ec057800 con 0x7f20f8105bf0 2024-04-22T23:57:45.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.162+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f20f81985e0 con 0x7f20f8105bf0 2024-04-22T23:57:45.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.163+0000 7f20f4ff9700 1 -- 172.21.15.31:0/935159145 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f20ec05aec0 con 0x7f20f8105bf0 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f20e006ecc0 msgr2=0x7f20e0071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f20e006ecc0 0x7f20e0071180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005100 tx=0x7f20e801a040 comp rx=0 tx=0).stop 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 msgr2=0x7f20f8197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8197cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00b4b0 tx=0x7f20ec00b870 comp rx=0 tx=0).stop 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 shutdown_connections 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f20e006ecc0 0x7f20e0071180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20f80ffb40 0x7f20f8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 --2- 172.21.15.31:0/935159145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f8105bf0 0x7f20f8197cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:45.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 >> 172.21.15.31:0/935159145 conn(0x7f20f80fb650 msgr2=0x7f20f80fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 shutdown_connections 2024-04-22T23:57:45.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:45.165+0000 7f20fda6f700 1 -- 172.21.15.31:0/935159145 wait complete. 2024-04-22T23:57:45.167 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-22T23:57:45.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-22T23:57:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:57:46.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:45 smithi031 ceph-mon[111289]: pgmap v293: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:46.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:46.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:45 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/935159145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:57:46.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:45 smithi044 ceph-mon[82146]: pgmap v293: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:46.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:57:46.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:45 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/935159145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 -- 172.21.15.31:0/2542372912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 msgr2=0x7f74940ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 --2- 172.21.15.31:0/2542372912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f74940ff9e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7490009910 tx=0x7f7490009c20 comp rx=0 tx=0).stop 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 -- 172.21.15.31:0/2542372912 shutdown_connections 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 --2- 172.21.15.31:0/2542372912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74940fffb0 0x7f74941058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 --2- 172.21.15.31:0/2542372912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f74940ff9e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 -- 172.21.15.31:0/2542372912 >> 172.21.15.31:0/2542372912 conn(0x7f7494074df0 msgr2=0x7f7494075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 -- 172.21.15.31:0/2542372912 shutdown_connections 2024-04-22T23:57:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 -- 172.21.15.31:0/2542372912 wait complete. 2024-04-22T23:57:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.554+0000 7f749c367700 1 Processor -- start 2024-04-22T23:57:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749c367700 1 -- start start 2024-04-22T23:57:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749c367700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f749419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749c367700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74940fffb0 0x7f749419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749c367700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749419d2c0 con 0x7f74940ff600 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749c367700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494196770 con 0x7f74940fffb0 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f749419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f749419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52238/0 (socket says 172.21.15.31:52238) 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 -- 172.21.15.31:0/2768672445 learned_addr learned my addr 172.21.15.31:0/2768672445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f7499902700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74940fffb0 0x7f749419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 -- 172.21.15.31:0/2768672445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74940fffb0 msgr2=0x7f749419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74940fffb0 0x7f749419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 -- 172.21.15.31:0/2768672445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74900095f0 con 0x7f74940ff600 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.555+0000 7f749a103700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f749419c6f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f749000f690 tx=0x7f749000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.556+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749001d070 con 0x7f74940ff600 2024-04-22T23:57:47.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.556+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74900050d0 con 0x7f74940ff600 2024-04-22T23:57:47.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.556+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74941969f0 con 0x7f74940ff600 2024-04-22T23:57:47.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.556+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7490022470 con 0x7f74940ff600 2024-04-22T23:57:47.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.556+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7494196ee0 con 0x7f74940ff600 2024-04-22T23:57:47.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.557+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7494109270 con 0x7f74940ff600 2024-04-22T23:57:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.557+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f7490003680 con 0x7f74940ff600 2024-04-22T23:57:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.558+0000 7f748b7fe700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f748006ecc0 0x7f7480071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.558+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f74900903a0 con 0x7f74940ff600 2024-04-22T23:57:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.558+0000 7f7499902700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f748006ecc0 0x7f7480071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.558+0000 7f7499902700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f748006ecc0 0x7f7480071180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7494197be0 tx=0x7f7484009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:47.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.563+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f749005cb90 con 0x7f74940ff600 2024-04-22T23:57:47.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.859+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7494108c50 con 0x7f74940ff600 2024-04-22T23:57:47.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.859+0000 7f748b7fe700 1 -- 172.21.15.31:0/2768672445 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f749005c720 con 0x7f74940ff600 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f748006ecc0 msgr2=0x7f7480071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f748006ecc0 0x7f7480071180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7494197be0 tx=0x7f7484009260 comp rx=0 tx=0).stop 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 msgr2=0x7f749419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f749419c6f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f749000f690 tx=0x7f749000f770 comp rx=0 tx=0).stop 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 shutdown_connections 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f748006ecc0 0x7f7480071180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:47.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74940ff600 0x7f749419c6f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:47.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.861+0000 7f749c367700 1 --2- 172.21.15.31:0/2768672445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74940fffb0 0x7f749419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:47.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.862+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 >> 172.21.15.31:0/2768672445 conn(0x7f7494074df0 msgr2=0x7f74940fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:47.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.862+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 shutdown_connections 2024-04-22T23:57:47.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:47.862+0000 7f749c367700 1 -- 172.21.15.31:0/2768672445 wait complete. 2024-04-22T23:57:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)" 2024-04-22T23:57:48.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:47 smithi031 ceph-mon[111289]: pgmap v294: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:48.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:47 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2768672445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:57:48.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:47 smithi044 ceph-mon[82146]: pgmap v294: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:48.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:47 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2768672445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:57:48.527 DEBUG:teuthology.parallel:result is None 2024-04-22T23:57:48.528 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:57:48.539 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-22T23:57:48.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid efa92c84-0100-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T23:57:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-22T23:57:50.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:49 smithi031 ceph-mon[111289]: pgmap v295: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:50.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:49 smithi044 ceph-mon[82146]: pgmap v295: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 -- 172.21.15.31:0/3157522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 msgr2=0x7f6a781058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 --2- 172.21.15.31:0/3157522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a781058a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009910 tx=0x7f6a68009c20 comp rx=0 tx=0).stop 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 -- 172.21.15.31:0/3157522720 shutdown_connections 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 --2- 172.21.15.31:0/3157522720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a781058a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 --2- 172.21.15.31:0/3157522720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a78100ea0 0x7f6a78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 -- 172.21.15.31:0/3157522720 >> 172.21.15.31:0/3157522720 conn(0x7f6a780fc750 msgr2=0x7f6a780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 -- 172.21.15.31:0/3157522720 shutdown_connections 2024-04-22T23:57:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 -- 172.21.15.31:0/3157522720 wait complete. 2024-04-22T23:57:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 Processor -- start 2024-04-22T23:57:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.368+0000 7f6a7f518700 1 -- start start 2024-04-22T23:57:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7f518700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a78100ea0 0x7f6a7819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7f518700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a7819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7f518700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819d240 con 0x7f6a78101850 2024-04-22T23:57:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7f518700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78196780 con 0x7f6a78100ea0 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a7819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a7819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52260/0 (socket says 172.21.15.31:52260) 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 -- 172.21.15.31:0/1969074408 learned_addr learned my addr 172.21.15.31:0/1969074408 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7d2b4700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a78100ea0 0x7f6a7819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 -- 172.21.15.31:0/1969074408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a78100ea0 msgr2=0x7f6a7819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a78100ea0 0x7f6a7819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 -- 172.21.15.31:0/1969074408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680095f0 con 0x7f6a78101850 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.369+0000 7f6a7cab3700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a7819cc40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800bda0 tx=0x7f6a6800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.370+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6801d070 con 0x7f6a78101850 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.370+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a78196a00 con 0x7f6a78101850 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.370+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a68004fb0 con 0x7f6a78101850 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.370+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6800f480 con 0x7f6a78101850 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.370+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a78196ef0 con 0x7f6a78101850 2024-04-22T23:57:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.371+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a781092a0 con 0x7f6a78101850 2024-04-22T23:57:50.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.371+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6a68022870 con 0x7f6a78101850 2024-04-22T23:57:50.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.371+0000 7f6a6e7fc700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6a6406ecc0 0x7f6a64071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:50.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.372+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6a68090560 con 0x7f6a78101850 2024-04-22T23:57:50.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.372+0000 7f6a7d2b4700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6a6406ecc0 0x7f6a64071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:50.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.372+0000 7f6a7d2b4700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6a6406ecc0 0x7f6a64071180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a74005ea0 tx=0x7f6a74005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:50.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.376+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6a6805cd80 con 0x7f6a78101850 2024-04-22T23:57:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.637+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6a7804f2e0 con 0x7f6a78101850 2024-04-22T23:57:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.638+0000 7f6a6e7fc700 1 -- 172.21.15.31:0/1969074408 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1962 (secure 0 0 0) 0x7f6a6805c910 con 0x7f6a78101850 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:e40 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 40 2024-04-22T23:57:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-22T23:42:26.948183+0000 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-22T23:55:54.549878+0000 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 92 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:57:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34406} 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data enabled 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34406 members: 34406 2024-04-22T23:57:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.ezuwxk{0:34406} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.31:6826/3446954000,v1:172.21.15.31:6827/3446954000] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.usiyoa{0:34410} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/1426404610,v1:172.21.15.31:6829/1426404610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.ibkztg{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6824/4021256007,v1:172.21.15.44:6825/4021256007] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:57:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi044.onzoro{-1:44345} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.44:6826/2995756226,v1:172.21.15.44:6827/2995756226] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6a6406ecc0 msgr2=0x7f6a64071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6a6406ecc0 0x7f6a64071180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a74005ea0 tx=0x7f6a74005e30 comp rx=0 tx=0).stop 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 msgr2=0x7f6a7819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a7819cc40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800bda0 tx=0x7f6a6800be80 comp rx=0 tx=0).stop 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 shutdown_connections 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6a6406ecc0 0x7f6a64071180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a78100ea0 0x7f6a7819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 --2- 172.21.15.31:0/1969074408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a78101850 0x7f6a7819cc40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 >> 172.21.15.31:0/1969074408 conn(0x7f6a780fc750 msgr2=0x7f6a780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 shutdown_connections 2024-04-22T23:57:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:50.640+0000 7f6a7f518700 1 -- 172.21.15.31:0/1969074408 wait complete. 2024-04-22T23:57:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 40 2024-04-22T23:57:51.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:50 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1969074408' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:57:51.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:50 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1969074408' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:57:51.256 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-22T23:57:51.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:57:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/3844047425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 msgr2=0x7fb3c4109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/3844047425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c4109fd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80098b0 tx=0x7fb3b802bcd0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/3844047425 shutdown_connections 2024-04-22T23:57:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/3844047425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c4109fd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/3844047425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 0x7fb3c4101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/3844047425 >> 172.21.15.31:0/3844047425 conn(0x7fb3c40fa9f0 msgr2=0x7fb3c40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/3844047425 shutdown_connections 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/3844047425 wait complete. 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 Processor -- start 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.383+0000 7fb3c9ee9640 1 -- start start 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c9ee9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 0x7fb3c40fee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c9ee9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c40ff360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c9ee9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40ff8a0 con 0x7fb3c4101af0 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c9ee9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40ffa10 con 0x7fb3c4101140 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c40ff360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c40ff360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51238/0 (socket says 172.21.15.31:51238) 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/1741342032 learned_addr learned my addr 172.21.15.31:0/1741342032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c37fe640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 0x7fb3c40fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/1741342032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 msgr2=0x7fb3c40fee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 0x7fb3c40fee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 -- 172.21.15.31:0/1741342032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8009560 con 0x7fb3c4101af0 2024-04-22T23:57:51.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c37fe640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 0x7fb3c40fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:57:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c2ffd640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c40ff360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009880 tx=0x7fb3b802c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b80035f0 con 0x7fb3c4101af0 2024-04-22T23:57:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3b8003c10 con 0x7fb3c4101af0 2024-04-22T23:57:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c40ffc30 con 0x7fb3c4101af0 2024-04-22T23:57:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b80055a0 con 0x7fb3c4101af0 2024-04-22T23:57:51.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.384+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c41a2290 con 0x7fb3c4101af0 2024-04-22T23:57:51.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.385+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c404e040 con 0x7fb3c4101af0 2024-04-22T23:57:51.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.386+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fb3b8004dd0 con 0x7fb3c4101af0 2024-04-22T23:57:51.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.386+0000 7fb3c0ff9640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb3ac06ec30 0x7fb3ac0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:51.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.386+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fb3b80b1320 con 0x7fb3c4101af0 2024-04-22T23:57:51.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.386+0000 7fb3c37fe640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb3ac06ec30 0x7fb3ac0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:51.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.387+0000 7fb3c37fe640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb3ac06ec30 0x7fb3ac0710f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009b50 tx=0x7fb3b4005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:51.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.391+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb3b8079f60 con 0x7fb3c4101af0 2024-04-22T23:57:51.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.581+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3c4107640 con 0x7fb3c4101af0 2024-04-22T23:57:51.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.582+0000 7fb3c0ff9640 1 -- 172.21.15.31:0/1741342032 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17209 (secure 0 0 0) 0x7fb3b807d620 con 0x7fb3c4101af0 2024-04-22T23:57:51.573 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:51.573 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":92,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:55:09.702849+0000","last_up_change":"2024-04-22T23:54:01.112143+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:42:25.830464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:42:26.380171+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":85,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6803","nonce":3402089089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6805","nonce":3402089089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6809","nonce":3402089089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6807","nonce":3402089089}]},"public_addr":"172.21.15.31:6803/3402089089","cluster_addr":"172.21.15.31:6805/3402089089","heartbeat_back_addr":"172.21.15.31:6809/3402089089","heartbeat_front_addr":"172.21.15.31:6807/3402089089","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":62,"up_thru":85,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6811","nonce":753330875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6813","nonce":753330875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6817","nonce":753330875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6815","nonce":753330875}]},"public_addr":"172.21.15.31:6811/753330875","cluster_addr":"172.21.15.31:6813/753330875","heartbeat_back_addr":"172.21.15.31:6817/753330875","heartbeat_front_addr":"172.21.15.31:6815/753330875","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":66,"up_from":70,"up_thru":85,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6819","nonce":1746177235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6821","nonce":1746177235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6825","nonce":1746177235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6823","nonce":1746177235}]},"public_addr":"172.21.15.31:6819/1746177235","cluster_addr":"172.21.15.31:6821/1746177235","heartbeat_back_addr":"172.21.15.31:6825/1746177235","heartbeat_front_addr":"172.21.15.31:6823/1746177235","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":71,"up_from":75,"up_thru":85,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6801","nonce":2576217506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6803","nonce":2576217506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6807","nonce":2576217506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6805","nonce":2576217506}]},"public_addr":"172.21.15.44:6801/2576217506","cluster_addr":"172.21.15.44:6803/2576217506","heartbeat_back_addr":"172.21.15.44:6807/2576217506","heartbeat_front_addr":"172.21.15.44:6805/2576217506","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":76,"up_from":80,"up_thru":85,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6809","nonce":2616608091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6811","nonce":2616608091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6815","nonce":2616608091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6813","nonce":2616608091}]},"public_addr":"172.21.15.44:6809/2616608091","cluster_addr":"172.21.15.44:6811/2616608091","heartbeat_back_addr":"172.21.15.44:6815/2616608091","heartbeat_front_addr":"172.21.15.44:6813/2616608091","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":81,"up_from":85,"up_thru":85,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6817","nonce":2666243870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6819","nonce":2666243870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6823","nonce":2666243870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6821","nonce":2666243870}]},"public_addr":"172.21.15.44:6817/2666243870","cluster_addr":"172.21.15.44:6819/2666243870","heartbeat_back_addr":"172.21.15.44:6823/2666243870","heartbeat_front_addr":"172.21.15.44:6821/2666243870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":66},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":71},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":76},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":81}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1","mappings":[{"from":0,"to":5}]},{"pgid":"2.7","mappings":[{"from":3,"to":1}]},{"pgid":"2.c","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1d","mappings":[{"from":0,"to":2}]},{"pgid":"2.1f","mappings":[{"from":0,"to":1}]},{"pgid":"3.a","mappings":[{"from":3,"to":0}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":4,"to":5}]},{"pgid":"3.1b","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6825/1404295013":"2024-04-23T23:54:44.571005+0000","172.21.15.31:6801/418479598":"2024-04-23T23:49:30.073919+0000","172.21.15.44:0/1074135904":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6801/2402370856":"2024-04-23T23:44:52.608776+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.44:6827/3065727368":"2024-04-23T23:55:09.702810+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3574495240":"2024-04-23T23:44:52.608776+0000","172.21.15.44:0/2312976544":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.44:6828/170975257":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/1857354965":"2024-04-23T23:44:52.608776+0000","172.21.15.44:6829/170975257":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/3332075151":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/127255297":"2024-04-23T23:44:52.608776+0000","172.21.15.31:6800/2402370856":"2024-04-23T23:44:52.608776+0000","172.21.15.44:0/3035083544":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6826/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000","172.21.15.44:6826/3065727368":"2024-04-23T23:55:09.702810+0000","172.21.15.44:0/2034571521":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6827/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.44:0/3936244184":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/1029776087":"2024-04-23T23:44:52.608776+0000","172.21.15.44:6824/1404295013":"2024-04-23T23:54:44.571005+0000","172.21.15.44:0/3485102811":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/1913595989":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/445726858":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/2085576080":"2024-04-23T23:49:30.073919+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/418479598":"2024-04-23T23:49:30.073919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb3ac06ec30 msgr2=0x7fb3ac0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb3ac06ec30 0x7fb3ac0710f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009b50 tx=0x7fb3b4005f90 comp rx=0 tx=0).stop 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 msgr2=0x7fb3c40ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c40ff360 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009880 tx=0x7fb3b802c740 comp rx=0 tx=0).stop 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 shutdown_connections 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb3ac06ec30 0x7fb3ac0710f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3c4101af0 0x7fb3c40ff360 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 --2- 172.21.15.31:0/1741342032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3c4101140 0x7fb3c40fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 >> 172.21.15.31:0/1741342032 conn(0x7fb3c40fa9f0 msgr2=0x7fb3c4104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 shutdown_connections 2024-04-22T23:57:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.584+0000 7fb3c9ee9640 1 -- 172.21.15.31:0/1741342032 wait complete. 2024-04-22T23:57:51.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:39:59.535490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:42:25.830464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:42:26.380171+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:57:51.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.702+0000 7fa64323f640 1 -- 172.21.15.31:0/2786128421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ffb60 msgr2=0x7fa63c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.702+0000 7fa64323f640 1 --2- 172.21.15.31:0/2786128421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ffb60 0x7fa63c103ba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa630009880 tx=0x7fa63002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.702+0000 7fa64323f640 1 -- 172.21.15.31:0/2786128421 shutdown_connections 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.702+0000 7fa64323f640 1 --2- 172.21.15.31:0/2786128421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ffb60 0x7fa63c103ba0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.702+0000 7fa64323f640 1 --2- 172.21.15.31:0/2786128421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c0ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.702+0000 7fa64323f640 1 -- 172.21.15.31:0/2786128421 >> 172.21.15.31:0/2786128421 conn(0x7fa63c0fa9f0 msgr2=0x7fa63c0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 -- 172.21.15.31:0/2786128421 shutdown_connections 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 -- 172.21.15.31:0/2786128421 wait complete. 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 Processor -- start 2024-04-22T23:57:51.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 -- start start 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c101180 0x7fa63c1015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1a1ee0 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.703+0000 7fa64323f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1a2020 con 0x7fa63c101180 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51254/0 (socket says 172.21.15.31:51254) 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 -- 172.21.15.31:0/1916337843 learned_addr learned my addr 172.21.15.31:0/1916337843 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 -- 172.21.15.31:0/1916337843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c101180 msgr2=0x7fa63c1015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c101180 0x7fa63c1015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 -- 172.21.15.31:0/1916337843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa630009560 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa640fdb640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c100c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c920 tx=0x7fa62c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62c007b80 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa62c0054e0 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62c00f450 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c1a2160 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.704+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c1a26c0 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.705+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c04e040 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.706+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa62c004f60 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.706+0000 7fa639ffb640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa62406ec30 0x7fa6240710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:51.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.706+0000 7fa63bfff640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa62406ec30 0x7fa6240710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:51.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.706+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa62c08ee60 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.707+0000 7fa63bfff640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa62406ec30 0x7fa6240710f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa630036040 tx=0x7fa630000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:51.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.711+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa62c057aa0 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.900+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa63c0ffc00 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.901+0000 7fa639ffb640 1 -- 172.21.15.31:0/1916337843 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa62c01d020 con 0x7fa63c0ff1b0 2024-04-22T23:57:51.891 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-22T23:57:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa62406ec30 msgr2=0x7fa6240710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa62406ec30 0x7fa6240710f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa630036040 tx=0x7fa630000bc0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 msgr2=0x7fa63c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c100c40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00c920 tx=0x7fa62c00cde0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 shutdown_connections 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa62406ec30 0x7fa6240710f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c101180 0x7fa63c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 --2- 172.21.15.31:0/1916337843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0ff1b0 0x7fa63c100c40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 >> 172.21.15.31:0/1916337843 conn(0x7fa63c0fa9f0 msgr2=0x7fa63c0686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 shutdown_connections 2024-04-22T23:57:51.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:51.903+0000 7fa64323f640 1 -- 172.21.15.31:0/1916337843 wait complete. 2024-04-22T23:57:51.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:57:52.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 -- 172.21.15.31:0/1269138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c0ffb60 msgr2=0x7f9f1c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 --2- 172.21.15.31:0/1269138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c0ffb60 0x7f9f1c103ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009880 tx=0x7f9f1802bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 -- 172.21.15.31:0/1269138 shutdown_connections 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 --2- 172.21.15.31:0/1269138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c0ffb60 0x7f9f1c103ba0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 --2- 172.21.15.31:0/1269138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f1c0ff1b0 0x7f9f1c0ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 -- 172.21.15.31:0/1269138 >> 172.21.15.31:0/1269138 conn(0x7f9f1c0fa9f0 msgr2=0x7f9f1c0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 -- 172.21.15.31:0/1269138 shutdown_connections 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.018+0000 7f9f239bc640 1 -- 172.21.15.31:0/1269138 wait complete. 2024-04-22T23:57:52.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f239bc640 1 Processor -- start 2024-04-22T23:57:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f239bc640 1 -- start start 2024-04-22T23:57:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f239bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f1c0ff1b0 0x7f9f1c195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f239bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 0x7f9f1c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f239bc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c196920 con 0x7f9f1c196320 2024-04-22T23:57:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f239bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c196a90 con 0x7f9f1c0ff1b0 2024-04-22T23:57:52.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 0x7f9f1c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 0x7f9f1c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51272/0 (socket says 172.21.15.31:51272) 2024-04-22T23:57:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 -- 172.21.15.31:0/3970537179 learned_addr learned my addr 172.21.15.31:0/3970537179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f21758640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f1c0ff1b0 0x7f9f1c195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 -- 172.21.15.31:0/3970537179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f1c0ff1b0 msgr2=0x7f9f1c195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f1c0ff1b0 0x7f9f1c195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 -- 172.21.15.31:0/3970537179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009560 con 0x7f9f1c196320 2024-04-22T23:57:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f20f57640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 0x7f9f1c19a780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f18036040 tx=0x7f9f1802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.019+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f180035f0 con 0x7f9f1c196320 2024-04-22T23:57:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.020+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f18003c10 con 0x7f9f1c196320 2024-04-22T23:57:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.020+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c19ad20 con 0x7f9f1c196320 2024-04-22T23:57:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.020+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f18034440 con 0x7f9f1c196320 2024-04-22T23:57:52.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.020+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c19b280 con 0x7f9f1c196320 2024-04-22T23:57:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.020+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c04e040 con 0x7f9f1c196320 2024-04-22T23:57:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.021+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f9f18003750 con 0x7f9f1c196320 2024-04-22T23:57:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.021+0000 7f9f127fc640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9f0806ec30 0x7f9f080710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.022+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f9f180b1390 con 0x7f9f1c196320 2024-04-22T23:57:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.022+0000 7f9f21758640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9f0806ec30 0x7f9f080710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.022+0000 7f9f21758640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9f0806ec30 0x7f9f080710f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c005fd0 tx=0x7f9f0c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.026+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9f18079fd0 con 0x7f9f1c196320 2024-04-22T23:57:52.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:51 smithi031 ceph-mon[111289]: pgmap v296: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:52.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:51 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1741342032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:57:52.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.214+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f9f1c197130 con 0x7f9f1c196320 2024-04-22T23:57:52.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.214+0000 7f9f127fc640 1 -- 172.21.15.31:0/3970537179 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9f1809ce60 con 0x7f9f1c196320 2024-04-22T23:57:52.205 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:57:52.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9f0806ec30 msgr2=0x7f9f080710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9f0806ec30 0x7f9f080710f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c005fd0 tx=0x7f9f0c005850 comp rx=0 tx=0).stop 2024-04-22T23:57:52.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 msgr2=0x7f9f1c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 0x7f9f1c19a780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f18036040 tx=0x7f9f1802c850 comp rx=0 tx=0).stop 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 shutdown_connections 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f9f0806ec30 0x7f9f080710f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f1c196320 0x7f9f1c19a780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 --2- 172.21.15.31:0/3970537179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f1c0ff1b0 0x7f9f1c195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 >> 172.21.15.31:0/3970537179 conn(0x7f9f1c0fa9f0 msgr2=0x7f9f1c0fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 shutdown_connections 2024-04-22T23:57:52.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.217+0000 7f9f239bc640 1 -- 172.21.15.31:0/3970537179 wait complete. 2024-04-22T23:57:52.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:51 smithi044 ceph-mon[82146]: pgmap v296: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:52.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:51 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1741342032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:57:52.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:57:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 -- 172.21.15.31:0/1037669043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 msgr2=0x7f41400fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 --2- 172.21.15.31:0/1037669043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400fee70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f413c009880 tx=0x7f413c02bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 -- 172.21.15.31:0/1037669043 shutdown_connections 2024-04-22T23:57:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 --2- 172.21.15.31:0/1037669043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400ff440 0x7f4140104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 --2- 172.21.15.31:0/1037669043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400fee70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 -- 172.21.15.31:0/1037669043 >> 172.21.15.31:0/1037669043 conn(0x7f4140074120 msgr2=0x7f4140074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 -- 172.21.15.31:0/1037669043 shutdown_connections 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 -- 172.21.15.31:0/1037669043 wait complete. 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.332+0000 7f4146ef4640 1 Processor -- start 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4146ef4640 1 -- start start 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4146ef4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4146ef4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400fff40 0x7f41401003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4146ef4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a1ef0 con 0x7f41400fea90 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4146ef4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a2030 con 0x7f41400fff40 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400ffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51302/0 (socket says 172.21.15.31:51302) 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 -- 172.21.15.31:0/2556871366 learned_addr learned my addr 172.21.15.31:0/2556871366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4137fff640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400fff40 0x7f41401003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 -- 172.21.15.31:0/2556871366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400fff40 msgr2=0x7f41401003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400fff40 0x7f41401003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 -- 172.21.15.31:0/2556871366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c009560 con 0x7f41400fea90 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4137fff640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400fff40 0x7f41401003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4144c90640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400ffa00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f413c009430 tx=0x7f413c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f413c0035f0 con 0x7f41400fea90 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f413c003c10 con 0x7f41400fea90 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41401a2170 con 0x7f41400fea90 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.333+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f413c005670 con 0x7f41400fea90 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.334+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a26a0 con 0x7f41400fea90 2024-04-22T23:57:52.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.335+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f414004e040 con 0x7f41400fea90 2024-04-22T23:57:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.335+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f413c004dd0 con 0x7f41400fea90 2024-04-22T23:57:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.336+0000 7f4135ffb640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f412806ebe0 0x7f41280710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.336+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f413c0b13c0 con 0x7f41400fea90 2024-04-22T23:57:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.336+0000 7f4137fff640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f412806ebe0 0x7f41280710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.336+0000 7f4137fff640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f412806ebe0 0x7f41280710a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4140072510 tx=0x7f4130005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.340+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f413c07a000 con 0x7f41400fea90 2024-04-22T23:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.526+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f41401085e0 con 0x7f41400fea90 2024-04-22T23:57:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.527+0000 7f4135ffb640 1 -- 172.21.15.31:0/2556871366 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7f413c07d6c0 con 0x7f41400fea90 2024-04-22T23:57:52.517 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:57:52.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f412806ebe0 msgr2=0x7f41280710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f412806ebe0 0x7f41280710a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4140072510 tx=0x7f4130005e50 comp rx=0 tx=0).stop 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 msgr2=0x7f41400ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400ffa00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f413c009430 tx=0x7f413c004040 comp rx=0 tx=0).stop 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 shutdown_connections 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f412806ebe0 0x7f41280710a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f41400fff40 0x7f41401003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 --2- 172.21.15.31:0/2556871366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41400fea90 0x7f41400ffa00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 >> 172.21.15.31:0/2556871366 conn(0x7f4140074120 msgr2=0x7f41400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 shutdown_connections 2024-04-22T23:57:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.529+0000 7f4146ef4640 1 -- 172.21.15.31:0/2556871366 wait complete. 2024-04-22T23:57:52.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T23:57:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.641+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/1513580315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0067f60 msgr2=0x7ff8c0104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.641+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/1513580315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0067f60 0x7ff8c0104cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009880 tx=0x7ff8b002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/1513580315 shutdown_connections 2024-04-22T23:57:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/1513580315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0067f60 0x7ff8c0104cb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/1513580315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c00675b0 0x7ff8c0067990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/1513580315 >> 172.21.15.31:0/1513580315 conn(0x7ff8c0075bb0 msgr2=0x7ff8c0075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/1513580315 shutdown_connections 2024-04-22T23:57:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/1513580315 wait complete. 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 Processor -- start 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- start start 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c00675b0 0x7ff8c0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 0x7ff8c019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0198a10 con 0x7ff8c0198410 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8c6fe1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0198b80 con 0x7ff8c00675b0 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 0x7ff8c019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 0x7ff8c019c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51318/0 (socket says 172.21.15.31:51318) 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.642+0000 7ff8bffff640 1 -- 172.21.15.31:0/3361347826 learned_addr learned my addr 172.21.15.31:0/3361347826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8c4d7d640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c00675b0 0x7ff8c0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bffff640 1 -- 172.21.15.31:0/3361347826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c00675b0 msgr2=0x7ff8c0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bffff640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c00675b0 0x7ff8c0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bffff640 1 -- 172.21.15.31:0/3361347826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b0009560 con 0x7ff8c0198410 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bffff640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 0x7ff8c019c870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0036040 tx=0x7ff8b002c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b00035f0 con 0x7ff8c0198410 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b0003c10 con 0x7ff8c0198410 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c019ce10 con 0x7ff8c0198410 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b0005630 con 0x7ff8c0198410 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.643+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c019d3f0 con 0x7ff8c0198410 2024-04-22T23:57:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.644+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c019d6d0 con 0x7ff8c0198410 2024-04-22T23:57:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.645+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff8b0003750 con 0x7ff8c0198410 2024-04-22T23:57:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.645+0000 7ff8bdffb640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff8ac06ec30 0x7ff8ac0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.645+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff8b00b1310 con 0x7ff8c0198410 2024-04-22T23:57:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.645+0000 7ff8c4d7d640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff8ac06ec30 0x7ff8ac0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.646+0000 7ff8c4d7d640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff8ac06ec30 0x7ff8ac0710f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009bd0 tx=0x7ff8a8005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.650+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff8b0079fe0 con 0x7ff8c0198410 2024-04-22T23:57:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.877+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c004e040 con 0x7ff8c0198410 2024-04-22T23:57:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.878+0000 7ff8bdffb640 1 -- 172.21.15.31:0/3361347826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5215 (secure 0 0 0) 0x7ff8b007d6a0 con 0x7ff8c0198410 2024-04-22T23:57:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:52.869 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:54.549878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-04-22T23:57:52.869 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 40 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff8ac06ec30 msgr2=0x7ff8ac0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff8ac06ec30 0x7ff8ac0710f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009bd0 tx=0x7ff8a8005f90 comp rx=0 tx=0).stop 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 msgr2=0x7ff8c019c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 0x7ff8c019c870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0036040 tx=0x7ff8b002c750 comp rx=0 tx=0).stop 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 shutdown_connections 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff8ac06ec30 0x7ff8ac0710f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8c0198410 0x7ff8c019c870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 --2- 172.21.15.31:0/3361347826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff8c00675b0 0x7ff8c0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 >> 172.21.15.31:0/3361347826 conn(0x7ff8c0075bb0 msgr2=0x7ff8c0101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 shutdown_connections 2024-04-22T23:57:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.880+0000 7ff8c6fe1640 1 -- 172.21.15.31:0/3361347826 wait complete. 2024-04-22T23:57:52.882 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-04-22T23:57:52.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-22T23:57:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 -- 172.21.15.31:0/233294088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 msgr2=0x7fc5580ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 --2- 172.21.15.31:0/233294088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc5580ff470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc554009880 tx=0x7fc55402bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 -- 172.21.15.31:0/233294088 shutdown_connections 2024-04-22T23:57:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 --2- 172.21.15.31:0/233294088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc5580ff470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 --2- 172.21.15.31:0/233294088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc558104fc0 0x7fc5581053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 -- 172.21.15.31:0/233294088 >> 172.21.15.31:0/233294088 conn(0x7fc558075b50 msgr2=0x7fc558075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 -- 172.21.15.31:0/233294088 shutdown_connections 2024-04-22T23:57:52.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 -- 172.21.15.31:0/233294088 wait complete. 2024-04-22T23:57:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 Processor -- start 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.993+0000 7fc55fc28640 1 -- start start 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55fc28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc558196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55fc28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc558104fc0 0x7fc5581970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55fc28640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc558197620 con 0x7fc5580ff030 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55fc28640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc558197790 con 0x7fc558104fc0 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc558196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc558196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51330/0 (socket says 172.21.15.31:51330) 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 -- 172.21.15.31:0/2042175196 learned_addr learned my addr 172.21.15.31:0/2042175196 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d1c3640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc558104fc0 0x7fc5581970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 -- 172.21.15.31:0/2042175196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc558104fc0 msgr2=0x7fc5581970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc558104fc0 0x7fc5581970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 -- 172.21.15.31:0/2042175196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc554009560 con 0x7fc5580ff030 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55d9c4640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc558196ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc54800c8a0 tx=0x7fc54800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc548007b80 con 0x7fc5580ff030 2024-04-22T23:57:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc548007ce0 con 0x7fc5580ff030 2024-04-22T23:57:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc558197a20 con 0x7fc5580ff030 2024-04-22T23:57:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc548018810 con 0x7fc5580ff030 2024-04-22T23:57:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.994+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55819ffe0 con 0x7fc5580ff030 2024-04-22T23:57:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.995+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55804e040 con 0x7fc5580ff030 2024-04-22T23:57:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.996+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fc548018970 con 0x7fc5580ff030 2024-04-22T23:57:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.996+0000 7fc54effd640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc54406ec30 0x7fc5440710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.996+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fc54808f140 con 0x7fc5580ff030 2024-04-22T23:57:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.996+0000 7fc55d1c3640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc54406ec30 0x7fc5440710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:52.997+0000 7fc55d1c3640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc54406ec30 0x7fc5440710f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc558067ba0 tx=0x7fc554000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:52.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.001+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc548057e00 con 0x7fc5580ff030 2024-04-22T23:57:53.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:52 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1916337843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:57:53.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:52 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3970537179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:57:53.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:52 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2556871366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:57:53.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:52 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3361347826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:57:53.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:52 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1916337843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:57:53.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:52 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3970537179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:57:53.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:52 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2556871366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:57:53.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:52 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3361347826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:57:53.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.226+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fc558107440 con 0x7fc5580ff030 2024-04-22T23:57:53.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.227+0000 7fc54effd640 1 -- 172.21.15.31:0/2042175196 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v40) v1 ==== 107+0+4088 (secure 0 0 0) 0x7fc54805b4c0 con 0x7fc5580ff030 2024-04-22T23:57:53.218 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14500,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2302742063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2302742063},{"type":"v1","addr":"172.21.15.31:6827","nonce":2302742063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:04.995710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24293,"qdb_cluster":[24293]},"id":1}]} 2024-04-22T23:57:53.218 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-04-22T23:57:53.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc54406ec30 msgr2=0x7fc5440710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc54406ec30 0x7fc5440710f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc558067ba0 tx=0x7fc554000bc0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 msgr2=0x7fc558196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc558196ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc54800c8a0 tx=0x7fc54800cd60 comp rx=0 tx=0).stop 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 shutdown_connections 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fc54406ec30 0x7fc5440710f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc558104fc0 0x7fc5581970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 --2- 172.21.15.31:0/2042175196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5580ff030 0x7fc558196ba0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 >> 172.21.15.31:0/2042175196 conn(0x7fc558075b50 msgr2=0x7fc5580fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 shutdown_connections 2024-04-22T23:57:53.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.230+0000 7fc55fc28640 1 -- 172.21.15.31:0/2042175196 wait complete. 2024-04-22T23:57:53.232 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-04-22T23:57:53.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-22T23:57:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 -- 172.21.15.31:0/2759802598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c100280 msgr2=0x7fd09c100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 --2- 172.21.15.31:0/2759802598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c100280 0x7fd09c100660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009880 tx=0x7fd08c02bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 -- 172.21.15.31:0/2759802598 shutdown_connections 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 --2- 172.21.15.31:0/2759802598 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd09c100c30 0x7fd09c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 --2- 172.21.15.31:0/2759802598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c100280 0x7fd09c100660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 -- 172.21.15.31:0/2759802598 >> 172.21.15.31:0/2759802598 conn(0x7fd09c0fbaa0 msgr2=0x7fd09c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 -- 172.21.15.31:0/2759802598 shutdown_connections 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 -- 172.21.15.31:0/2759802598 wait complete. 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.343+0000 7fd0a2163640 1 Processor -- start 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd0a2163640 1 -- start start 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd0a2163640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd09c100280 0x7fd09c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd0a2163640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 0x7fd09c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd0a2163640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c1968f0 con 0x7fd09c1962f0 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd0a2163640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c196a60 con 0x7fd09c100280 2024-04-22T23:57:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 0x7fd09c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 0x7fd09c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51346/0 (socket says 172.21.15.31:51346) 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 -- 172.21.15.31:0/4272770474 learned_addr learned my addr 172.21.15.31:0/4272770474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09b7fe640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd09c100280 0x7fd09c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 -- 172.21.15.31:0/4272770474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd09c100280 msgr2=0x7fd09c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd09c100280 0x7fd09c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 -- 172.21.15.31:0/4272770474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c009560 con 0x7fd09c1962f0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.344+0000 7fd09affd640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 0x7fd09c19a750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d6f0 tx=0x7fd09000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.345+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0900051f0 con 0x7fd09c1962f0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.345+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c19ad50 con 0x7fd09c1962f0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.345+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd090009ce0 con 0x7fd09c1962f0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.345+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd090004e80 con 0x7fd09c1962f0 2024-04-22T23:57:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.345+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c19b2b0 con 0x7fd09c1962f0 2024-04-22T23:57:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.346+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c19b580 con 0x7fd09c1962f0 2024-04-22T23:57:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.346+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fd0900046c0 con 0x7fd09c1962f0 2024-04-22T23:57:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.347+0000 7fd098ff9640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd08406ebe0 0x7fd0840710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.347+0000 7fd09b7fe640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd08406ebe0 0x7fd0840710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.347+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fd09008eb30 con 0x7fd09c1962f0 2024-04-22T23:57:53.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.347+0000 7fd09b7fe640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd08406ebe0 0x7fd0840710a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd08c006010 tx=0x7fd08c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:53.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.352+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fd0900576d0 con 0x7fd09c1962f0 2024-04-22T23:57:53.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.582+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fd09c04e040 con 0x7fd09c1962f0 2024-04-22T23:57:53.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.583+0000 7fd098ff9640 1 -- 172.21.15.31:0/4272770474 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v40) v1 ==== 107+0+4864 (secure 0 0 0) 0x7fd09005ad90 con 0x7fd09c1962f0 2024-04-22T23:57:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:53.574 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14500,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2302742063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2302742063},{"type":"v1","addr":"172.21.15.31:6827","nonce":2302742063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34374,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/367136352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":367136352},{"type":"v1","addr":"172.21.15.31:6829","nonce":367136352}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:04.995710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24293,"qdb_cluster":[24293]},"id":1}]} 2024-04-22T23:57:53.574 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd08406ebe0 msgr2=0x7fd0840710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd08406ebe0 0x7fd0840710a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd08c006010 tx=0x7fd08c000bc0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 msgr2=0x7fd09c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 0x7fd09c19a750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd09000d6f0 tx=0x7fd09000dbb0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 shutdown_connections 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fd08406ebe0 0x7fd0840710a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd09c1962f0 0x7fd09c19a750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 --2- 172.21.15.31:0/4272770474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd09c100280 0x7fd09c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 >> 172.21.15.31:0/4272770474 conn(0x7fd09c0fbaa0 msgr2=0x7fd09c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 shutdown_connections 2024-04-22T23:57:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.585+0000 7fd0a2163640 1 -- 172.21.15.31:0/4272770474 wait complete. 2024-04-22T23:57:53.587 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-04-22T23:57:53.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-22T23:57:53.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.699+0000 7fba18586640 1 -- 172.21.15.31:0/3554585677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 msgr2=0x7fba100ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.699+0000 7fba18586640 1 --2- 172.21.15.31:0/3554585677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba100ffd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba00009880 tx=0x7fba0002bd70 comp rx=0 tx=0).stop 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.699+0000 7fba18586640 1 -- 172.21.15.31:0/3554585677 shutdown_connections 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.699+0000 7fba18586640 1 --2- 172.21.15.31:0/3554585677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba100ffd90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.699+0000 7fba18586640 1 --2- 172.21.15.31:0/3554585677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba10105070 0x7fba10105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.699+0000 7fba18586640 1 -- 172.21.15.31:0/3554585677 >> 172.21.15.31:0/3554585677 conn(0x7fba10074120 msgr2=0x7fba10074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 -- 172.21.15.31:0/3554585677 shutdown_connections 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 -- 172.21.15.31:0/3554585677 wait complete. 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 Processor -- start 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 -- start start 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba10197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba10105070 0x7fba10198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba10198a90 con 0x7fba100ff930 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba18586640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1019c860 con 0x7fba10105070 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba16322640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba10197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:53.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba16322640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba10197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51374/0 (socket says 172.21.15.31:51374) 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.700+0000 7fba16322640 1 -- 172.21.15.31:0/3114276611 learned_addr learned my addr 172.21.15.31:0/3114276611 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba15b21640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba10105070 0x7fba10198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba16322640 1 -- 172.21.15.31:0/3114276611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba10105070 msgr2=0x7fba10198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba16322640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba10105070 0x7fba10198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba16322640 1 -- 172.21.15.31:0/3114276611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00009560 con 0x7fba100ff930 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba16322640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba10197ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00e970 tx=0x7fba0c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0c00cc90 con 0x7fba100ff930 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1019cb40 con 0x7fba100ff930 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba0c00cdf0 con 0x7fba100ff930 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba0c018810 con 0x7fba100ff930 2024-04-22T23:57:53.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.701+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1019d120 con 0x7fba100ff930 2024-04-22T23:57:53.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.702+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1004e040 con 0x7fba100ff930 2024-04-22T23:57:53.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.702+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fba0c010c10 con 0x7fba100ff930 2024-04-22T23:57:53.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.703+0000 7fba077fe640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb9fc06ec30 0x7fb9fc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:53.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.703+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fba0c014070 con 0x7fba100ff930 2024-04-22T23:57:53.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.703+0000 7fba15b21640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb9fc06ec30 0x7fb9fc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:53.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.703+0000 7fba15b21640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb9fc06ec30 0x7fb9fc0710f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba101994d0 tx=0x7fba00000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:53.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.708+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fba0c054150 con 0x7fba100ff930 2024-04-22T23:57:53.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.934+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fba10199210 con 0x7fba100ff930 2024-04-22T23:57:53.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.934+0000 7fba077fe640 1 -- 172.21.15.31:0/3114276611 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v40) v1 ==== 107+0+4085 (secure 0 0 0) 0x7fba10199210 con 0x7fba100ff930 2024-04-22T23:57:53.925 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:53.925 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34374,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/367136352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":367136352},{"type":"v1","addr":"172.21.15.31:6829","nonce":367136352}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:04.995710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24293,"qdb_cluster":[24293]},"id":1}]} 2024-04-22T23:57:53.925 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-04-22T23:57:53.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb9fc06ec30 msgr2=0x7fb9fc0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb9fc06ec30 0x7fb9fc0710f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba101994d0 tx=0x7fba00000bc0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 msgr2=0x7fba10197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba10197ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00e970 tx=0x7fba0c00ee30 comp rx=0 tx=0).stop 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 shutdown_connections 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fb9fc06ec30 0x7fb9fc0710f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba10105070 0x7fba10198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 --2- 172.21.15.31:0/3114276611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba100ff930 0x7fba10197ec0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.936+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 >> 172.21.15.31:0/3114276611 conn(0x7fba10074120 msgr2=0x7fba100fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.937+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 shutdown_connections 2024-04-22T23:57:53.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:53.937+0000 7fba18586640 1 -- 172.21.15.31:0/3114276611 wait complete. 2024-04-22T23:57:53.938 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-04-22T23:57:53.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/4269508797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101ba0 msgr2=0x7fa6e010a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/4269508797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101ba0 0x7fa6e010a080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009880 tx=0x7fa6c802bd70 comp rx=0 tx=0).stop 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/4269508797 shutdown_connections 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/4269508797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101ba0 0x7fa6e010a080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/4269508797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e0101280 0x7fa6e0101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/4269508797 >> 172.21.15.31:0/4269508797 conn(0x7fa6e00fa9c0 msgr2=0x7fa6e00fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/4269508797 shutdown_connections 2024-04-22T23:57:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.050+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/4269508797 wait complete. 2024-04-22T23:57:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6e4ac0640 1 Processor -- start 2024-04-22T23:57:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6e4ac0640 1 -- start start 2024-04-22T23:57:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6e4ac0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 0x7fa6e0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6e4ac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e0101ba0 0x7fa6e0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6e4ac0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0196950 con 0x7fa6e0101280 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6e4ac0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01995d0 con 0x7fa6e0101ba0 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 0x7fa6e0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 0x7fa6e0195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51402/0 (socket says 172.21.15.31:51402) 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 -- 172.21.15.31:0/2305469975 learned_addr learned my addr 172.21.15.31:0/2305469975 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6ddd9b640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e0101ba0 0x7fa6e0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 -- 172.21.15.31:0/2305469975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e0101ba0 msgr2=0x7fa6e0196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e0101ba0 0x7fa6e0196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 -- 172.21.15.31:0/2305469975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c8009560 con 0x7fa6e0101280 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.051+0000 7fa6de59c640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 0x7fa6e0195d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800db80 tx=0x7fa6d800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.052+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d800de40 con 0x7fa6e0101280 2024-04-22T23:57:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.052+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6d8003e30 con 0x7fa6e0101280 2024-04-22T23:57:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.052+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e01998b0 con 0x7fa6e0101280 2024-04-22T23:57:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.052+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d8010830 con 0x7fa6e0101280 2024-04-22T23:57:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.052+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e0199de0 con 0x7fa6e0101280 2024-04-22T23:57:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.053+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e004e040 con 0x7fa6e0101280 2024-04-22T23:57:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.053+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa6d8004020 con 0x7fa6e0101280 2024-04-22T23:57:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.053+0000 7fa6d77fe640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa6cc06ec30 0x7fa6cc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.054+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa6d808f340 con 0x7fa6e0101280 2024-04-22T23:57:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.054+0000 7fa6ddd9b640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa6cc06ec30 0x7fa6cc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.054+0000 7fa6ddd9b640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa6cc06ec30 0x7fa6cc0710f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0197350 tx=0x7fa6c8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:54.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.058+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa6d8057f80 con 0x7fa6e0101280 2024-04-22T23:57:54.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:53 smithi031 ceph-mon[111289]: pgmap v297: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:54.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:53 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2042175196' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-22T23:57:54.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:53 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4272770474' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-22T23:57:54.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:53 smithi044 ceph-mon[82146]: pgmap v297: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:57:54.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:53 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2042175196' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-22T23:57:54.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:53 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4272770474' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-22T23:57:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.285+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fa6e01076f0 con 0x7fa6e0101280 2024-04-22T23:57:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.286+0000 7fa6d77fe640 1 -- 172.21.15.31:0/2305469975 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v40) v1 ==== 107+0+4932 (secure 0 0 0) 0x7fa6d80042d0 con 0x7fa6e0101280 2024-04-22T23:57:54.277 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:54.277 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34374,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/367136352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":367136352},{"type":"v1","addr":"172.21.15.31:6829","nonce":367136352}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:04.995710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24293,"qdb_cluster":[24293]},"id":1}]} 2024-04-22T23:57:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa6cc06ec30 msgr2=0x7fa6cc0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa6cc06ec30 0x7fa6cc0710f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0197350 tx=0x7fa6c8000bc0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 msgr2=0x7fa6e0195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 0x7fa6e0195d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800db80 tx=0x7fa6d800b630 comp rx=0 tx=0).stop 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 shutdown_connections 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa6cc06ec30 0x7fa6cc0710f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6e0101ba0 0x7fa6e0196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 --2- 172.21.15.31:0/2305469975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e0101280 0x7fa6e0195d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 >> 172.21.15.31:0/2305469975 conn(0x7fa6e00fa9c0 msgr2=0x7fa6e01048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 shutdown_connections 2024-04-22T23:57:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.288+0000 7fa6e4ac0640 1 -- 172.21.15.31:0/2305469975 wait complete. 2024-04-22T23:57:54.290 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-04-22T23:57:54.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 -- 172.21.15.31:0/2514704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9188100c30 msgr2=0x7f9188104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 --2- 172.21.15.31:0/2514704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9188100c30 0x7f9188104c70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f917c009880 tx=0x7f917c02bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 -- 172.21.15.31:0/2514704963 shutdown_connections 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 --2- 172.21.15.31:0/2514704963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9188100c30 0x7f9188104c70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 --2- 172.21.15.31:0/2514704963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 0x7f9188100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 -- 172.21.15.31:0/2514704963 >> 172.21.15.31:0/2514704963 conn(0x7f91880fbaa0 msgr2=0x7f91880fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 -- 172.21.15.31:0/2514704963 shutdown_connections 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 -- 172.21.15.31:0/2514704963 wait complete. 2024-04-22T23:57:54.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.402+0000 7f918e0b6640 1 Processor -- start 2024-04-22T23:57:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f918e0b6640 1 -- start start 2024-04-22T23:57:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f918e0b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 0x7f9188195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f918e0b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 0x7f918819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f918e0b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91881968a0 con 0x7f91881962a0 2024-04-22T23:57:54.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f918e0b6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188196a10 con 0x7f9188100280 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 0x7f918819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 0x7f918819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51428/0 (socket says 172.21.15.31:51428) 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f9186ffd640 1 -- 172.21.15.31:0/3181294231 learned_addr learned my addr 172.21.15.31:0/3181294231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f91877fe640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 0x7f9188195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f9186ffd640 1 -- 172.21.15.31:0/3181294231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 msgr2=0x7f9188195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f9186ffd640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 0x7f9188195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f9186ffd640 1 -- 172.21.15.31:0/3181294231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009560 con 0x7f91881962a0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.403+0000 7f91877fe640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 0x7f9188195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.404+0000 7f9186ffd640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 0x7f918819a700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f917c036040 tx=0x7f917c02c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.404+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f917c004520 con 0x7f91881962a0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.404+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f917c003ab0 con 0x7f91881962a0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.404+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918819aca0 con 0x7f91881962a0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.404+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f917c034440 con 0x7f91881962a0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.404+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918819b200 con 0x7f91881962a0 2024-04-22T23:57:54.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.405+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918804e040 con 0x7f91881962a0 2024-04-22T23:57:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.405+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f917c003c20 con 0x7f91881962a0 2024-04-22T23:57:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.406+0000 7f9184ff9640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f917006ec80 0x7f9170071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.406+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f917c0b1270 con 0x7f91881962a0 2024-04-22T23:57:54.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.406+0000 7f91877fe640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f917006ec80 0x7f9170071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.406+0000 7f91877fe640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f917006ec80 0x7f9170071140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9178009b40 tx=0x7f9178005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.411+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f917c079f30 con 0x7f91881962a0 2024-04-22T23:57:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.638+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f9188108580 con 0x7f91881962a0 2024-04-22T23:57:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.638+0000 7f9184ff9640 1 -- 172.21.15.31:0/3181294231 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v40) v1 ==== 107+0+4156 (secure 0 0 0) 0x7f917c07d5f0 con 0x7f91881962a0 2024-04-22T23:57:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:54.629 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:04.995710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24293,"qdb_cluster":[24293]},"id":1}]} 2024-04-22T23:57:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-22T23:57:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f917006ec80 msgr2=0x7f9170071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f917006ec80 0x7f9170071140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9178009b40 tx=0x7f9178005e50 comp rx=0 tx=0).stop 2024-04-22T23:57:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 msgr2=0x7f918819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 0x7f918819a700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f917c036040 tx=0x7f917c02c7a0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 shutdown_connections 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f917006ec80 0x7f9170071140 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91881962a0 0x7f918819a700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 --2- 172.21.15.31:0/3181294231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9188100280 0x7f9188195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.640+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 >> 172.21.15.31:0/3181294231 conn(0x7f91880fbaa0 msgr2=0x7f91880fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.641+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 shutdown_connections 2024-04-22T23:57:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.641+0000 7f918e0b6640 1 -- 172.21.15.31:0/3181294231 wait complete. 2024-04-22T23:57:54.642 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-04-22T23:57:54.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/803423387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 msgr2=0x7f6ebc1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/803423387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc1015d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009880 tx=0x7f6eac02bd70 comp rx=0 tx=0).stop 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/803423387 shutdown_connections 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/803423387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ebc101b10 0x7f6ebc109ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/803423387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc1015d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/803423387 >> 172.21.15.31:0/803423387 conn(0x7f6ebc0faa30 msgr2=0x7f6ebc0fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/803423387 shutdown_connections 2024-04-22T23:57:54.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.754+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/803423387 wait complete. 2024-04-22T23:57:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ec2a2f640 1 Processor -- start 2024-04-22T23:57:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ec2a2f640 1 -- start start 2024-04-22T23:57:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ec2a2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc0fee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ec2a2f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ebc101b10 0x7f6ebc0ff380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ec2a2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0ff8c0 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ec2a2f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0ffa30 con 0x7f6ebc101b10 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc0fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc0fee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51436/0 (socket says 172.21.15.31:51436) 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebbfff640 1 -- 172.21.15.31:0/4088642691 learned_addr learned my addr 172.21.15.31:0/4088642691 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebb7fe640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ebc101b10 0x7f6ebc0ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebbfff640 1 -- 172.21.15.31:0/4088642691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ebc101b10 msgr2=0x7f6ebc0ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebbfff640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ebc101b10 0x7f6ebc0ff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.755+0000 7f6ebbfff640 1 -- 172.21.15.31:0/4088642691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009560 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.756+0000 7f6ebbfff640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc0fee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009230 tx=0x7f6eac02c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.756+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eac0044a0 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.756+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eac003ab0 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.756+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc0ffc50 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.756+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eac005600 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.756+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc1a22c0 con 0x7f6ebc1011f0 2024-04-22T23:57:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.757+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc04e040 con 0x7f6ebc1011f0 2024-04-22T23:57:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.757+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f6eac004dd0 con 0x7f6ebc1011f0 2024-04-22T23:57:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.758+0000 7f6eb97fa640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6ea406ec30 0x7f6ea40710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.758+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f6eac0b1a60 con 0x7f6ebc1011f0 2024-04-22T23:57:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.758+0000 7f6ebb7fe640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6ea406ec30 0x7f6ea40710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.758+0000 7f6ebb7fe640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6ea406ec30 0x7f6ea40710f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc100630 tx=0x7f6eb0005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:54.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.763+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6eac07a6a0 con 0x7f6ebc1011f0 2024-04-22T23:57:54.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:54 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3114276611' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-22T23:57:54.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:54 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2305469975' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-22T23:57:54.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:54 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3181294231' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-22T23:57:54.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.990+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f6ebc107660 con 0x7f6ebc1011f0 2024-04-22T23:57:54.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.991+0000 7f6eb97fa640 1 -- 172.21.15.31:0/4088642691 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v40) v1 ==== 107+0+5003 (secure 0 0 0) 0x7f6eac07dd60 con 0x7f6ebc1011f0 2024-04-22T23:57:54.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:54.981 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:04.995710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24293":{"gid":24293,"name":"cephfs.smithi044.ibkztg","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.44:6825/1404295013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1404295013},{"type":"v1","addr":"172.21.15.44:6825","nonce":1404295013}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24293,"qdb_cluster":[24293]},"id":1}]} 2024-04-22T23:57:54.982 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2024-04-22T23:57:54.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6ea406ec30 msgr2=0x7f6ea40710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6ea406ec30 0x7f6ea40710f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc100630 tx=0x7f6eb0005f90 comp rx=0 tx=0).stop 2024-04-22T23:57:54.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 msgr2=0x7f6ebc0fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc0fee40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009230 tx=0x7f6eac02c7a0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 shutdown_connections 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f6ea406ec30 0x7f6ea40710f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6ebc101b10 0x7f6ebc0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 --2- 172.21.15.31:0/4088642691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ebc1011f0 0x7f6ebc0fee40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 >> 172.21.15.31:0/4088642691 conn(0x7f6ebc0faa30 msgr2=0x7f6ebc104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 shutdown_connections 2024-04-22T23:57:54.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:54.993+0000 7f6ec2a2f640 1 -- 172.21.15.31:0/4088642691 wait complete. 2024-04-22T23:57:54.994 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-04-22T23:57:54.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-22T23:57:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 -- 172.21.15.31:0/3923907931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 msgr2=0x7f06b00ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 --2- 172.21.15.31:0/3923907931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b00ff1d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009880 tx=0x7f06a002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 -- 172.21.15.31:0/3923907931 shutdown_connections 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 --2- 172.21.15.31:0/3923907931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 0x7f06b010c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 --2- 172.21.15.31:0/3923907931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b00ff1d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 -- 172.21.15.31:0/3923907931 >> 172.21.15.31:0/3923907931 conn(0x7f06b00fa9f0 msgr2=0x7f06b00fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 -- 172.21.15.31:0/3923907931 shutdown_connections 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 -- 172.21.15.31:0/3923907931 wait complete. 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.107+0000 7f06b587d640 1 Processor -- start 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06b587d640 1 -- start start 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06b587d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06b587d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 0x7f06b01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06b587d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0198a70 con 0x7f06b00fedf0 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06b587d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b019c840 con 0x7f06b00ff7a0 2024-04-22T23:57:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51454/0 (socket says 172.21.15.31:51454) 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 -- 172.21.15.31:0/1717620552 learned_addr learned my addr 172.21.15.31:0/1717620552 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06a7fff640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 0x7f06b01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 -- 172.21.15.31:0/1717620552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 msgr2=0x7f06b01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 0x7f06b01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 -- 172.21.15.31:0/1717620552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0009560 con 0x7f06b00fedf0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06a7fff640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 0x7f06b01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06aeffd640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b0197ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009230 tx=0x7f06a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.108+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a00037c0 con 0x7f06b00fedf0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.109+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06a0005d50 con 0x7f06b00fedf0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.109+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b019cac0 con 0x7f06b00fedf0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.109+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a0034440 con 0x7f06b00fedf0 2024-04-22T23:57:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.109+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b019cfc0 con 0x7f06b00fedf0 2024-04-22T23:57:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.110+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b004e040 con 0x7f06b00fedf0 2024-04-22T23:57:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.110+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f06a0004dd0 con 0x7f06b00fedf0 2024-04-22T23:57:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.111+0000 7f06b487b640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f069c06ec30 0x7f069c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.111+0000 7f06a7fff640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f069c06ec30 0x7f069c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.111+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f06a00b14a0 con 0x7f06b00fedf0 2024-04-22T23:57:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.111+0000 7f06a7fff640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f069c06ec30 0x7f069c0710f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0698005d00 tx=0x7f0698009550 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.116+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f06a007a040 con 0x7f06b00fedf0 2024-04-22T23:57:55.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:54 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3114276611' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-22T23:57:55.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:54 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2305469975' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-22T23:57:55.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:54 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3181294231' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-22T23:57:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.343+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f06b0109b40 con 0x7f06b00fedf0 2024-04-22T23:57:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.344+0000 7f06b487b640 1 -- 172.21.15.31:0/1717620552 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v40) v1 ==== 107+0+4205 (secure 0 0 0) 0x7f06a00056b0 con 0x7f06b00fedf0 2024-04-22T23:57:55.335 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:55.335 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:44.571279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f069c06ec30 msgr2=0x7f069c0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f069c06ec30 0x7f069c0710f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0698005d00 tx=0x7f0698009550 comp rx=0 tx=0).stop 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 msgr2=0x7f06b0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b0197ea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009230 tx=0x7f06a0004270 comp rx=0 tx=0).stop 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 shutdown_connections 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f069c06ec30 0x7f069c0710f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06b00ff7a0 0x7f06b01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 --2- 172.21.15.31:0/1717620552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b00fedf0 0x7f06b0197ea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 >> 172.21.15.31:0/1717620552 conn(0x7f06b00fa9f0 msgr2=0x7f06b00fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 shutdown_connections 2024-04-22T23:57:55.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.346+0000 7f06b587d640 1 -- 172.21.15.31:0/1717620552 wait complete. 2024-04-22T23:57:55.347 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-04-22T23:57:55.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-22T23:57:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 -- 172.21.15.31:0/2128690540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf00fe9e0 msgr2=0x7ffbf00fedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 --2- 172.21.15.31:0/2128690540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf00fedc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009c20 tx=0x7ffbe002c1b0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 -- 172.21.15.31:0/2128690540 shutdown_connections 2024-04-22T23:57:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 --2- 172.21.15.31:0/2128690540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00ff390 0x7ffbf0104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 --2- 172.21.15.31:0/2128690540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf00fedc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 -- 172.21.15.31:0/2128690540 >> 172.21.15.31:0/2128690540 conn(0x7ffbf00752f0 msgr2=0x7ffbf0075700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 -- 172.21.15.31:0/2128690540 shutdown_connections 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 -- 172.21.15.31:0/2128690540 wait complete. 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 Processor -- start 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 -- start start 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.460+0000 7ffbf827d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf0195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf827d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 0x7ffbf019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf827d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0196950 con 0x7ffbf0196350 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf827d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0196ac0 con 0x7ffbf00fe9e0 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 0x7ffbf019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 0x7ffbf019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51470/0 (socket says 172.21.15.31:51470) 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 -- 172.21.15.31:0/1710658285 learned_addr learned my addr 172.21.15.31:0/1710658285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf6019640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf0195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 -- 172.21.15.31:0/1710658285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00fe9e0 msgr2=0x7ffbf0195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf0195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 -- 172.21.15.31:0/1710658285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe0009900 con 0x7ffbf0196350 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf6019640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf0195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf5818640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 0x7ffbf019a7b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9a0 tx=0x7ffbe400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbe400cc90 con 0x7ffbf0196350 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbe400cdf0 con 0x7ffbf0196350 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbf019adb0 con 0x7ffbf0196350 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbe4010430 con 0x7ffbf0196350 2024-04-22T23:57:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.461+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf019b310 con 0x7ffbf0196350 2024-04-22T23:57:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.462+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbf019b5f0 con 0x7ffbf0196350 2024-04-22T23:57:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.463+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ffbe4005020 con 0x7ffbf0196350 2024-04-22T23:57:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.463+0000 7ffbef7fe640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbd806ec30 0x7ffbd80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.464+0000 7ffbf6019640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbd806ec30 0x7ffbd80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.464+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ffbe4014070 con 0x7ffbf0196350 2024-04-22T23:57:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.464+0000 7ffbf6019640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbd806ec30 0x7ffbd80710f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009430 tx=0x7ffbe0009360 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:55.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.468+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ffbe40581c0 con 0x7ffbf0196350 2024-04-22T23:57:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.694+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7ffbf0197160 con 0x7ffbf0196350 2024-04-22T23:57:55.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.695+0000 7ffbef7fe640 1 -- 172.21.15.31:0/1710658285 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v40) v1 ==== 107+0+4216 (secure 0 0 0) 0x7ffbe405b880 con 0x7ffbf0196350 2024-04-22T23:57:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:55.686 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:44.575741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24309":{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:55.686 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbd806ec30 msgr2=0x7ffbd80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbd806ec30 0x7ffbd80710f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009430 tx=0x7ffbe0009360 comp rx=0 tx=0).stop 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 msgr2=0x7ffbf019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 0x7ffbf019a7b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9a0 tx=0x7ffbe400ee60 comp rx=0 tx=0).stop 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 shutdown_connections 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ffbd806ec30 0x7ffbd80710f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbf0196350 0x7ffbf019a7b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 --2- 172.21.15.31:0/1710658285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffbf00fe9e0 0x7ffbf0195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 >> 172.21.15.31:0/1710658285 conn(0x7ffbf00752f0 msgr2=0x7ffbf00fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 shutdown_connections 2024-04-22T23:57:55.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.697+0000 7ffbf827d640 1 -- 172.21.15.31:0/1710658285 wait complete. 2024-04-22T23:57:55.699 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-04-22T23:57:55.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-22T23:57:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 -- 172.21.15.31:0/888476146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 msgr2=0x7fe1f01000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/888476146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f01000c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009880 tx=0x7fe1e002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 -- 172.21.15.31:0/888476146 shutdown_connections 2024-04-22T23:57:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/888476146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f0100690 0x7fe1f0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/888476146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f01000c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 -- 172.21.15.31:0/888476146 >> 172.21.15.31:0/888476146 conn(0x7fe1f00f9cf0 msgr2=0x7fe1f00fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 -- 172.21.15.31:0/888476146 shutdown_connections 2024-04-22T23:57:55.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.811+0000 7fe1f4d86640 1 -- 172.21.15.31:0/888476146 wait complete. 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1f4d86640 1 Processor -- start 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1f4d86640 1 -- start start 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1f4d86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f0195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1f4d86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f01962f0 0x7fe1f019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1f4d86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01968f0 con 0x7fe1f00ffce0 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1f4d86640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0196a60 con 0x7fe1f01962f0 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f0195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f0195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51494/0 (socket says 172.21.15.31:51494) 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 -- 172.21.15.31:0/2745050800 learned_addr learned my addr 172.21.15.31:0/2745050800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 -- 172.21.15.31:0/2745050800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f01962f0 msgr2=0x7fe1f019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:55.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1e7fff640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f01962f0 0x7fe1f019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f01962f0 0x7fe1f019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 -- 172.21.15.31:0/2745050800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e0009560 con 0x7fe1f00ffce0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.812+0000 7fe1ee59c640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f0195db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009430 tx=0x7fe1e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.813+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e00035f0 con 0x7fe1f00ffce0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.813+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1e0003c10 con 0x7fe1f00ffce0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.813+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f019acf0 con 0x7fe1f00ffce0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.813+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e0034440 con 0x7fe1f00ffce0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.813+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f019b250 con 0x7fe1f00ffce0 2024-04-22T23:57:55.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.814+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f004e040 con 0x7fe1f00ffce0 2024-04-22T23:57:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.814+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe1e0004fd0 con 0x7fe1f00ffce0 2024-04-22T23:57:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.815+0000 7fe1e77fe640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe1dc06ec30 0x7fe1dc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.815+0000 7fe1e7fff640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe1dc06ec30 0x7fe1dc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.815+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe1e00b14d0 con 0x7fe1f00ffce0 2024-04-22T23:57:55.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.815+0000 7fe1e7fff640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe1dc06ec30 0x7fe1dc0710f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80058c0 tx=0x7fe1d8005c60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:55.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:55.820+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe1e007a070 con 0x7fe1f00ffce0 2024-04-22T23:57:56.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.058+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fe1f0100690 con 0x7fe1f00ffce0 2024-04-22T23:57:56.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.059+0000 7fe1e77fe640 1 -- 172.21.15.31:0/2745050800 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v40) v1 ==== 107+0+5063 (secure 0 0 0) 0x7fe1e007d730 con 0x7fe1f00ffce0 2024-04-22T23:57:56.050 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:56.050 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:44.575741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24309":{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:56.050 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe1dc06ec30 msgr2=0x7fe1dc0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe1dc06ec30 0x7fe1dc0710f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80058c0 tx=0x7fe1d8005c60 comp rx=0 tx=0).stop 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 msgr2=0x7fe1f0195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f0195db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009430 tx=0x7fe1e0004040 comp rx=0 tx=0).stop 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 shutdown_connections 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe1dc06ec30 0x7fe1dc0710f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe1f01962f0 0x7fe1f019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 --2- 172.21.15.31:0/2745050800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f00ffce0 0x7fe1f0195db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 >> 172.21.15.31:0/2745050800 conn(0x7fe1f00f9cf0 msgr2=0x7fe1f00fb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:56.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 shutdown_connections 2024-04-22T23:57:56.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.061+0000 7fe1f4d86640 1 -- 172.21.15.31:0/2745050800 wait complete. 2024-04-22T23:57:56.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:55 smithi031 ceph-mon[111289]: pgmap v298: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:56.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:55 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4088642691' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-22T23:57:56.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:55 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1717620552' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-22T23:57:56.062 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:55 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1710658285' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-22T23:57:56.063 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-04-22T23:57:56.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-22T23:57:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 -- 172.21.15.31:0/3313847368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66040ff1a0 msgr2=0x7f66040ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 --2- 172.21.15.31:0/3313847368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66040ff1a0 0x7f66040ff580 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009880 tx=0x7f65f402bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 -- 172.21.15.31:0/3313847368 shutdown_connections 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 --2- 172.21.15.31:0/3313847368 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66040ffb50 0x7f6604103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 --2- 172.21.15.31:0/3313847368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66040ff1a0 0x7f66040ff580 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 -- 172.21.15.31:0/3313847368 >> 172.21.15.31:0/3313847368 conn(0x7f66040fa9e0 msgr2=0x7f66040fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 -- 172.21.15.31:0/3313847368 shutdown_connections 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.176+0000 7f660aa7d640 1 -- 172.21.15.31:0/3313847368 wait complete. 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f660aa7d640 1 Processor -- start 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f660aa7d640 1 -- start start 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f660aa7d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66040ff1a0 0x7f6604100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f660aa7d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 0x7f66041015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f660aa7d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041a2010 con 0x7f6604101170 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f660aa7d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66041a2150 con 0x7f66040ff1a0 2024-04-22T23:57:56.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f6603fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 0x7f66041015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f6603fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 0x7f66041015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51510/0 (socket says 172.21.15.31:51510) 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f6603fff640 1 -- 172.21.15.31:0/2534850095 learned_addr learned my addr 172.21.15.31:0/2534850095 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.177+0000 7f6608819640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66040ff1a0 0x7f6604100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6603fff640 1 -- 172.21.15.31:0/2534850095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66040ff1a0 msgr2=0x7f6604100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6603fff640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66040ff1a0 0x7f6604100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6603fff640 1 -- 172.21.15.31:0/2534850095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4009560 con 0x7f6604101170 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6603fff640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 0x7f66041015d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00e970 tx=0x7f65ec00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65ec009700 con 0x7f6604101170 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65ec0054e0 con 0x7f6604101170 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65ec010430 con 0x7f6604101170 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66041a22f0 con 0x7f6604101170 2024-04-22T23:57:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.178+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66041a2800 con 0x7f6604101170 2024-04-22T23:57:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.179+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660404e040 con 0x7f6604101170 2024-04-22T23:57:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.180+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f65ec00cc90 con 0x7f6604101170 2024-04-22T23:57:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.180+0000 7f6601ffb640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f65f006ec30 0x7f65f00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.180+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f65ec014070 con 0x7f6604101170 2024-04-22T23:57:56.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.180+0000 7f6608819640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f65f006ec30 0x7f65f00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.180+0000 7f6608819640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f65f006ec30 0x7f65f00710f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006010 tx=0x7f65f4005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.185+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f65ec058100 con 0x7f6604101170 2024-04-22T23:57:56.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:55 smithi044 ceph-mon[82146]: pgmap v298: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:57:56.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:55 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4088642691' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-22T23:57:56.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:55 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1717620552' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-22T23:57:56.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:55 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1710658285' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-22T23:57:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.422+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f6604102160 con 0x7f6604101170 2024-04-22T23:57:56.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.422+0000 7f6601ffb640 1 -- 172.21.15.31:0/2534850095 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v40) v1 ==== 107+0+5068 (secure 0 0 0) 0x7f65ec05b7c0 con 0x7f6604101170 2024-04-22T23:57:56.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:56.413 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:54.196336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24309":{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":185,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:56.413 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2024-04-22T23:57:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f65f006ec30 msgr2=0x7f65f00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f65f006ec30 0x7f65f00710f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006010 tx=0x7f65f4005dd0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 msgr2=0x7f66041015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 0x7f66041015d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00e970 tx=0x7f65ec00ee30 comp rx=0 tx=0).stop 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 shutdown_connections 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f65f006ec30 0x7f65f00710f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6604101170 0x7f66041015d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 --2- 172.21.15.31:0/2534850095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f66040ff1a0 0x7f6604100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 >> 172.21.15.31:0/2534850095 conn(0x7f66040fa9e0 msgr2=0x7f6604103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 shutdown_connections 2024-04-22T23:57:56.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.425+0000 7f660aa7d640 1 -- 172.21.15.31:0/2534850095 wait complete. 2024-04-22T23:57:56.426 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-04-22T23:57:56.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-22T23:57:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/3843335191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 msgr2=0x7f4f58101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/3843335191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58101640 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f480098b0 tx=0x7f4f4802bd60 comp rx=0 tx=0).stop 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/3843335191 shutdown_connections 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/3843335191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58101b80 0x7f4f5810a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/3843335191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58101640 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/3843335191 >> 172.21.15.31:0/3843335191 conn(0x7f4f580faa00 msgr2=0x7f4f580fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/3843335191 shutdown_connections 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.538+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/3843335191 wait complete. 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f5d6c7640 1 Processor -- start 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f5d6c7640 1 -- start start 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f5d6c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f5d6c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58101b80 0x7f4f58196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f5d6c7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581969d0 con 0x7f4f58101260 2024-04-22T23:57:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f5d6c7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5819a7a0 con 0x7f4f58101b80 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f56ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51522/0 (socket says 172.21.15.31:51522) 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.539+0000 7f4f56ffd640 1 -- 172.21.15.31:0/4267921942 learned_addr learned my addr 172.21.15.31:0/4267921942 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f567fc640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58101b80 0x7f4f58196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f56ffd640 1 -- 172.21.15.31:0/4267921942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58101b80 msgr2=0x7f4f58196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f56ffd640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58101b80 0x7f4f58196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f56ffd640 1 -- 172.21.15.31:0/4267921942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48009560 con 0x7f4f58101260 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f56ffd640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58195e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009880 tx=0x7f4f4802c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f480035f0 con 0x7f4f58101260 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5819aa20 con 0x7f4f58101260 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f48003c10 con 0x7f4f58101260 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f48005690 con 0x7f4f58101260 2024-04-22T23:57:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.540+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5819af80 con 0x7f4f58101260 2024-04-22T23:57:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.541+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5804e040 con 0x7f4f58101260 2024-04-22T23:57:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.542+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f4f48004ed0 con 0x7f4f58101260 2024-04-22T23:57:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.542+0000 7f4f3ffff640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4f4006ebe0 0x7f4f400710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.542+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f4f480b1c00 con 0x7f4f58101260 2024-04-22T23:57:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.542+0000 7f4f567fc640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4f4006ebe0 0x7f4f400710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.543+0000 7f4f567fc640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4f4006ebe0 0x7f4f400710a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f58197410 tx=0x7f4f4c00b590 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:56.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.547+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f4f480b7020 con 0x7f4f58101260 2024-04-22T23:57:56.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.773+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f4f581971f0 con 0x7f4f58101260 2024-04-22T23:57:56.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.773+0000 7f4f3ffff640 1 -- 172.21.15.31:0/4267921942 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v40) v1 ==== 107+0+5065 (secure 0 0 0) 0x7f4f4807a840 con 0x7f4f58101260 2024-04-22T23:57:56.764 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:56.764 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:54.592056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24309":{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":186,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:56.764 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 29 2024-04-22T23:57:56.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4f4006ebe0 msgr2=0x7f4f400710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4f4006ebe0 0x7f4f400710a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f58197410 tx=0x7f4f4c00b590 comp rx=0 tx=0).stop 2024-04-22T23:57:56.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 msgr2=0x7f4f58195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58195e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009880 tx=0x7f4f4802c8d0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 shutdown_connections 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4f4006ebe0 0x7f4f400710a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4f58101b80 0x7f4f58196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 --2- 172.21.15.31:0/4267921942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f58101260 0x7f4f58195e00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.775+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 >> 172.21.15.31:0/4267921942 conn(0x7f4f580faa00 msgr2=0x7f4f581048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.776+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 shutdown_connections 2024-04-22T23:57:56.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.776+0000 7f4f5d6c7640 1 -- 172.21.15.31:0/4267921942 wait complete. 2024-04-22T23:57:56.777 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-04-22T23:57:56.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 -- 172.21.15.31:0/695497749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 msgr2=0x7fa75810c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/695497749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa75810c390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa748009880 tx=0x7fa74802bd70 comp rx=0 tx=0).stop 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 -- 172.21.15.31:0/695497749 shutdown_connections 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/695497749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa75810c390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/695497749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa758072650 0x7fa758072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 -- 172.21.15.31:0/695497749 >> 172.21.15.31:0/695497749 conn(0x7fa7580fbaa0 msgr2=0x7fa7580fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 -- 172.21.15.31:0/695497749 shutdown_connections 2024-04-22T23:57:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.889+0000 7fa75e9d7640 1 -- 172.21.15.31:0/695497749 wait complete. 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa75e9d7640 1 Processor -- start 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa75e9d7640 1 -- start start 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa75e9d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa758072650 0x7fa758197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa75e9d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa758198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa75e9d7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa758198ab0 con 0x7fa758072f70 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa75e9d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819c880 con 0x7fa758072650 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa7577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa758198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa7577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa758198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51546/0 (socket says 172.21.15.31:51546) 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa7577fe640 1 -- 172.21.15.31:0/2497549530 learned_addr learned my addr 172.21.15.31:0/2497549530 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.890+0000 7fa757fff640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa758072650 0x7fa758197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7577fe640 1 -- 172.21.15.31:0/2497549530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa758072650 msgr2=0x7fa758197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7577fe640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa758072650 0x7fa758197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7577fe640 1 -- 172.21.15.31:0/2497549530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa748009560 con 0x7fa758072f70 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7577fe640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa758198420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa748009230 tx=0x7fa748004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7480035f0 con 0x7fa758072f70 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75819cb00 con 0x7fa758072f70 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa748003c10 con 0x7fa758072f70 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa74803e450 con 0x7fa758072f70 2024-04-22T23:57:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.891+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75819d060 con 0x7fa758072f70 2024-04-22T23:57:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.892+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75804e040 con 0x7fa758072f70 2024-04-22T23:57:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.892+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa748004f60 con 0x7fa758072f70 2024-04-22T23:57:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.893+0000 7fa7557fa640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa74406ec30 0x7fa7440710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.893+0000 7fa757fff640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa74406ec30 0x7fa7440710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.893+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa7480b1b80 con 0x7fa758072f70 2024-04-22T23:57:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.894+0000 7fa757fff640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa74406ec30 0x7fa7440710f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa740005fd0 tx=0x7fa74000b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:56.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:56.898+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa74807a7c0 con 0x7fa758072f70 2024-04-22T23:57:57.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.131+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fa758109a00 con 0x7fa758072f70 2024-04-22T23:57:57.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.132+0000 7fa7557fa640 1 -- 172.21.15.31:0/2497549530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v40) v1 ==== 107+0+5074 (secure 0 0 0) 0x7fa74807de80 con 0x7fa758072f70 2024-04-22T23:57:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:57.123 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:54:56.594662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24309},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24309":{"gid":24309,"name":"cephfs.smithi044.onzoro","rank":0,"incarnation":26,"state":"up:active","state_seq":187,"addr":"172.21.15.44:6827/3065727368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":3065727368},{"type":"v1","addr":"172.21.15.44:6827","nonce":3065727368}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24309,"qdb_cluster":[24309]},"id":1}]} 2024-04-22T23:57:57.123 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 30 2024-04-22T23:57:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa74406ec30 msgr2=0x7fa7440710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa74406ec30 0x7fa7440710f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa740005fd0 tx=0x7fa74000b390 comp rx=0 tx=0).stop 2024-04-22T23:57:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 msgr2=0x7fa758198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa758198420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa748009230 tx=0x7fa748004040 comp rx=0 tx=0).stop 2024-04-22T23:57:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 shutdown_connections 2024-04-22T23:57:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa74406ec30 0x7fa7440710f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa758072f70 0x7fa758198420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 --2- 172.21.15.31:0/2497549530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa758072650 0x7fa758197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 >> 172.21.15.31:0/2497549530 conn(0x7fa7580fbaa0 msgr2=0x7fa758106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:57.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 shutdown_connections 2024-04-22T23:57:57.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.134+0000 7fa75e9d7640 1 -- 172.21.15.31:0/2497549530 wait complete. 2024-04-22T23:57:57.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:56 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2745050800' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-22T23:57:57.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:56 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2534850095' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-22T23:57:57.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:56 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4267921942' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-22T23:57:57.136 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-04-22T23:57:57.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-22T23:57:57.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:56 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2745050800' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-22T23:57:57.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:56 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2534850095' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-22T23:57:57.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:56 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4267921942' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 -- 172.21.15.31:0/485304998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 msgr2=0x7f50100ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 --2- 172.21.15.31:0/485304998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f50100ff470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5000009880 tx=0x7f500002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 -- 172.21.15.31:0/485304998 shutdown_connections 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 --2- 172.21.15.31:0/485304998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f50100ff470 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 --2- 172.21.15.31:0/485304998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5010104fc0 0x7f50101053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 -- 172.21.15.31:0/485304998 >> 172.21.15.31:0/485304998 conn(0x7f5010075b50 msgr2=0x7f5010075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 -- 172.21.15.31:0/485304998 shutdown_connections 2024-04-22T23:57:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.250+0000 7f501874e640 1 -- 172.21.15.31:0/485304998 wait complete. 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f501874e640 1 Processor -- start 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f501874e640 1 -- start start 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f501874e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f5010195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f501874e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5010104fc0 0x7f5010196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f501874e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5010196900 con 0x7f50100ff030 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f501874e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501019a6d0 con 0x7f5010104fc0 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f50164ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f5010195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f50164ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f5010195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51564/0 (socket says 172.21.15.31:51564) 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f50164ea640 1 -- 172.21.15.31:0/3861552374 learned_addr learned my addr 172.21.15.31:0/3861552374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f5015ce9640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5010104fc0 0x7f5010196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f50164ea640 1 -- 172.21.15.31:0/3861552374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5010104fc0 msgr2=0x7f5010196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f50164ea640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5010104fc0 0x7f5010196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.251+0000 7f50164ea640 1 -- 172.21.15.31:0/3861552374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000009560 con 0x7f50100ff030 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.252+0000 7f50164ea640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f5010195d30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e970 tx=0x7f500c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.252+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500c00cc90 con 0x7f50100ff030 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.252+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f500c00cdf0 con 0x7f50100ff030 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.252+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501019a9b0 con 0x7f50100ff030 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.252+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500c010430 con 0x7f50100ff030 2024-04-22T23:57:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.252+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501019aee0 con 0x7f50100ff030 2024-04-22T23:57:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.253+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501004e040 con 0x7f50100ff030 2024-04-22T23:57:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.253+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f500c0046c0 con 0x7f50100ff030 2024-04-22T23:57:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.254+0000 7f50077fe640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4ffc06ec30 0x7f4ffc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.254+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f500c014070 con 0x7f50100ff030 2024-04-22T23:57:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.254+0000 7f5015ce9640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4ffc06ec30 0x7f4ffc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.255+0000 7f5015ce9640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4ffc06ec30 0x7f4ffc0710f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5000006010 tx=0x7f500002cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.259+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f500c058150 con 0x7f50100ff030 2024-04-22T23:57:57.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.489+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f5010197080 con 0x7f50100ff030 2024-04-22T23:57:57.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.490+0000 7f50077fe640 1 -- 172.21.15.31:0/3861552374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v40) v1 ==== 107+0+4273 (secure 0 0 0) 0x7f500c05b810 con 0x7f50100ff030 2024-04-22T23:57:57.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:57.481 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:09.703071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 31 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4ffc06ec30 msgr2=0x7f4ffc0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4ffc06ec30 0x7f4ffc0710f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5000006010 tx=0x7f500002cf90 comp rx=0 tx=0).stop 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 msgr2=0x7f5010195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f5010195d30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f500c00e970 tx=0x7f500c00ed70 comp rx=0 tx=0).stop 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 shutdown_connections 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f4ffc06ec30 0x7f4ffc0710f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5010104fc0 0x7f5010196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 --2- 172.21.15.31:0/3861552374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50100ff030 0x7f5010195d30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 >> 172.21.15.31:0/3861552374 conn(0x7f5010075b50 msgr2=0x7f5010067480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 shutdown_connections 2024-04-22T23:57:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.493+0000 7f501874e640 1 -- 172.21.15.31:0/3861552374 wait complete. 2024-04-22T23:57:57.495 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-04-22T23:57:57.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-22T23:57:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 -- 172.21.15.31:0/3746026893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067c60 msgr2=0x7f001810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 --2- 172.21.15.31:0/3746026893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067c60 0x7f001810c6a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f000c009880 tx=0x7f000c02bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 -- 172.21.15.31:0/3746026893 shutdown_connections 2024-04-22T23:57:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 --2- 172.21.15.31:0/3746026893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067c60 0x7f001810c6a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 --2- 172.21.15.31:0/3746026893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0018067340 0x7f0018067720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 -- 172.21.15.31:0/3746026893 >> 172.21.15.31:0/3746026893 conn(0x7f0018075b50 msgr2=0x7f0018075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 -- 172.21.15.31:0/3746026893 shutdown_connections 2024-04-22T23:57:57.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.607+0000 7f001d1f0640 1 -- 172.21.15.31:0/3746026893 wait complete. 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.608+0000 7f001d1f0640 1 Processor -- start 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f001d1f0640 1 -- start start 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f001d1f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 0x7f0018197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f001d1f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0018067c60 0x7f0018198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f001d1f0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018198ae0 con 0x7f0018067340 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f001d1f0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001819c8b0 con 0x7f0018067c60 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f0016d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 0x7f0018197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f0016d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 0x7f0018197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51588/0 (socket says 172.21.15.31:51588) 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f0016d9d640 1 -- 172.21.15.31:0/2284478213 learned_addr learned my addr 172.21.15.31:0/2284478213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f001659c640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0018067c60 0x7f0018198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f0016d9d640 1 -- 172.21.15.31:0/2284478213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0018067c60 msgr2=0x7f0018198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f0016d9d640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0018067c60 0x7f0018198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.609+0000 7f0016d9d640 1 -- 172.21.15.31:0/2284478213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c009560 con 0x7f0018067340 2024-04-22T23:57:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.610+0000 7f0016d9d640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 0x7f0018197f10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f000800d6f0 tx=0x7f000800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.610+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0008009a30 con 0x7f0018067340 2024-04-22T23:57:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.610+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001819cb90 con 0x7f0018067340 2024-04-22T23:57:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.610+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001819d0c0 con 0x7f0018067340 2024-04-22T23:57:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.610+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00080054e0 con 0x7f0018067340 2024-04-22T23:57:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.610+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0008004e20 con 0x7f0018067340 2024-04-22T23:57:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.611+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001804e040 con 0x7f0018067340 2024-04-22T23:57:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.611+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0008010460 con 0x7f0018067340 2024-04-22T23:57:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.612+0000 7efffffff640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f000006ec30 0x7f00000710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.612+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f000808eaa0 con 0x7f0018067340 2024-04-22T23:57:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.612+0000 7f001659c640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f000006ec30 0x7f00000710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.612+0000 7f001659c640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f000006ec30 0x7f00000710f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f000c006010 tx=0x7f000c005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:57.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.617+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0008057790 con 0x7f0018067340 2024-04-22T23:57:57.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.847+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f0018109d80 con 0x7f0018067340 2024-04-22T23:57:57.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.848+0000 7efffffff640 1 -- 172.21.15.31:0/2284478213 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v40) v1 ==== 107+0+4352 (secure 0 0 0) 0x7f000805ae50 con 0x7f0018067340 2024-04-22T23:57:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:57.838 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:09.707761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:57.839 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 32 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f000006ec30 msgr2=0x7f00000710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f000006ec30 0x7f00000710f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f000c006010 tx=0x7f000c005dd0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 msgr2=0x7f0018197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 0x7f0018197f10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f000800d6f0 tx=0x7f000800dbb0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 shutdown_connections 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f000006ec30 0x7f00000710f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0018067c60 0x7f0018198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 --2- 172.21.15.31:0/2284478213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0018067340 0x7f0018197f10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 >> 172.21.15.31:0/2284478213 conn(0x7f0018075b50 msgr2=0x7f00180fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 shutdown_connections 2024-04-22T23:57:57.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.850+0000 7f001d1f0640 1 -- 172.21.15.31:0/2284478213 wait complete. 2024-04-22T23:57:57.852 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-04-22T23:57:57.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-22T23:57:57.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 -- 172.21.15.31:0/2238425700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8788100280 msgr2=0x7f8788100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2238425700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8788100280 0x7f8788100660 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8780009880 tx=0x7f878002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 -- 172.21.15.31:0/2238425700 shutdown_connections 2024-04-22T23:57:57.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2238425700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8788100c30 0x7f8788104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2238425700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8788100280 0x7f8788100660 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 -- 172.21.15.31:0/2238425700 >> 172.21.15.31:0/2238425700 conn(0x7f87880fbaa0 msgr2=0x7f87880fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 -- 172.21.15.31:0/2238425700 shutdown_connections 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.965+0000 7f878ce5b640 1 -- 172.21.15.31:0/2238425700 wait complete. 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878ce5b640 1 Processor -- start 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878ce5b640 1 -- start start 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878ce5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8788100280 0x7f8788195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878ce5b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 0x7f878819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878ce5b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87881968d0 con 0x7f87881962d0 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878ce5b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8788196a40 con 0x7f8788100280 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f8785d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 0x7f878819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f878659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8788100280 0x7f8788195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f8785d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 0x7f878819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51608/0 (socket says 172.21.15.31:51608) 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f8785d9b640 1 -- 172.21.15.31:0/2557393025 learned_addr learned my addr 172.21.15.31:0/2557393025 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f8785d9b640 1 -- 172.21.15.31:0/2557393025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8788100280 msgr2=0x7f8788195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f8785d9b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8788100280 0x7f8788195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.966+0000 7f8785d9b640 1 -- 172.21.15.31:0/2557393025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780009560 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.967+0000 7f8785d9b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 0x7f878819a730 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f877c00e970 tx=0x7f877c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.967+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f877c009700 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.967+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f877c0054e0 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.967+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878819ad30 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.967+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f877c010430 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.967+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878819b290 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.968+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878819b570 con 0x7f87881962d0 2024-04-22T23:57:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.968+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f877c00cc90 con 0x7f87881962d0 2024-04-22T23:57:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.969+0000 7f87777fe640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f877006ec30 0x7f87700710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.969+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f877c014070 con 0x7f87881962d0 2024-04-22T23:57:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.969+0000 7f878659c640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f877006ec30 0x7f87700710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.969+0000 7f878659c640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f877006ec30 0x7f87700710f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8780006010 tx=0x7f8780005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:57.973+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f877c058100 con 0x7f87881962d0 2024-04-22T23:57:58.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.201+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f87881970e0 con 0x7f87881962d0 2024-04-22T23:57:58.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: pgmap v299: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2497549530' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-22T23:57:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:57:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3861552374' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-22T23:57:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:58.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:57 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2284478213' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-22T23:57:58.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.202+0000 7f87777fe640 1 -- 172.21.15.31:0/2557393025 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v40) v1 ==== 107+0+5199 (secure 0 0 0) 0x7f877c05b7c0 con 0x7f87881962d0 2024-04-22T23:57:58.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:58.194 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:09.707761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 33 2024-04-22T23:57:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f877006ec30 msgr2=0x7f87700710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f877006ec30 0x7f87700710f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8780006010 tx=0x7f8780005dd0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 msgr2=0x7f878819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 0x7f878819a730 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f877c00e970 tx=0x7f877c00ee30 comp rx=0 tx=0).stop 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 shutdown_connections 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f877006ec30 0x7f87700710f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87881962d0 0x7f878819a730 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 --2- 172.21.15.31:0/2557393025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8788100280 0x7f8788195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 >> 172.21.15.31:0/2557393025 conn(0x7f87880fbaa0 msgr2=0x7f87880fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 shutdown_connections 2024-04-22T23:57:58.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.204+0000 7f878ce5b640 1 -- 172.21.15.31:0/2557393025 wait complete. 2024-04-22T23:57:58.206 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-04-22T23:57:58.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: pgmap v299: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2497549530' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3861552374' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:58.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:57 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2284478213' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-22T23:57:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 -- 172.21.15.31:0/409213608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 msgr2=0x7f5fbc100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/409213608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc100660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009880 tx=0x7f5fac02bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 -- 172.21.15.31:0/409213608 shutdown_connections 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/409213608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc100c30 0x7f5fbc104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/409213608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc100660 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 -- 172.21.15.31:0/409213608 >> 172.21.15.31:0/409213608 conn(0x7f5fbc0fbaa0 msgr2=0x7f5fbc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 -- 172.21.15.31:0/409213608 shutdown_connections 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.318+0000 7f5fc15a2640 1 -- 172.21.15.31:0/409213608 wait complete. 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fc15a2640 1 Processor -- start 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fc15a2640 1 -- start start 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fc15a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fc15a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc072ae0 0x7f5fbc072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fc15a2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1a1f70 con 0x7f5fbc100280 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fc15a2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1a20b0 con 0x7f5fbc072ae0 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fbaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fbaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc0725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51640/0 (socket says 172.21.15.31:51640) 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fbaffd640 1 -- 172.21.15.31:0/1257996127 learned_addr learned my addr 172.21.15.31:0/1257996127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fb3fff640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc072ae0 0x7f5fbc072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fbaffd640 1 -- 172.21.15.31:0/1257996127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc072ae0 msgr2=0x7f5fbc072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fbaffd640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc072ae0 0x7f5fbc072f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.319+0000 7f5fbaffd640 1 -- 172.21.15.31:0/1257996127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac009560 con 0x7f5fbc100280 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.320+0000 7f5fbaffd640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc0725a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009430 tx=0x7f5fac005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.320+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fac0044a0 con 0x7f5fbc100280 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.320+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fac003ab0 con 0x7f5fbc100280 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.320+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc1a21f0 con 0x7f5fbc100280 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.320+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fac0055f0 con 0x7f5fbc100280 2024-04-22T23:57:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.320+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc1a26a0 con 0x7f5fbc100280 2024-04-22T23:57:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.321+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc04e040 con 0x7f5fbc100280 2024-04-22T23:57:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.321+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f5fac004e60 con 0x7f5fbc100280 2024-04-22T23:57:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.322+0000 7f5fb8ff9640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5fa806ec30 0x7f5fa80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.322+0000 7f5fb3fff640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5fa806ec30 0x7f5fa80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.322+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f5fac0b1240 con 0x7f5fbc100280 2024-04-22T23:57:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.322+0000 7f5fb3fff640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5fa806ec30 0x7f5fa80710f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0696d0 tx=0x7f5fa4005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:58.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.327+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5fac079f30 con 0x7f5fbc100280 2024-04-22T23:57:58.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.558+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f5fbc108580 con 0x7f5fbc100280 2024-04-22T23:57:58.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.558+0000 7f5fb8ff9640 1 -- 172.21.15.31:0/1257996127 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v40) v1 ==== 107+0+5203 (secure 0 0 0) 0x7f5fac07d5f0 con 0x7f5fbc100280 2024-04-22T23:57:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:58.549 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:19.284517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:58.549 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2024-04-22T23:57:58.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.560+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5fa806ec30 msgr2=0x7f5fa80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.560+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5fa806ec30 0x7f5fa80710f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0696d0 tx=0x7f5fa4005f90 comp rx=0 tx=0).stop 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.560+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 msgr2=0x7f5fbc0725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.560+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc0725a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009430 tx=0x7f5fac005e50 comp rx=0 tx=0).stop 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 shutdown_connections 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f5fa806ec30 0x7f5fa80710f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fbc072ae0 0x7f5fbc072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 --2- 172.21.15.31:0/1257996127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc100280 0x7f5fbc0725a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 >> 172.21.15.31:0/1257996127 conn(0x7f5fbc0fbaa0 msgr2=0x7f5fbc0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 shutdown_connections 2024-04-22T23:57:58.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.561+0000 7f5fc15a2640 1 -- 172.21.15.31:0/1257996127 wait complete. 2024-04-22T23:57:58.563 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-04-22T23:57:58.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-22T23:57:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.675+0000 7f11b1688640 1 -- 172.21.15.31:0/406142480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 msgr2=0x7f11ac100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.675+0000 7f11b1688640 1 --2- 172.21.15.31:0/406142480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac100660 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009880 tx=0x7f11a402bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- 172.21.15.31:0/406142480 shutdown_connections 2024-04-22T23:57:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 --2- 172.21.15.31:0/406142480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11ac100c30 0x7f11ac104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 --2- 172.21.15.31:0/406142480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac100660 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- 172.21.15.31:0/406142480 >> 172.21.15.31:0/406142480 conn(0x7f11ac0fbac0 msgr2=0x7f11ac0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- 172.21.15.31:0/406142480 shutdown_connections 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- 172.21.15.31:0/406142480 wait complete. 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 Processor -- start 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- start start 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11ac072ae0 0x7f11ac072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac1a1f40 con 0x7f11ac100280 2024-04-22T23:57:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.676+0000 7f11b1688640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac1a2080 con 0x7f11ac072ae0 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac0725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51656/0 (socket says 172.21.15.31:51656) 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 -- 172.21.15.31:0/3654769179 learned_addr learned my addr 172.21.15.31:0/3654769179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aa7fc640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11ac072ae0 0x7f11ac072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 -- 172.21.15.31:0/3654769179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11ac072ae0 msgr2=0x7f11ac072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11ac072ae0 0x7f11ac072f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 -- 172.21.15.31:0/3654769179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4009560 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11aaffd640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac0725a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009430 tx=0x7f11a4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a40044a0 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11a4003ab0 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a40055f0 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac1a21c0 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.677+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac1a2670 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.678+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac04e040 con 0x7f11ac100280 2024-04-22T23:57:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.679+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f11a4004e60 con 0x7f11ac100280 2024-04-22T23:57:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.679+0000 7f1193fff640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f119406ec30 0x7f11940710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.679+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f11a40b1310 con 0x7f11ac100280 2024-04-22T23:57:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.679+0000 7f11aa7fc640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f119406ec30 0x7f11940710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.680+0000 7f11aa7fc640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f119406ec30 0x7f11940710f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0696f0 tx=0x7f11a0005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.684+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f11a407a000 con 0x7f11ac100280 2024-04-22T23:57:58.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.912+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f11ac108580 con 0x7f11ac100280 2024-04-22T23:57:58.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.912+0000 7f1193fff640 1 -- 172.21.15.31:0/3654769179 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v40) v1 ==== 107+0+5200 (secure 0 0 0) 0x7f11a407d6c0 con 0x7f11ac100280 2024-04-22T23:57:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:58.903 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:19.914631+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:57:58.903 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 35 2024-04-22T23:57:58.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f119406ec30 msgr2=0x7f11940710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f119406ec30 0x7f11940710f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0696f0 tx=0x7f11a0005f90 comp rx=0 tx=0).stop 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 msgr2=0x7f11ac0725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac0725a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009430 tx=0x7f11a4005e50 comp rx=0 tx=0).stop 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 shutdown_connections 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f119406ec30 0x7f11940710f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11ac072ae0 0x7f11ac072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 --2- 172.21.15.31:0/3654769179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ac100280 0x7f11ac0725a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 >> 172.21.15.31:0/3654769179 conn(0x7f11ac0fbac0 msgr2=0x7f11ac0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 shutdown_connections 2024-04-22T23:57:58.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:58.915+0000 7f11b1688640 1 -- 172.21.15.31:0/3654769179 wait complete. 2024-04-22T23:57:58.916 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-04-22T23:57:58.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-22T23:57:59.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 -- 172.21.15.31:0/1984670197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ffb60 msgr2=0x7f71c8103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/1984670197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ffb60 0x7f71c8103ba0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009880 tx=0x7f71b002bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 -- 172.21.15.31:0/1984670197 shutdown_connections 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/1984670197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ffb60 0x7f71c8103ba0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/1984670197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c80ff1b0 0x7f71c80ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 -- 172.21.15.31:0/1984670197 >> 172.21.15.31:0/1984670197 conn(0x7f71c80fa9f0 msgr2=0x7f71c80fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 -- 172.21.15.31:0/1984670197 shutdown_connections 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.028+0000 7f71cd9e4640 1 -- 172.21.15.31:0/1984670197 wait complete. 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71cd9e4640 1 Processor -- start 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71cd9e4640 1 -- start start 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71cd9e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 0x7f71c8194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71cd9e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c8195210 0x7f71c81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71cd9e4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8195810 con 0x7f71c80ff1b0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71cd9e4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8195950 con 0x7f71c8195210 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 0x7f71c8194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 0x7f71c8194cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51680/0 (socket says 172.21.15.31:51680) 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 -- 172.21.15.31:0/2311960622 learned_addr learned my addr 172.21.15.31:0/2311960622 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71bffff640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c8195210 0x7f71c81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 -- 172.21.15.31:0/2311960622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c8195210 msgr2=0x7f71c81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c8195210 0x7f71c81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 -- 172.21.15.31:0/2311960622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0009560 con 0x7f71c80ff1b0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.029+0000 7f71c6ffd640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 0x7f71c8194cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d8d0 tx=0x7f71b800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.030+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71b800bea0 con 0x7f71c80ff1b0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.030+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71b8009c00 con 0x7f71c80ff1b0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.030+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71b800f710 con 0x7f71c80ff1b0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.030+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c8198bc0 con 0x7f71c80ff1b0 2024-04-22T23:57:59.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.030+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c8199120 con 0x7f71c80ff1b0 2024-04-22T23:57:59.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.031+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c81993f0 con 0x7f71c80ff1b0 2024-04-22T23:57:59.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.031+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f71b8010460 con 0x7f71c80ff1b0 2024-04-22T23:57:59.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.032+0000 7f71cc9e2640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f71b406ec30 0x7f71b40710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.032+0000 7f71bffff640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f71b406ec30 0x7f71b40710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.032+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f71b808ec50 con 0x7f71c80ff1b0 2024-04-22T23:57:59.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.032+0000 7f71bffff640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f71b406ec30 0x7f71b40710f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006010 tx=0x7f71b0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.037+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f71b8057910 con 0x7f71c80ff1b0 2024-04-22T23:57:59.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:58 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2557393025' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-22T23:57:59.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:57:59.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:57:59.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:58 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:59.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:58 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1257996127' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-22T23:57:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:58 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2557393025' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-22T23:57:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:57:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:57:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:58 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:57:59.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:58 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1257996127' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-22T23:57:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.266+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f71c804e040 con 0x7f71c80ff1b0 2024-04-22T23:57:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.267+0000 7f71cc9e2640 1 -- 172.21.15.31:0/2311960622 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v40) v1 ==== 107+0+5209 (secure 0 0 0) 0x7f71b805afd0 con 0x7f71c80ff1b0 2024-04-22T23:57:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:59.257 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:21.918098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-04-22T23:57:59.258 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2024-04-22T23:57:59.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f71b406ec30 msgr2=0x7f71b40710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f71b406ec30 0x7f71b40710f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006010 tx=0x7f71b0000bc0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 msgr2=0x7f71c8194cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 0x7f71c8194cd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d8d0 tx=0x7f71b800dd90 comp rx=0 tx=0).stop 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 shutdown_connections 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f71b406ec30 0x7f71b40710f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c8195210 0x7f71c81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 --2- 172.21.15.31:0/2311960622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80ff1b0 0x7f71c8194cd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 >> 172.21.15.31:0/2311960622 conn(0x7f71c80fa9f0 msgr2=0x7f71c80fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 shutdown_connections 2024-04-22T23:57:59.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.269+0000 7f71cd9e4640 1 -- 172.21.15.31:0/2311960622 wait complete. 2024-04-22T23:57:59.271 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-04-22T23:57:59.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 -- 172.21.15.31:0/1640986537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 msgr2=0x7f4128101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 --2- 172.21.15.31:0/1640986537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f4128101650 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f411c0098b0 tx=0x7f411c02bd60 comp rx=0 tx=0).stop 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 -- 172.21.15.31:0/1640986537 shutdown_connections 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 --2- 172.21.15.31:0/1640986537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4128101b90 0x7f412810a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 --2- 172.21.15.31:0/1640986537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f4128101650 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 -- 172.21.15.31:0/1640986537 >> 172.21.15.31:0/1640986537 conn(0x7f41280fa9f0 msgr2=0x7f41280fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 -- 172.21.15.31:0/1640986537 shutdown_connections 2024-04-22T23:57:59.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.382+0000 7f412f236640 1 -- 172.21.15.31:0/1640986537 wait complete. 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412f236640 1 Processor -- start 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412f236640 1 -- start start 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412f236640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f41280feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412f236640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4128101b90 0x7f41280ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412f236640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280ff940 con 0x7f4128101270 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412f236640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280ffab0 con 0x7f4128101b90 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412cfd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f41280feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412cfd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f41280feec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51698/0 (socket says 172.21.15.31:51698) 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412cfd2640 1 -- 172.21.15.31:0/2335630712 learned_addr learned my addr 172.21.15.31:0/2335630712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f4127fff640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4128101b90 0x7f41280ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412cfd2640 1 -- 172.21.15.31:0/2335630712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4128101b90 msgr2=0x7f41280ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412cfd2640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4128101b90 0x7f41280ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.383+0000 7f412cfd2640 1 -- 172.21.15.31:0/2335630712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009560 con 0x7f4128101270 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.384+0000 7f412cfd2640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f41280feec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f411c009880 tx=0x7f411c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.384+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411c0037c0 con 0x7f4128101270 2024-04-22T23:57:59.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.384+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f411c003de0 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.384+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41280ffcd0 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.384+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411c03e450 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.384+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41281a22b0 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.385+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412804e040 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.385+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f411c004fd0 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.386+0000 7f4125ffb640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f411806ec30 0x7f41180710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.386+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f411c0b1b10 con 0x7f4128101270 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.386+0000 7f4127fff640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f411806ec30 0x7f41180710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.386+0000 7f4127fff640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f411806ec30 0x7f41180710f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4114005fd0 tx=0x7f4114005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:59.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.391+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f411c07a7d0 con 0x7f4128101270 2024-04-22T23:57:59.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.618+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f41281076e0 con 0x7f4128101270 2024-04-22T23:57:59.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.618+0000 7f4125ffb640 1 -- 172.21.15.31:0/2335630712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v40) v1 ==== 107+0+5209 (secure 0 0 0) 0x7f411c07de90 con 0x7f4128101270 2024-04-22T23:57:59.609 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:59.609 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:26.582136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-04-22T23:57:59.609 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 37 2024-04-22T23:57:59.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.620+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f411806ec30 msgr2=0x7f41180710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.620+0000 7f412f236640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f411806ec30 0x7f41180710f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4114005fd0 tx=0x7f4114005e20 comp rx=0 tx=0).stop 2024-04-22T23:57:59.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.620+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 msgr2=0x7f41280feec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.620+0000 7f412f236640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f41280feec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f411c009880 tx=0x7f411c004270 comp rx=0 tx=0).stop 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.620+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 shutdown_connections 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.621+0000 7f412f236640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f411806ec30 0x7f41180710f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.621+0000 7f412f236640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4128101b90 0x7f41280ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.621+0000 7f412f236640 1 --2- 172.21.15.31:0/2335630712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4128101270 0x7f41280feec0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.621+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 >> 172.21.15.31:0/2335630712 conn(0x7f41280fa9f0 msgr2=0x7f41281048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.621+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 shutdown_connections 2024-04-22T23:57:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.621+0000 7f412f236640 1 -- 172.21.15.31:0/2335630712 wait complete. 2024-04-22T23:57:59.622 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-04-22T23:57:59.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-04-22T23:57:59.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 -- 172.21.15.31:0/659789475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440ff440 msgr2=0x7f0844104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 --2- 172.21.15.31:0/659789475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440ff440 0x7f0844104cf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f082c009880 tx=0x7f082c02bce0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 -- 172.21.15.31:0/659789475 shutdown_connections 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 --2- 172.21.15.31:0/659789475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440ff440 0x7f0844104cf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 --2- 172.21.15.31:0/659789475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08440fea90 0x7f08440fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 -- 172.21.15.31:0/659789475 >> 172.21.15.31:0/659789475 conn(0x7f0844074120 msgr2=0x7f0844074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 -- 172.21.15.31:0/659789475 shutdown_connections 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.735+0000 7f084930c640 1 -- 172.21.15.31:0/659789475 wait complete. 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f084930c640 1 Processor -- start 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f084930c640 1 -- start start 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f084930c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 0x7f0844195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f084930c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0844196260 0x7f084419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f084930c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0844196860 con 0x7f08440fea90 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f084930c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08441969d0 con 0x7f0844196260 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 0x7f0844195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 0x7f0844195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51708/0 (socket says 172.21.15.31:51708) 2024-04-22T23:57:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 -- 172.21.15.31:0/2228699168 learned_addr learned my addr 172.21.15.31:0/2228699168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f083bfff640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0844196260 0x7f084419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 -- 172.21.15.31:0/2228699168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0844196260 msgr2=0x7f084419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0844196260 0x7f084419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 -- 172.21.15.31:0/2228699168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c009560 con 0x7f08440fea90 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0842ffd640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 0x7f0844195d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f083400d720 tx=0x7f083400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.736+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08340051f0 con 0x7f08440fea90 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.737+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0834009ce0 con 0x7f08440fea90 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.737+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084419acc0 con 0x7f08440fea90 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.737+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0834010460 con 0x7f08440fea90 2024-04-22T23:57:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.737+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084419b2a0 con 0x7f08440fea90 2024-04-22T23:57:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.738+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084404e040 con 0x7f08440fea90 2024-04-22T23:57:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.738+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f0834005350 con 0x7f08440fea90 2024-04-22T23:57:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.738+0000 7f0840ff9640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f083006ec30 0x7f08300710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:57:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.738+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f083408ead0 con 0x7f08440fea90 2024-04-22T23:57:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.739+0000 7f083bfff640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f083006ec30 0x7f08300710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:57:59.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.739+0000 7f083bfff640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f083006ec30 0x7f08300710f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f082c009230 tx=0x7f082c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:57:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.743+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0834057790 con 0x7f08440fea90 2024-04-22T23:57:59.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.973+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f0844197110 con 0x7f08440fea90 2024-04-22T23:57:59.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.974+0000 7f0840ff9640 1 -- 172.21.15.31:0/2228699168 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v40) v1 ==== 107+0+5208 (secure 0 0 0) 0x7f0834017090 con 0x7f08440fea90 2024-04-22T23:57:59.965 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:57:59.965 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:27.584750+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-04-22T23:57:59.965 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2024-04-22T23:57:59.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f083006ec30 msgr2=0x7f08300710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f083006ec30 0x7f08300710f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f082c009230 tx=0x7f082c005fd0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 msgr2=0x7f0844195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 0x7f0844195d20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f083400d720 tx=0x7f083400dbe0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 shutdown_connections 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f083006ec30 0x7f08300710f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0844196260 0x7f084419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 --2- 172.21.15.31:0/2228699168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08440fea90 0x7f0844195d20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 >> 172.21.15.31:0/2228699168 conn(0x7f0844074120 msgr2=0x7f08440fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 shutdown_connections 2024-04-22T23:57:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:57:59.976+0000 7f084930c640 1 -- 172.21.15.31:0/2228699168 wait complete. 2024-04-22T23:57:59.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-04-22T23:58:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 -- 172.21.15.31:0/1397869976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1280ff030 msgr2=0x7ff1280ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/1397869976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1280ff030 0x7ff1280ff470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff110009880 tx=0x7ff11002bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 -- 172.21.15.31:0/1397869976 shutdown_connections 2024-04-22T23:58:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/1397869976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1280ff030 0x7ff1280ff470 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/1397869976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff128104fc0 0x7ff1281053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 -- 172.21.15.31:0/1397869976 >> 172.21.15.31:0/1397869976 conn(0x7ff128075b50 msgr2=0x7ff128075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.089+0000 7ff12ebbc640 1 -- 172.21.15.31:0/1397869976 shutdown_connections 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 -- 172.21.15.31:0/1397869976 wait complete. 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 Processor -- start 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 -- start start 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1280ff030 0x7ff128196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 0x7ff1281970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128197600 con 0x7ff128104fc0 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff12ebbc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128197740 con 0x7ff1280ff030 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff127fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 0x7ff1281970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff127fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 0x7ff1281970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51734/0 (socket says 172.21.15.31:51734) 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.090+0000 7ff127fff640 1 -- 172.21.15.31:0/2867808502 learned_addr learned my addr 172.21.15.31:0/2867808502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff12c958640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1280ff030 0x7ff128196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff127fff640 1 -- 172.21.15.31:0/2867808502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1280ff030 msgr2=0x7ff128196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff127fff640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1280ff030 0x7ff128196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff127fff640 1 -- 172.21.15.31:0/2867808502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff110009560 con 0x7ff128104fc0 2024-04-22T23:58:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff127fff640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 0x7ff1281970c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff110009230 tx=0x7ff110004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1100035f0 con 0x7ff128104fc0 2024-04-22T23:58:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff110003c10 con 0x7ff128104fc0 2024-04-22T23:58:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1281979c0 con 0x7ff128104fc0 2024-04-22T23:58:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff110034440 con 0x7ff128104fc0 2024-04-22T23:58:00.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.091+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12819ff30 con 0x7ff128104fc0 2024-04-22T23:58:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.092+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12804e040 con 0x7ff128104fc0 2024-04-22T23:58:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.093+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7ff110004f60 con 0x7ff128104fc0 2024-04-22T23:58:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.093+0000 7ff125ffb640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff11406ec30 0x7ff1140710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.093+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7ff1100b1430 con 0x7ff128104fc0 2024-04-22T23:58:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.094+0000 7ff12c958640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff11406ec30 0x7ff1140710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.094+0000 7ff12c958640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff11406ec30 0x7ff1140710f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff11800abe0 tx=0x7ff11800a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:00.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.098+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff110079fd0 con 0x7ff128104fc0 2024-04-22T23:58:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:59 smithi031 ceph-mon[111289]: pgmap v300: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:59 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/3654769179' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-22T23:58:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:59 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2311960622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-22T23:58:00.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:57:59 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2335630712' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-22T23:58:00.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:59 smithi044 ceph-mon[82146]: pgmap v300: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:00.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:59 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/3654769179' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-22T23:58:00.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:59 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2311960622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-22T23:58:00.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:57:59 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2335630712' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-22T23:58:00.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.324+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7ff128107440 con 0x7ff128104fc0 2024-04-22T23:58:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.324+0000 7ff125ffb640 1 -- 172.21.15.31:0/2867808502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v40) v1 ==== 107+0+5215 (secure 0 0 0) 0x7ff11009ce60 con 0x7ff128104fc0 2024-04-22T23:58:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:58:00.315 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi044.ibkztg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6825/4021256007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":4021256007},{"type":"v1","addr":"172.21.15.44:6825","nonce":4021256007}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44345,"name":"cephfs.smithi044.onzoro","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6827/2995756226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2995756226},{"type":"v1","addr":"172.21.15.44:6827","nonce":2995756226}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T23:42:26.948183+0000","modified":"2024-04-22T23:55:28.588901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34406":{"gid":34406,"name":"cephfs.smithi031.ezuwxk","rank":0,"incarnation":32,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/3446954000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3446954000},{"type":"v1","addr":"172.21.15.31:6827","nonce":3446954000}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi031.usiyoa","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6829/1426404610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1426404610},{"type":"v1","addr":"172.21.15.31:6829","nonce":1426404610}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34406,"qdb_cluster":[34406]},"id":1}]} 2024-04-22T23:58:00.315 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 39 2024-04-22T23:58:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff11406ec30 msgr2=0x7ff1140710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff11406ec30 0x7ff1140710f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff11800abe0 tx=0x7ff11800a2d0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 msgr2=0x7ff1281970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 0x7ff1281970c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff110009230 tx=0x7ff110004040 comp rx=0 tx=0).stop 2024-04-22T23:58:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 shutdown_connections 2024-04-22T23:58:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7ff11406ec30 0x7ff1140710f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128104fc0 0x7ff1281970c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 --2- 172.21.15.31:0/2867808502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1280ff030 0x7ff128196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.326+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 >> 172.21.15.31:0/2867808502 conn(0x7ff128075b50 msgr2=0x7ff1280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.327+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 shutdown_connections 2024-04-22T23:58:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.327+0000 7ff12ebbc640 1 -- 172.21.15.31:0/2867808502 wait complete. 2024-04-22T23:58:00.328 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-22T23:58:00.339 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-22T23:58:00.339 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:58:00.340 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:58:00.357 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:58:00.357 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:58:00.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 -- 172.21.15.31:0/706385014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 msgr2=0x7f86c8067990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/706385014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c8067990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009880 tx=0x7f86c402bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 -- 172.21.15.31:0/706385014 shutdown_connections 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/706385014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c8067f60 0x7f86c8104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/706385014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c8067990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 -- 172.21.15.31:0/706385014 >> 172.21.15.31:0/706385014 conn(0x7f86c8075bb0 msgr2=0x7f86c8075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 -- 172.21.15.31:0/706385014 shutdown_connections 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 -- 172.21.15.31:0/706385014 wait complete. 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.564+0000 7f86cfb7a640 1 Processor -- start 2024-04-22T23:58:00.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cfb7a640 1 -- start start 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cfb7a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cfb7a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c819c020 0x7f86c8195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cfb7a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c819c560 con 0x7f86c80675b0 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cfb7a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c819c6a0 con 0x7f86c819c020 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51748/0 (socket says 172.21.15.31:51748) 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 -- 172.21.15.31:0/637298331 learned_addr learned my addr 172.21.15.31:0/637298331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd115640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c819c020 0x7f86c8195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 -- 172.21.15.31:0/637298331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c819c020 msgr2=0x7f86c8195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c819c020 0x7f86c8195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 -- 172.21.15.31:0/637298331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4009560 con 0x7f86c80675b0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86cd916640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c819bae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009230 tx=0x7f86c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.565+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86c40035f0 con 0x7f86c80675b0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.566+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86c4003c10 con 0x7f86c80675b0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.566+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c81960f0 con 0x7f86c80675b0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.566+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86c4034440 con 0x7f86c80675b0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.566+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c81966d0 con 0x7f86c80675b0 2024-04-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.567+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c804e040 con 0x7f86c80675b0 2024-04-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.567+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f86c4004f60 con 0x7f86c80675b0 2024-04-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.567+0000 7f86beffd640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f86b406ec30 0x7f86b40710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.568+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f86c40b1410 con 0x7f86c80675b0 2024-04-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.568+0000 7f86cd115640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f86b406ec30 0x7f86b40710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.568+0000 7f86cd115640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f86b406ec30 0x7f86b40710f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86c8068370 tx=0x7f86b800a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:00.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.572+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f86c4079fb0 con 0x7f86c80675b0 2024-04-22T23:58:00.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.761+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c8067f60 con 0x7f86c80675b0 2024-04-22T23:58:00.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.763+0000 7f86beffd640 1 -- 172.21.15.31:0/637298331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17209 (secure 0 0 0) 0x7f86c407d670 con 0x7f86c80675b0 2024-04-22T23:58:00.754 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:58:00.754 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":92,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:55:09.702849+0000","last_up_change":"2024-04-22T23:54:01.112143+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:42:25.830464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:42:26.380171+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":85,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6803","nonce":3402089089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6805","nonce":3402089089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6809","nonce":3402089089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6807","nonce":3402089089}]},"public_addr":"172.21.15.31:6803/3402089089","cluster_addr":"172.21.15.31:6805/3402089089","heartbeat_back_addr":"172.21.15.31:6809/3402089089","heartbeat_front_addr":"172.21.15.31:6807/3402089089","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":62,"up_thru":85,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6811","nonce":753330875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6813","nonce":753330875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6817","nonce":753330875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6815","nonce":753330875}]},"public_addr":"172.21.15.31:6811/753330875","cluster_addr":"172.21.15.31:6813/753330875","heartbeat_back_addr":"172.21.15.31:6817/753330875","heartbeat_front_addr":"172.21.15.31:6815/753330875","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":66,"up_from":70,"up_thru":85,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6819","nonce":1746177235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6821","nonce":1746177235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6825","nonce":1746177235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6823","nonce":1746177235}]},"public_addr":"172.21.15.31:6819/1746177235","cluster_addr":"172.21.15.31:6821/1746177235","heartbeat_back_addr":"172.21.15.31:6825/1746177235","heartbeat_front_addr":"172.21.15.31:6823/1746177235","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":71,"up_from":75,"up_thru":85,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6801","nonce":2576217506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6803","nonce":2576217506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6807","nonce":2576217506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6805","nonce":2576217506}]},"public_addr":"172.21.15.44:6801/2576217506","cluster_addr":"172.21.15.44:6803/2576217506","heartbeat_back_addr":"172.21.15.44:6807/2576217506","heartbeat_front_addr":"172.21.15.44:6805/2576217506","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":76,"up_from":80,"up_thru":85,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6809","nonce":2616608091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6811","nonce":2616608091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6815","nonce":2616608091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6813","nonce":2616608091}]},"public_addr":"172.21.15.44:6809/2616608091","cluster_addr":"172.21.15.44:6811/2616608091","heartbeat_back_addr":"172.21.15.44:6815/2616608091","heartbeat_front_addr":"172.21.15.44:6813/2616608091","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":81,"up_from":85,"up_thru":85,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6817","nonce":2666243870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6819","nonce":2666243870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6823","nonce":2666243870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6821","nonce":2666243870}]},"public_addr":"172.21.15.44:6817/2666243870","cluster_addr":"172.21.15.44:6819/2666243870","heartbeat_back_addr":"172.21.15.44:6823/2666243870","heartbeat_front_addr":"172.21.15.44:6821/2666243870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":66},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":71},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":76},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":81}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1","mappings":[{"from":0,"to":5}]},{"pgid":"2.7","mappings":[{"from":3,"to":1}]},{"pgid":"2.c","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1d","mappings":[{"from":0,"to":2}]},{"pgid":"2.1f","mappings":[{"from":0,"to":1}]},{"pgid":"3.a","mappings":[{"from":3,"to":0}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":4,"to":5}]},{"pgid":"3.1b","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6825/1404295013":"2024-04-23T23:54:44.571005+0000","172.21.15.31:6801/418479598":"2024-04-23T23:49:30.073919+0000","172.21.15.44:0/1074135904":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6801/2402370856":"2024-04-23T23:44:52.608776+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.44:6827/3065727368":"2024-04-23T23:55:09.702810+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3574495240":"2024-04-23T23:44:52.608776+0000","172.21.15.44:0/2312976544":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.44:6828/170975257":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/1857354965":"2024-04-23T23:44:52.608776+0000","172.21.15.44:6829/170975257":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/3332075151":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/127255297":"2024-04-23T23:44:52.608776+0000","172.21.15.31:6800/2402370856":"2024-04-23T23:44:52.608776+0000","172.21.15.44:0/3035083544":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6826/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000","172.21.15.44:6826/3065727368":"2024-04-23T23:55:09.702810+0000","172.21.15.44:0/2034571521":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6827/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.44:0/3936244184":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/1029776087":"2024-04-23T23:44:52.608776+0000","172.21.15.44:6824/1404295013":"2024-04-23T23:54:44.571005+0000","172.21.15.44:0/3485102811":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/1913595989":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/445726858":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/2085576080":"2024-04-23T23:49:30.073919+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/418479598":"2024-04-23T23:49:30.073919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:58:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f86b406ec30 msgr2=0x7f86b40710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f86b406ec30 0x7f86b40710f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f86c8068370 tx=0x7f86b800a2d0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 msgr2=0x7f86c819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c819bae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009230 tx=0x7f86c4004040 comp rx=0 tx=0).stop 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 shutdown_connections 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f86b406ec30 0x7f86b40710f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86c819c020 0x7f86c8195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 --2- 172.21.15.31:0/637298331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86c80675b0 0x7f86c819bae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 >> 172.21.15.31:0/637298331 conn(0x7f86c8075bb0 msgr2=0x7f86c80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 shutdown_connections 2024-04-22T23:58:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.765+0000 7f86cfb7a640 1 -- 172.21.15.31:0/637298331 wait complete. 2024-04-22T23:58:00.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:39:59.535490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:42:25.830464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:42:26.380171+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:58:00.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 -- 172.21.15.31:0/2584328307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 msgr2=0x7fe3fc0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 --2- 172.21.15.31:0/2584328307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc0fee70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009880 tx=0x7fe3f802bd10 comp rx=0 tx=0).stop 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 -- 172.21.15.31:0/2584328307 shutdown_connections 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 --2- 172.21.15.31:0/2584328307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc0ff440 0x7fe3fc104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 --2- 172.21.15.31:0/2584328307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc0fee70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 -- 172.21.15.31:0/2584328307 >> 172.21.15.31:0/2584328307 conn(0x7fe3fc074120 msgr2=0x7fe3fc074530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 -- 172.21.15.31:0/2584328307 shutdown_connections 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.880+0000 7fe40334e640 1 -- 172.21.15.31:0/2584328307 wait complete. 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe40334e640 1 Processor -- start 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe40334e640 1 -- start start 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe40334e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe40334e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc196330 0x7fe3fc19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe40334e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc196930 con 0x7fe3fc0fea90 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe40334e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc196aa0 con 0x7fe3fc196330 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe4010ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe4010ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42386/0 (socket says 172.21.15.31:42386) 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.881+0000 7fe4010ea640 1 -- 172.21.15.31:0/2055330150 learned_addr learned my addr 172.21.15.31:0/2055330150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe4008e9640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc196330 0x7fe3fc19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe4010ea640 1 -- 172.21.15.31:0/2055330150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc196330 msgr2=0x7fe3fc19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe4010ea640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc196330 0x7fe3fc19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe4010ea640 1 -- 172.21.15.31:0/2055330150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009560 con 0x7fe3fc0fea90 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe4010ea640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc195df0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009430 tx=0x7fe3f8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f80035f0 con 0x7fe3fc0fea90 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3f8003c10 con 0x7fe3fc0fea90 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc19ad90 con 0x7fe3fc0fea90 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3f80344e0 con 0x7fe3fc0fea90 2024-04-22T23:58:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.882+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc19b370 con 0x7fe3fc0fea90 2024-04-22T23:58:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.883+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc19b590 con 0x7fe3fc0fea90 2024-04-22T23:58:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.884+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fe3f8004fd0 con 0x7fe3fc0fea90 2024-04-22T23:58:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.884+0000 7fe3f27fc640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe3e806ec30 0x7fe3e80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.884+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fe3f80b1b50 con 0x7fe3fc0fea90 2024-04-22T23:58:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.884+0000 7fe4008e9640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe3e806ec30 0x7fe3e80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.884+0000 7fe4008e9640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe3e806ec30 0x7fe3e80710f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc072510 tx=0x7fe3ec008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:00.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:00.889+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe3f807a790 con 0x7fe3fc0fea90 2024-04-22T23:58:01.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.098+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3fc1085e0 con 0x7fe3fc0fea90 2024-04-22T23:58:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.098+0000 7fe3f27fc640 1 -- 172.21.15.31:0/2055330150 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3f807de50 con 0x7fe3fc0fea90 2024-04-22T23:58:01.089 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-22T23:58:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe3e806ec30 msgr2=0x7fe3e80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe3e806ec30 0x7fe3e80710f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc072510 tx=0x7fe3ec008040 comp rx=0 tx=0).stop 2024-04-22T23:58:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 msgr2=0x7fe3fc195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc195df0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009430 tx=0x7fe3f8004040 comp rx=0 tx=0).stop 2024-04-22T23:58:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 shutdown_connections 2024-04-22T23:58:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fe3e806ec30 0x7fe3e80710f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3fc196330 0x7fe3fc19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.100+0000 7fe40334e640 1 --2- 172.21.15.31:0/2055330150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3fc0fea90 0x7fe3fc195df0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.101+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 >> 172.21.15.31:0/2055330150 conn(0x7fe3fc074120 msgr2=0x7fe3fc0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.101+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 shutdown_connections 2024-04-22T23:58:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.101+0000 7fe40334e640 1 -- 172.21.15.31:0/2055330150 wait complete. 2024-04-22T23:58:01.101 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:00 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2228699168' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-22T23:58:01.101 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:01.101 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:00 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2867808502' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-22T23:58:01.101 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:00 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/637298331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:58:01.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 -- 172.21.15.31:0/3776476301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78100c30 msgr2=0x7f3d78104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/3776476301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78100c30 0x7f3d78104c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009880 tx=0x7f3d6802bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 -- 172.21.15.31:0/3776476301 shutdown_connections 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/3776476301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78100c30 0x7f3d78104c70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/3776476301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d78100280 0x7f3d78100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 -- 172.21.15.31:0/3776476301 >> 172.21.15.31:0/3776476301 conn(0x7f3d780fbaa0 msgr2=0x7f3d780fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 -- 172.21.15.31:0/3776476301 shutdown_connections 2024-04-22T23:58:01.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.216+0000 7f3d7f62f640 1 -- 172.21.15.31:0/3776476301 wait complete. 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7f62f640 1 Processor -- start 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7f62f640 1 -- start start 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7f62f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d78100280 0x7f3d780725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7f62f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 0x7f3d78072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7f62f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d781a1f20 con 0x7f3d78072ae0 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7f62f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d781a2060 con 0x7f3d78100280 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7cbca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 0x7f3d78072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7cbca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 0x7f3d78072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42396/0 (socket says 172.21.15.31:42396) 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7cbca640 1 -- 172.21.15.31:0/1910777334 learned_addr learned my addr 172.21.15.31:0/1910777334 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7d3cb640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d78100280 0x7f3d780725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7cbca640 1 -- 172.21.15.31:0/1910777334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d78100280 msgr2=0x7f3d780725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7cbca640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d78100280 0x7f3d780725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.217+0000 7f3d7cbca640 1 -- 172.21.15.31:0/1910777334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009560 con 0x7f3d78072ae0 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.218+0000 7f3d7cbca640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 0x7f3d78072f40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3d68036040 tx=0x7f3d6802c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.218+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d680035f0 con 0x7f3d78072ae0 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.218+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d781a21a0 con 0x7f3d78072ae0 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.218+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d68003c10 con 0x7f3d78072ae0 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.218+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d68005630 con 0x7f3d78072ae0 2024-04-22T23:58:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.218+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d781a2650 con 0x7f3d78072ae0 2024-04-22T23:58:01.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:00 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2228699168' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-22T23:58:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:00 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2867808502' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-22T23:58:01.211 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:00 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/637298331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:58:01.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.219+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7804e040 con 0x7f3d78072ae0 2024-04-22T23:58:01.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.220+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f3d68003750 con 0x7f3d78072ae0 2024-04-22T23:58:01.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.220+0000 7f3d6e7fc640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d6406ec30 0x7f3d640710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.220+0000 7f3d7d3cb640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d6406ec30 0x7f3d640710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.220+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f3d680b1390 con 0x7f3d78072ae0 2024-04-22T23:58:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.221+0000 7f3d7d3cb640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d6406ec30 0x7f3d640710f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009bd0 tx=0x7f3d74005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:01.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.225+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f3d68079fe0 con 0x7f3d78072ae0 2024-04-22T23:58:01.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.413+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f3d78108580 con 0x7f3d78072ae0 2024-04-22T23:58:01.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.413+0000 7f3d6e7fc640 1 -- 172.21.15.31:0/1910777334 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3d6807d6a0 con 0x7f3d78072ae0 2024-04-22T23:58:01.404 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:58:01.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.415+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d6406ec30 msgr2=0x7f3d640710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.415+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d6406ec30 0x7f3d640710f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009bd0 tx=0x7f3d74005f90 comp rx=0 tx=0).stop 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.415+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 msgr2=0x7f3d78072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.415+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 0x7f3d78072f40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3d68036040 tx=0x7f3d6802c750 comp rx=0 tx=0).stop 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 shutdown_connections 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f3d6406ec30 0x7f3d640710f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d78072ae0 0x7f3d78072f40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 --2- 172.21.15.31:0/1910777334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d78100280 0x7f3d780725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 >> 172.21.15.31:0/1910777334 conn(0x7f3d780fbaa0 msgr2=0x7f3d780fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 shutdown_connections 2024-04-22T23:58:01.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.416+0000 7f3d7f62f640 1 -- 172.21.15.31:0/1910777334 wait complete. 2024-04-22T23:58:01.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:58:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 -- 172.21.15.31:0/349969874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110100280 msgr2=0x7f2110100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 --2- 172.21.15.31:0/349969874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110100280 0x7f2110100660 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2104009880 tx=0x7f210402bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 -- 172.21.15.31:0/349969874 shutdown_connections 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 --2- 172.21.15.31:0/349969874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2110100c30 0x7f2110104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 --2- 172.21.15.31:0/349969874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110100280 0x7f2110100660 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 -- 172.21.15.31:0/349969874 >> 172.21.15.31:0/349969874 conn(0x7f21100fbaa0 msgr2=0x7f21100fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 -- 172.21.15.31:0/349969874 shutdown_connections 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.536+0000 7f211707d640 1 -- 172.21.15.31:0/349969874 wait complete. 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f211707d640 1 Processor -- start 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f211707d640 1 -- start start 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f211707d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2110100280 0x7f21100725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f211707d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 0x7f2110072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f211707d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2110074550 con 0x7f2110072ae0 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f211707d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21101a1f70 con 0x7f2110100280 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 0x7f2110072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 0x7f2110072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42416/0 (socket says 172.21.15.31:42416) 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 -- 172.21.15.31:0/1913653911 learned_addr learned my addr 172.21.15.31:0/1913653911 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f2114e19640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2110100280 0x7f21100725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 -- 172.21.15.31:0/1913653911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2110100280 msgr2=0x7f21100725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2110100280 0x7f21100725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 -- 172.21.15.31:0/1913653911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2104009560 con 0x7f2110072ae0 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.537+0000 7f210ffff640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 0x7f2110072f40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00d6c0 tx=0x7f20fc00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.538+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc0051f0 con 0x7f2110072ae0 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.538+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20fc009ce0 con 0x7f2110072ae0 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.538+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21101a21d0 con 0x7f2110072ae0 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.538+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20fc010460 con 0x7f2110072ae0 2024-04-22T23:58:01.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.538+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21101a2700 con 0x7f2110072ae0 2024-04-22T23:58:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.539+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211004e040 con 0x7f2110072ae0 2024-04-22T23:58:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.539+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f20fc005350 con 0x7f2110072ae0 2024-04-22T23:58:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.540+0000 7f210dffb640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f210006ec30 0x7f21000710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.540+0000 7f2114e19640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f210006ec30 0x7f21000710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.540+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f20fc08e9b0 con 0x7f2110072ae0 2024-04-22T23:58:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.540+0000 7f2114e19640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f210006ec30 0x7f21000710f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2104036040 tx=0x7f2104005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.545+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f20fc057670 con 0x7f2110072ae0 2024-04-22T23:58:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.730+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2110100c30 con 0x7f2110072ae0 2024-04-22T23:58:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.731+0000 7f210dffb640 1 -- 172.21.15.31:0/1913653911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7f20fc017020 con 0x7f2110072ae0 2024-04-22T23:58:01.721 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f210006ec30 msgr2=0x7f21000710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f210006ec30 0x7f21000710f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2104036040 tx=0x7f2104005fd0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 msgr2=0x7f2110072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 0x7f2110072f40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00d6c0 tx=0x7f20fc00db80 comp rx=0 tx=0).stop 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 shutdown_connections 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f210006ec30 0x7f21000710f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2110072ae0 0x7f2110072f40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 --2- 172.21.15.31:0/1913653911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2110100280 0x7f21100725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 >> 172.21.15.31:0/1913653911 conn(0x7f21100fbaa0 msgr2=0x7f21100fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 shutdown_connections 2024-04-22T23:58:01.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.733+0000 7f211707d640 1 -- 172.21.15.31:0/1913653911 wait complete. 2024-04-22T23:58:01.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T23:58:01.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 -- 172.21.15.31:0/3294236188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 msgr2=0x7fa3d40ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/3294236188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d40ff1a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009880 tx=0x7fa3d002bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 -- 172.21.15.31:0/3294236188 shutdown_connections 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/3294236188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 0x7fa3d410c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/3294236188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d40ff1a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 -- 172.21.15.31:0/3294236188 >> 172.21.15.31:0/3294236188 conn(0x7fa3d40fa9c0 msgr2=0x7fa3d40fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 -- 172.21.15.31:0/3294236188 shutdown_connections 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.847+0000 7fa3dbbae640 1 -- 172.21.15.31:0/3294236188 wait complete. 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3dbbae640 1 Processor -- start 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3dbbae640 1 -- start start 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3dbbae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3dbbae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 0x7fa3d4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3dbbae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d4198ac0 con 0x7fa3d40fedc0 2024-04-22T23:58:01.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3dbbae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419c890 con 0x7fa3d40ff6e0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42432/0 (socket says 172.21.15.31:42432) 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 -- 172.21.15.31:0/2572377811 learned_addr learned my addr 172.21.15.31:0/2572377811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d9149640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 0x7fa3d4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 -- 172.21.15.31:0/2572377811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 msgr2=0x7fa3d4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 0x7fa3d4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 -- 172.21.15.31:0/2572377811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009560 con 0x7fa3d40fedc0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d9149640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 0x7fa3d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.848+0000 7fa3d994a640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d4197ef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009230 tx=0x7fa3d0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.849+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3d00044a0 con 0x7fa3d40fedc0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.849+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d419cb10 con 0x7fa3d40fedc0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.849+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3d0003ab0 con 0x7fa3d40fedc0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.849+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3d00055f0 con 0x7fa3d40fedc0 2024-04-22T23:58:01.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.849+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d419d010 con 0x7fa3d40fedc0 2024-04-22T23:58:01.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.850+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d404e040 con 0x7fa3d40fedc0 2024-04-22T23:58:01.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.850+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa3d0004e60 con 0x7fa3d40fedc0 2024-04-22T23:58:01.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.850+0000 7fa3caffd640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa3c006ec30 0x7fa3c00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:01.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.851+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa3d00b1320 con 0x7fa3d40fedc0 2024-04-22T23:58:01.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.851+0000 7fa3d9149640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa3c006ec30 0x7fa3c00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:01.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.851+0000 7fa3d9149640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa3c006ec30 0x7fa3c00710f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4199500 tx=0x7fa3c4005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:01.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:01.855+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa3d0079f60 con 0x7fa3d40fedc0 2024-04-22T23:58:02.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.056+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3d4109b10 con 0x7fa3d40fedc0 2024-04-22T23:58:02.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.057+0000 7fa3caffd640 1 -- 172.21.15.31:0/2572377811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v92) v1 ==== 81+0+2423 (secure 0 0 0) 0x7fa3d007d620 con 0x7fa3d40fedc0 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6825/1404295013 2024-04-23T23:54:44.571005+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/418479598 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/1074135904 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2402370856 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3977887042 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6827/3065727368 2024-04-23T23:55:09.702810+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2198454822 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3574495240 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/2312976544 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1131104582 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:02.048 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/863972114 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6828/170975257 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/543782996 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2599356862 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1857354965 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6829/170975257 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3332075151 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/127255297 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2402370856 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/3035083544 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6826/3543943016 2024-04-23T23:42:43.116369+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1932777197 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6826/3065727368 2024-04-23T23:55:09.702810+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/2034571521 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/863972114 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6827/3543943016 2024-04-23T23:42:43.116369+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/3936244184 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3177298026 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4163451302 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3579734707 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3579734707 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1029776087 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6824/1404295013 2024-04-23T23:54:44.571005+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/3485102811 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1913595989 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1858202996 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/445726858 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2085576080 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2198454822 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:02.050 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1049639276 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:02.051 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/418479598 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:listed 41 entries 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa3c006ec30 msgr2=0x7fa3c00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa3c006ec30 0x7fa3c00710f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4199500 tx=0x7fa3c4005f90 comp rx=0 tx=0).stop 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 msgr2=0x7fa3d4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d4197ef0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009230 tx=0x7fa3d0005e50 comp rx=0 tx=0).stop 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 shutdown_connections 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa3c006ec30 0x7fa3c00710f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3d40ff6e0 0x7fa3d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 --2- 172.21.15.31:0/2572377811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fedc0 0x7fa3d4197ef0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 >> 172.21.15.31:0/2572377811 conn(0x7fa3d40fa9c0 msgr2=0x7fa3d40fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 shutdown_connections 2024-04-22T23:58:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.059+0000 7fa3dbbae640 1 -- 172.21.15.31:0/2572377811 wait complete. 2024-04-22T23:58:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:01 smithi031 ceph-mon[111289]: pgmap v301: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:01 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2055330150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:58:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:01 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1910777334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:58:02.060 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:01 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1913653911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:58:02.061 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-22T23:58:02.061 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:58:02.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:58:02.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:01 smithi044 ceph-mon[82146]: pgmap v301: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:02.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:01 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2055330150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:58:02.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:01 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1910777334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:58:02.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:01 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1913653911' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:58:02.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 -- 172.21.15.31:0/3731660599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4380ff360 msgr2=0x7fa438104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 --2- 172.21.15.31:0/3731660599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4380ff360 0x7fa438104ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa420009880 tx=0x7fa42002bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 -- 172.21.15.31:0/3731660599 shutdown_connections 2024-04-22T23:58:02.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 --2- 172.21.15.31:0/3731660599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4380ff360 0x7fa438104ce0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 --2- 172.21.15.31:0/3731660599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4380fe9b0 0x7fa4380fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 -- 172.21.15.31:0/3731660599 >> 172.21.15.31:0/3731660599 conn(0x7fa438075250 msgr2=0x7fa438075660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:02.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 -- 172.21.15.31:0/3731660599 shutdown_connections 2024-04-22T23:58:02.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.232+0000 7fa43d01c640 1 -- 172.21.15.31:0/3731660599 wait complete. 2024-04-22T23:58:02.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.233+0000 7fa43d01c640 1 Processor -- start 2024-04-22T23:58:02.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.233+0000 7fa43d01c640 1 -- start start 2024-04-22T23:58:02.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.233+0000 7fa43d01c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4380fe9b0 0x7fa438195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.233+0000 7fa43d01c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 0x7fa43819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.233+0000 7fa43d01c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4381968d0 con 0x7fa4381962d0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.233+0000 7fa43d01c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa438196a40 con 0x7fa4380fe9b0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 0x7fa43819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 0x7fa43819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42452/0 (socket says 172.21.15.31:42452) 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 -- 172.21.15.31:0/1923771383 learned_addr learned my addr 172.21.15.31:0/1923771383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 -- 172.21.15.31:0/1923771383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4380fe9b0 msgr2=0x7fa438195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4380fe9b0 0x7fa438195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 -- 172.21.15.31:0/1923771383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009560 con 0x7fa4381962d0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa42ffff640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 0x7fa43819a730 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa420036040 tx=0x7fa42002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa420004520 con 0x7fa4381962d0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa420003ab0 con 0x7fa4381962d0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43819acd0 con 0x7fa4381962d0 2024-04-22T23:58:02.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa420034440 con 0x7fa4381962d0 2024-04-22T23:58:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.234+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43819b230 con 0x7fa4381962d0 2024-04-22T23:58:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.235+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43804e040 con 0x7fa4381962d0 2024-04-22T23:58:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.236+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa420003c20 con 0x7fa4381962d0 2024-04-22T23:58:02.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.236+0000 7fa434d99640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa42406ec30 0x7fa4240710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.236+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa4200b1240 con 0x7fa4381962d0 2024-04-22T23:58:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.236+0000 7fa436d9d640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa42406ec30 0x7fa4240710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.237+0000 7fa436d9d640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa42406ec30 0x7fa4240710f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa428005fd0 tx=0x7fa42800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.241+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa420079f00 con 0x7fa4381962d0 2024-04-22T23:58:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.431+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa43819b4a0 con 0x7fa4381962d0 2024-04-22T23:58:02.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.432+0000 7fa434d99640 1 -- 172.21.15.31:0/1923771383 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17209 (secure 0 0 0) 0x7fa42007d5c0 con 0x7fa4381962d0 2024-04-22T23:58:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-22T23:58:02.423 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":92,"fsid":"efa92c84-0100-11ef-bc93-c7b262605968","created":"2024-04-22T23:35:35.005605+0000","modified":"2024-04-22T23:55:09.702849+0000","last_up_change":"2024-04-22T23:54:01.112143+0000","last_in_change":"2024-04-22T23:40:55.087287+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:39:59.535490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:42:25.830464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:42:26.380171+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a20f711-3635-473e-a87d-02f16913468e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":85,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6803","nonce":3402089089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6805","nonce":3402089089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6809","nonce":3402089089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":3402089089},{"type":"v1","addr":"172.21.15.31:6807","nonce":3402089089}]},"public_addr":"172.21.15.31:6803/3402089089","cluster_addr":"172.21.15.31:6805/3402089089","heartbeat_back_addr":"172.21.15.31:6809/3402089089","heartbeat_front_addr":"172.21.15.31:6807/3402089089","state":["exists","up"]},{"osd":1,"uuid":"b77430fc-df87-47da-a6c3-d237e3e0bc31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":62,"up_thru":85,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6811","nonce":753330875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6813","nonce":753330875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6817","nonce":753330875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":753330875},{"type":"v1","addr":"172.21.15.31:6815","nonce":753330875}]},"public_addr":"172.21.15.31:6811/753330875","cluster_addr":"172.21.15.31:6813/753330875","heartbeat_back_addr":"172.21.15.31:6817/753330875","heartbeat_front_addr":"172.21.15.31:6815/753330875","state":["exists","up"]},{"osd":2,"uuid":"27f0f9ea-73fe-4b71-84f8-31bd466c08e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":66,"up_from":70,"up_thru":85,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6819","nonce":1746177235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6821","nonce":1746177235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6825","nonce":1746177235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1746177235},{"type":"v1","addr":"172.21.15.31:6823","nonce":1746177235}]},"public_addr":"172.21.15.31:6819/1746177235","cluster_addr":"172.21.15.31:6821/1746177235","heartbeat_back_addr":"172.21.15.31:6825/1746177235","heartbeat_front_addr":"172.21.15.31:6823/1746177235","state":["exists","up"]},{"osd":3,"uuid":"2b88b2ea-7239-40b7-ad29-5ed83a3c5635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":71,"up_from":75,"up_thru":85,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6801","nonce":2576217506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6803","nonce":2576217506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6807","nonce":2576217506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2576217506},{"type":"v1","addr":"172.21.15.44:6805","nonce":2576217506}]},"public_addr":"172.21.15.44:6801/2576217506","cluster_addr":"172.21.15.44:6803/2576217506","heartbeat_back_addr":"172.21.15.44:6807/2576217506","heartbeat_front_addr":"172.21.15.44:6805/2576217506","state":["exists","up"]},{"osd":4,"uuid":"8f3c56b7-a76a-4e54-8327-4d99d8ac6ac5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":76,"up_from":80,"up_thru":85,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6809","nonce":2616608091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6811","nonce":2616608091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6815","nonce":2616608091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2616608091},{"type":"v1","addr":"172.21.15.44:6813","nonce":2616608091}]},"public_addr":"172.21.15.44:6809/2616608091","cluster_addr":"172.21.15.44:6811/2616608091","heartbeat_back_addr":"172.21.15.44:6815/2616608091","heartbeat_front_addr":"172.21.15.44:6813/2616608091","state":["exists","up"]},{"osd":5,"uuid":"1ef19c59-4032-4263-a6e0-519ebe782468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":81,"up_from":85,"up_thru":85,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6817","nonce":2666243870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6819","nonce":2666243870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6823","nonce":2666243870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2666243870},{"type":"v1","addr":"172.21.15.44:6821","nonce":2666243870}]},"public_addr":"172.21.15.44:6817/2666243870","cluster_addr":"172.21.15.44:6819/2666243870","heartbeat_back_addr":"172.21.15.44:6823/2666243870","heartbeat_front_addr":"172.21.15.44:6821/2666243870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:03.131418+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:29.992545+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:39:56.679257+0000","dead_epoch":66},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:20.646225+0000","dead_epoch":71},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:40:46.784122+0000","dead_epoch":76},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:41:12.495954+0000","dead_epoch":81}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1","mappings":[{"from":0,"to":5}]},{"pgid":"2.7","mappings":[{"from":3,"to":1}]},{"pgid":"2.c","mappings":[{"from":3,"to":1}]},{"pgid":"2.1a","mappings":[{"from":3,"to":1}]},{"pgid":"2.1d","mappings":[{"from":0,"to":2}]},{"pgid":"2.1f","mappings":[{"from":0,"to":1}]},{"pgid":"3.a","mappings":[{"from":3,"to":0}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":4,"to":5}]},{"pgid":"3.1b","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:6825/1404295013":"2024-04-23T23:54:44.571005+0000","172.21.15.31:6801/418479598":"2024-04-23T23:49:30.073919+0000","172.21.15.44:0/1074135904":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6801/2402370856":"2024-04-23T23:44:52.608776+0000","172.21.15.31:0/3977887042":"2024-04-23T23:36:39.456309+0000","172.21.15.44:6827/3065727368":"2024-04-23T23:55:09.702810+0000","172.21.15.31:6801/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/3574495240":"2024-04-23T23:44:52.608776+0000","172.21.15.44:0/2312976544":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/1131104582":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6801/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.44:6828/170975257":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/543782996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/2599356862":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/1857354965":"2024-04-23T23:44:52.608776+0000","172.21.15.44:6829/170975257":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/3332075151":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/127255297":"2024-04-23T23:44:52.608776+0000","172.21.15.31:6800/2402370856":"2024-04-23T23:44:52.608776+0000","172.21.15.44:0/3035083544":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6826/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.31:0/1932777197":"2024-04-23T23:37:45.021519+0000","172.21.15.44:6826/3065727368":"2024-04-23T23:55:09.702810+0000","172.21.15.44:0/2034571521":"2024-04-23T23:45:58.514810+0000","172.21.15.31:6800/863972114":"2024-04-23T23:36:39.456309+0000","172.21.15.31:6827/3543943016":"2024-04-23T23:42:43.116369+0000","172.21.15.44:0/3936244184":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/3177298026":"2024-04-23T23:36:39.456309+0000","172.21.15.31:0/4163451302":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:6801/3579734707":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/1029776087":"2024-04-23T23:44:52.608776+0000","172.21.15.44:6824/1404295013":"2024-04-23T23:54:44.571005+0000","172.21.15.44:0/3485102811":"2024-04-23T23:45:58.514810+0000","172.21.15.31:0/1913595989":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/1858202996":"2024-04-23T23:37:45.021519+0000","172.21.15.31:0/445726858":"2024-04-23T23:49:30.073919+0000","172.21.15.31:0/2085576080":"2024-04-23T23:49:30.073919+0000","172.21.15.31:6800/2198454822":"2024-04-23T23:36:06.362590+0000","172.21.15.31:0/1049639276":"2024-04-23T23:36:06.362590+0000","172.21.15.31:6800/418479598":"2024-04-23T23:49:30.073919+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:58:02.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.434+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa42406ec30 msgr2=0x7fa4240710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa42406ec30 0x7fa4240710f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa428005fd0 tx=0x7fa42800b410 comp rx=0 tx=0).stop 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 msgr2=0x7fa43819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 0x7fa43819a730 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa420036040 tx=0x7fa42002c7a0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 shutdown_connections 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa42406ec30 0x7fa4240710f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4381962d0 0x7fa43819a730 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 --2- 172.21.15.31:0/1923771383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa4380fe9b0 0x7fa438195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 >> 172.21.15.31:0/1923771383 conn(0x7fa438075250 msgr2=0x7fa4380fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 shutdown_connections 2024-04-22T23:58:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.435+0000 7fa43d01c640 1 -- 172.21.15.31:0/1923771383 wait complete. 2024-04-22T23:58:02.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:39:59.535490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:42:25.830464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:42:26.380171+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:58:02.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 -- 172.21.15.31:0/852638970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 msgr2=0x7fa370067780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 --2- 172.21.15.31:0/852638970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370067780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa364009880 tx=0x7fa36402bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 -- 172.21.15.31:0/852638970 shutdown_connections 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 --2- 172.21.15.31:0/852638970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370067780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 --2- 172.21.15.31:0/852638970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3701060c0 0x7fa3701064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 -- 172.21.15.31:0/852638970 >> 172.21.15.31:0/852638970 conn(0x7fa370075b50 msgr2=0x7fa370075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:02.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.551+0000 7fa376300640 1 -- 172.21.15.31:0/852638970 shutdown_connections 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 -- 172.21.15.31:0/852638970 wait complete. 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 Processor -- start 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 -- start start 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3701060c0 0x7fa370196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370196910 con 0x7fa370067340 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.552+0000 7fa376300640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37019a6e0 con 0x7fa3701060c0 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42462/0 (socket says 172.21.15.31:42462) 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 -- 172.21.15.31:0/711316875 learned_addr learned my addr 172.21.15.31:0/711316875 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 -- 172.21.15.31:0/711316875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3701060c0 msgr2=0x7fa370196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3701060c0 0x7fa370196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 -- 172.21.15.31:0/711316875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364009560 con 0x7fa370067340 2024-04-22T23:58:02.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36ffff640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370195d40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa36000d6f0 tx=0x7fa36000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:02.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa360009a30 con 0x7fa370067340 2024-04-22T23:58:02.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3600054e0 con 0x7fa370067340 2024-04-22T23:58:02.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37019a9c0 con 0x7fa370067340 2024-04-22T23:58:02.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa360010460 con 0x7fa370067340 2024-04-22T23:58:02.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.553+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37019af20 con 0x7fa370067340 2024-04-22T23:58:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.554+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37004e040 con 0x7fa370067340 2024-04-22T23:58:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.555+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7fa360009b90 con 0x7fa370067340 2024-04-22T23:58:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.555+0000 7fa36d7fa640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa35806ec30 0x7fa3580710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.555+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7fa36008eb80 con 0x7fa370067340 2024-04-22T23:58:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.555+0000 7fa36f7fe640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa35806ec30 0x7fa3580710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.556+0000 7fa36f7fe640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa35806ec30 0x7fa3580710f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa364036040 tx=0x7fa364000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:02.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.560+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa3600577c0 con 0x7fa370067340 2024-04-22T23:58:02.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.749+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa37010d2c0 con 0x7fa370067340 2024-04-22T23:58:02.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.749+0000 7fa36d7fa640 1 -- 172.21.15.31:0/711316875 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa360020020 con 0x7fa370067340 2024-04-22T23:58:02.740 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-22T23:58:02.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.751+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa35806ec30 msgr2=0x7fa3580710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.751+0000 7fa376300640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa35806ec30 0x7fa3580710f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa364036040 tx=0x7fa364000bc0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.751+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 msgr2=0x7fa370195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.751+0000 7fa376300640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370195d40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa36000d6f0 tx=0x7fa36000dbb0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 shutdown_connections 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7fa35806ec30 0x7fa3580710f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3701060c0 0x7fa370196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 --2- 172.21.15.31:0/711316875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa370067340 0x7fa370195d40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 >> 172.21.15.31:0/711316875 conn(0x7fa370075b50 msgr2=0x7fa3700fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 shutdown_connections 2024-04-22T23:58:02.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.752+0000 7fa376300640 1 -- 172.21.15.31:0/711316875 wait complete. 2024-04-22T23:58:02.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:58:02.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.866+0000 7f80d4990640 1 -- 172.21.15.31:0/2925997150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 msgr2=0x7f80d0105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.866+0000 7f80d4990640 1 --2- 172.21.15.31:0/2925997150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0105550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009880 tx=0x7f80c002bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.866+0000 7f80d4990640 1 -- 172.21.15.31:0/2925997150 shutdown_connections 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 --2- 172.21.15.31:0/2925997150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d00ff8e0 0x7f80d00ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 --2- 172.21.15.31:0/2925997150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0105550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 -- 172.21.15.31:0/2925997150 >> 172.21.15.31:0/2925997150 conn(0x7f80d00741e0 msgr2=0x7f80d00745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 -- 172.21.15.31:0/2925997150 shutdown_connections 2024-04-22T23:58:02.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 -- 172.21.15.31:0/2925997150 wait complete. 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 Processor -- start 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.867+0000 7f80d4990640 1 -- start start 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80d4990640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d00ff8e0 0x7f80d0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80d4990640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80d4990640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d0196b00 con 0x7f80d0105170 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80d4990640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d019a8d0 con 0x7f80d00ff8e0 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42470/0 (socket says 172.21.15.31:42470) 2024-04-22T23:58:02.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 -- 172.21.15.31:0/4200135328 learned_addr learned my addr 172.21.15.31:0/4200135328 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80ce59c640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d00ff8e0 0x7f80d0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 -- 172.21.15.31:0/4200135328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d00ff8e0 msgr2=0x7f80d0195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d00ff8e0 0x7f80d0195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 -- 172.21.15.31:0/4200135328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c0009560 con 0x7f80d0105170 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80cdd9b640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0196470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c810 tx=0x7f80c400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c40041f0 con 0x7f80d0105170 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d019abb0 con 0x7f80d0105170 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.868+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80c4007e70 con 0x7f80d0105170 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.869+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80c40049d0 con 0x7f80d0105170 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.869+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d019b110 con 0x7f80d0105170 2024-04-22T23:58:02.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.869+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d004e040 con 0x7f80d0105170 2024-04-22T23:58:02.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.870+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f80c40079b0 con 0x7f80d0105170 2024-04-22T23:58:02.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.870+0000 7f80bf7fe640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80b806ec30 0x7f80b80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:02.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.871+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f80c408ed70 con 0x7f80d0105170 2024-04-22T23:58:02.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.871+0000 7f80ce59c640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80b806ec30 0x7f80b80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:02.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.871+0000 7f80ce59c640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80b806ec30 0x7f80b80710f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f80c0006010 tx=0x7f80c0005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:02.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:02.875+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f80c4057a60 con 0x7f80d0105170 2024-04-22T23:58:03.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.066+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f80d01085e0 con 0x7f80d0105170 2024-04-22T23:58:03.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.067+0000 7f80bf7fe640 1 -- 172.21.15.31:0/4200135328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7f80c401d020 con 0x7f80d0105170 2024-04-22T23:58:03.057 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80b806ec30 msgr2=0x7f80b80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80b806ec30 0x7f80b80710f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f80c0006010 tx=0x7f80c0005b50 comp rx=0 tx=0).stop 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 msgr2=0x7f80d0196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0196470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f80c400c810 tx=0x7f80c400ccd0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 shutdown_connections 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f80b806ec30 0x7f80b80710f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80d0105170 0x7f80d0196470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 --2- 172.21.15.31:0/4200135328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80d00ff8e0 0x7f80d0195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 >> 172.21.15.31:0/4200135328 conn(0x7f80d00741e0 msgr2=0x7f80d00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 shutdown_connections 2024-04-22T23:58:03.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.069+0000 7f80d4990640 1 -- 172.21.15.31:0/4200135328 wait complete. 2024-04-22T23:58:03.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:02 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/2572377811' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:58:03.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:02 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1923771383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:58:03.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:02 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/711316875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:58:03.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 -- 172.21.15.31:0/4154472883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0fede0 msgr2=0x7f32cc0ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4154472883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0fede0 0x7f32cc0ff1c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009880 tx=0x7f32b402bce0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 -- 172.21.15.31:0/4154472883 shutdown_connections 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4154472883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32cc0ff790 0x7f32cc10c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4154472883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0fede0 0x7f32cc0ff1c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 -- 172.21.15.31:0/4154472883 >> 172.21.15.31:0/4154472883 conn(0x7f32cc0faa00 msgr2=0x7f32cc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 -- 172.21.15.31:0/4154472883 shutdown_connections 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.184+0000 7f32d23c5640 1 -- 172.21.15.31:0/4154472883 wait complete. 2024-04-22T23:58:03.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32d23c5640 1 Processor -- start 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32d23c5640 1 -- start start 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32d23c5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32cc0fede0 0x7f32cc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32d23c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 0x7f32cc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32d23c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc198aa0 con 0x7f32cc0ff790 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32d23c5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc19c870 con 0x7f32cc0fede0 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 0x7f32cc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 0x7f32cc198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42492/0 (socket says 172.21.15.31:42492) 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32cb7fe640 1 -- 172.21.15.31:0/4290854947 learned_addr learned my addr 172.21.15.31:0/4290854947 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.185+0000 7f32cbfff640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32cc0fede0 0x7f32cc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:03.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32cb7fe640 1 -- 172.21.15.31:0/4290854947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32cc0fede0 msgr2=0x7f32cc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32cb7fe640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32cc0fede0 0x7f32cc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32cb7fe640 1 -- 172.21.15.31:0/4290854947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b4009560 con 0x7f32cc0ff790 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32cb7fe640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 0x7f32cc198410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e970 tx=0x7f32bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32bc00cc90 con 0x7f32cc0ff790 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc19cb50 con 0x7f32cc0ff790 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32bc00cdf0 con 0x7f32cc0ff790 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32bc010430 con 0x7f32cc0ff790 2024-04-22T23:58:03.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.186+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc19d0b0 con 0x7f32cc0ff790 2024-04-22T23:58:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.187+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc04e040 con 0x7f32cc0ff790 2024-04-22T23:58:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.188+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f32bc0046c0 con 0x7f32cc0ff790 2024-04-22T23:58:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.188+0000 7f32c97fa640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f32b806ec30 0x7f32b80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:03.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.188+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f32bc014070 con 0x7f32cc0ff790 2024-04-22T23:58:03.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.188+0000 7f32cbfff640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f32b806ec30 0x7f32b80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:03.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.189+0000 7f32cbfff640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f32b806ec30 0x7f32b80710f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32b4006010 tx=0x7f32b402cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.193+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f32bc058180 con 0x7f32cc0ff790 2024-04-22T23:58:03.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:02 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/2572377811' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:58:03.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:02 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1923771383' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:58:03.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:02 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/711316875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:58:03.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.378+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f32cc109b30 con 0x7f32cc0ff790 2024-04-22T23:58:03.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.378+0000 7f32c97fa640 1 -- 172.21.15.31:0/4290854947 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7f32bc05b840 con 0x7f32cc0ff790 2024-04-22T23:58:03.369 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-22T23:58:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.380+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f32b806ec30 msgr2=0x7f32b80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.380+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f32b806ec30 0x7f32b80710f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32b4006010 tx=0x7f32b402cf90 comp rx=0 tx=0).stop 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.380+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 msgr2=0x7f32cc198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.380+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 0x7f32cc198410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e970 tx=0x7f32bc00ed70 comp rx=0 tx=0).stop 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 shutdown_connections 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f32b806ec30 0x7f32b80710f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32cc0ff790 0x7f32cc198410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 --2- 172.21.15.31:0/4290854947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32cc0fede0 0x7f32cc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 >> 172.21.15.31:0/4290854947 conn(0x7f32cc0faa00 msgr2=0x7f32cc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 shutdown_connections 2024-04-22T23:58:03.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.381+0000 7f32d23c5640 1 -- 172.21.15.31:0/4290854947 wait complete. 2024-04-22T23:58:03.382 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 -- 172.21.15.31:0/3846721879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8100260 msgr2=0x7f1ee8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 --2- 172.21.15.31:0/3846721879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8100260 0x7f1ee8100640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009c20 tx=0x7f1ee002c1b0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 -- 172.21.15.31:0/3846721879 shutdown_connections 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 --2- 172.21.15.31:0/3846721879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ee8100c10 0x7f1ee8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 --2- 172.21.15.31:0/3846721879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8100260 0x7f1ee8100640 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 -- 172.21.15.31:0/3846721879 >> 172.21.15.31:0/3846721879 conn(0x7f1ee80fbac0 msgr2=0x7f1ee80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 -- 172.21.15.31:0/3846721879 shutdown_connections 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.494+0000 7f1eeed63640 1 -- 172.21.15.31:0/3846721879 wait complete. 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eeed63640 1 Processor -- start 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eeed63640 1 -- start start 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eeed63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ee8100260 0x7f1ee80725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eeed63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 0x7f1ee8072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eeed63640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8074530 con 0x7f1ee8072ae0 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eeed63640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee81a1f00 con 0x7f1ee8100260 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1ee7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 0x7f1ee8072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1ee7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 0x7f1ee8072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42496/0 (socket says 172.21.15.31:42496) 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1ee7fff640 1 -- 172.21.15.31:0/1523027100 learned_addr learned my addr 172.21.15.31:0/1523027100 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.495+0000 7f1eecaff640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ee8100260 0x7f1ee80725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee7fff640 1 -- 172.21.15.31:0/1523027100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ee8100260 msgr2=0x7f1ee80725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee7fff640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ee8100260 0x7f1ee80725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee7fff640 1 -- 172.21.15.31:0/1523027100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0009900 con 0x7f1ee8072ae0 2024-04-22T23:58:03.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee7fff640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 0x7f1ee8072f40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c920 tx=0x7f1edc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:03.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc004410 con 0x7f1ee8072ae0 2024-04-22T23:58:03.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1edc007c60 con 0x7f1ee8072ae0 2024-04-22T23:58:03.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee81a2100 con 0x7f1ee8072ae0 2024-04-22T23:58:03.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc00f450 con 0x7f1ee8072ae0 2024-04-22T23:58:03.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.496+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee81a2660 con 0x7f1ee8072ae0 2024-04-22T23:58:03.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.497+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee804e040 con 0x7f1ee8072ae0 2024-04-22T23:58:03.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.497+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92745+0+0 (secure 0 0 0) 0x7f1edc0077a0 con 0x7f1ee8072ae0 2024-04-22T23:58:03.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.498+0000 7f1ee5ffb640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f1ed006ec30 0x7f1ed00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:58:03.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.498+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6876+0+0 (secure 0 0 0) 0x7f1edc08eee0 con 0x7f1ee8072ae0 2024-04-22T23:58:03.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.498+0000 7f1eecaff640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f1ed006ec30 0x7f1ed00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:58:03.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.498+0000 7f1eecaff640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f1ed006ec30 0x7f1ed00710f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0033000 tx=0x7f1ee0005f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:58:03.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.503+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f1edc057ba0 con 0x7f1ee8072ae0 2024-04-22T23:58:03.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.703+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1ee8108560 con 0x7f1ee8072ae0 2024-04-22T23:58:03.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.703+0000 7f1ee5ffb640 1 -- 172.21.15.31:0/1523027100 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v92) v1 ==== 81+0+2423 (secure 0 0 0) 0x7f1edc014990 con 0x7f1ee8072ae0 2024-04-22T23:58:03.694 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6825/1404295013 2024-04-23T23:54:44.571005+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/418479598 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/1074135904 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2402370856 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3977887042 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6827/3065727368 2024-04-23T23:55:09.702810+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2198454822 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3574495240 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/2312976544 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1131104582 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/863972114 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6828/170975257 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.695 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/543782996 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2599356862 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1857354965 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6829/170975257 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3332075151 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/127255297 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2402370856 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/3035083544 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6826/3543943016 2024-04-23T23:42:43.116369+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1932777197 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6826/3065727368 2024-04-23T23:55:09.702810+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/2034571521 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/863972114 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6827/3543943016 2024-04-23T23:42:43.116369+0000 2024-04-22T23:58:03.696 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/3936244184 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3177298026 2024-04-23T23:36:39.456309+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4163451302 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3579734707 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3579734707 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1029776087 2024-04-23T23:44:52.608776+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:6824/1404295013 2024-04-23T23:54:44.571005+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.44:0/3485102811 2024-04-23T23:45:58.514810+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1913595989 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1858202996 2024-04-23T23:37:45.021519+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/445726858 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2085576080 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2198454822 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1049639276 2024-04-23T23:36:06.362590+0000 2024-04-22T23:58:03.697 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/418479598 2024-04-23T23:49:30.073919+0000 2024-04-22T23:58:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:listed 41 entries 2024-04-22T23:58:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.705+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f1ed006ec30 msgr2=0x7f1ed00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.705+0000 7f1eeed63640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f1ed006ec30 0x7f1ed00710f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0033000 tx=0x7f1ee0005f10 comp rx=0 tx=0).stop 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.705+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 msgr2=0x7f1ee8072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.705+0000 7f1eeed63640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 0x7f1ee8072f40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c920 tx=0x7f1edc00cde0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.705+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 shutdown_connections 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.705+0000 7f1eeed63640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:6800/2602450284,v1:172.21.15.31:6801/2602450284] conn(0x7f1ed006ec30 0x7f1ed00710f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.706+0000 7f1eeed63640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ee8072ae0 0x7f1ee8072f40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.706+0000 7f1eeed63640 1 --2- 172.21.15.31:0/1523027100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ee8100260 0x7f1ee80725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.706+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 >> 172.21.15.31:0/1523027100 conn(0x7f1ee80fbac0 msgr2=0x7f1ee80fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.706+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 shutdown_connections 2024-04-22T23:58:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-22T23:58:03.706+0000 7f1eeed63640 1 -- 172.21.15.31:0/1523027100 wait complete. 2024-04-22T23:58:03.707 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-04-22T23:58:03.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:58:03.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-22T23:58:03.732 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T23:58:04.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:03 smithi031 ceph-mon[111289]: pgmap v302: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:04.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:03 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4200135328' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:58:04.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:03 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/4290854947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:58:04.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:03 smithi031 ceph-mon[111289]: from='client.? 172.21.15.31:0/1523027100' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:58:04.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:03 smithi044 ceph-mon[82146]: pgmap v302: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:04.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:03 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4200135328' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:58:04.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:03 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/4290854947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:58:04.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:03 smithi044 ceph-mon[82146]: from='client.? 172.21.15.31:0/1523027100' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:58:06.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:05 smithi031 ceph-mon[111289]: pgmap v303: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:06.200 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:05 smithi044 ceph-mon[82146]: pgmap v303: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:08.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:07 smithi031 ceph-mon[111289]: pgmap v304: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:08.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:07 smithi044 ceph-mon[82146]: pgmap v304: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:10.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:09 smithi031 ceph-mon[111289]: pgmap v305: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:10.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:09 smithi044 ceph-mon[82146]: pgmap v305: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:12.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:11 smithi031 ceph-mon[111289]: pgmap v306: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:12.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:11 smithi044 ceph-mon[82146]: pgmap v306: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:14.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:13 smithi031 ceph-mon[111289]: pgmap v307: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:14.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:13 smithi044 ceph-mon[82146]: pgmap v307: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:15 smithi031 ceph-mon[111289]: pgmap v308: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:16.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:15 smithi044 ceph-mon[82146]: pgmap v308: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:16.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:18.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:17 smithi031 ceph-mon[111289]: pgmap v309: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:18.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:17 smithi044 ceph-mon[82146]: pgmap v309: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:20.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:19 smithi031 ceph-mon[111289]: pgmap v310: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:20.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:19 smithi044 ceph-mon[82146]: pgmap v310: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:22.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:21 smithi031 ceph-mon[111289]: pgmap v311: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:22.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:21 smithi044 ceph-mon[82146]: pgmap v311: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:24.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:23 smithi044 ceph-mon[82146]: pgmap v312: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:24.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:23 smithi031 ceph-mon[111289]: pgmap v312: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:26.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:25 smithi044 ceph-mon[82146]: pgmap v313: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:26.229 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:25 smithi031 ceph-mon[111289]: pgmap v313: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:28.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:27 smithi044 ceph-mon[82146]: pgmap v314: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:28.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:27 smithi031 ceph-mon[111289]: pgmap v314: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:30.209 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:29 smithi044 ceph-mon[82146]: pgmap v315: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:30.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:29 smithi031 ceph-mon[111289]: pgmap v315: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:31.285 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:31.448 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:32.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:31 smithi031 ceph-mon[111289]: pgmap v316: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:32.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:31 smithi044 ceph-mon[82146]: pgmap v316: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:34.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:33 smithi031 ceph-mon[111289]: pgmap v317: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:58:34.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:33 smithi044 ceph-mon[82146]: pgmap v317: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:58:36.343 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:35 smithi031 ceph-mon[111289]: pgmap v318: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:58:36.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:35 smithi044 ceph-mon[82146]: pgmap v318: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-22T23:58:38.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:37 smithi031 ceph-mon[111289]: pgmap v319: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:58:38.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:37 smithi044 ceph-mon[82146]: pgmap v319: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:58:40.243 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:39 smithi031 ceph-mon[111289]: pgmap v320: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:58:40.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:39 smithi044 ceph-mon[82146]: pgmap v320: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-04-22T23:58:42.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:41 smithi031 ceph-mon[111289]: pgmap v321: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:42.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:41 smithi044 ceph-mon[82146]: pgmap v321: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:44.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:43 smithi031 ceph-mon[111289]: pgmap v322: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:44.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:43 smithi044 ceph-mon[82146]: pgmap v322: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:46.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:45 smithi031 ceph-mon[111289]: pgmap v323: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:46.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:46.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:45 smithi044 ceph-mon[82146]: pgmap v323: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:46.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:58:48.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:47 smithi031 ceph-mon[111289]: pgmap v324: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:48.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:47 smithi044 ceph-mon[82146]: pgmap v324: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:50.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:50 smithi031 ceph-mon[111289]: pgmap v325: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:50.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:50 smithi044 ceph-mon[82146]: pgmap v325: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:51.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:51 smithi031 ceph-mon[111289]: pgmap v326: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:51.459 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:51 smithi044 ceph-mon[82146]: pgmap v326: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:53.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:53 smithi031 ceph-mon[111289]: pgmap v327: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:53.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:53 smithi044 ceph-mon[82146]: pgmap v327: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:55.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:55 smithi031 ceph-mon[111289]: pgmap v328: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:55.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:55 smithi044 ceph-mon[82146]: pgmap v328: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:58:57.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:57 smithi031 ceph-mon[111289]: pgmap v329: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:57.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:57 smithi044 ceph-mon[82146]: pgmap v329: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s 2024-04-22T23:58:59.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:59 smithi031 ceph-mon[111289]: pgmap v330: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:59.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:58:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:58:59.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:59 smithi044 ceph-mon[82146]: pgmap v330: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:58:59.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:58:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:59:00.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:59:00.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:59:00.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:59:00.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:00.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:59:00.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:59:00.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-22T23:59:00.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:01.631 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:01 smithi031 ceph-mon[111289]: pgmap v331: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:01.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:01 smithi044 ceph-mon[82146]: pgmap v331: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:03.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:03 smithi031 ceph-mon[111289]: pgmap v332: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-22T23:59:03.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:03 smithi044 ceph-mon[82146]: pgmap v332: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-22T23:59:05.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:05 smithi031 ceph-mon[111289]: pgmap v333: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:05.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:05 smithi044 ceph-mon[82146]: pgmap v333: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:07.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:07 smithi031 ceph-mon[111289]: pgmap v334: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-22T23:59:07.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:07 smithi044 ceph-mon[82146]: pgmap v334: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 2 op/s 2024-04-22T23:59:09.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:09 smithi031 ceph-mon[111289]: pgmap v335: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-22T23:59:09.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:09 smithi044 ceph-mon[82146]: pgmap v335: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-22T23:59:11.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:11 smithi031 ceph-mon[111289]: pgmap v336: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:11.710 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:11 smithi044 ceph-mon[82146]: pgmap v336: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:13.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:13 smithi031 ceph-mon[111289]: pgmap v337: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-22T23:59:13.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:13 smithi044 ceph-mon[82146]: pgmap v337: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-04-22T23:59:15.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:15 smithi031 ceph-mon[111289]: pgmap v338: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:15.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:15.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:15 smithi044 ceph-mon[82146]: pgmap v338: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:15.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:17.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:17 smithi031 ceph-mon[111289]: pgmap v339: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:17.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:17 smithi044 ceph-mon[82146]: pgmap v339: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:19.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:19 smithi031 ceph-mon[111289]: pgmap v340: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:19.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:19 smithi044 ceph-mon[82146]: pgmap v340: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:21.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:21 smithi031 ceph-mon[111289]: pgmap v341: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:21.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:21 smithi044 ceph-mon[82146]: pgmap v341: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:23.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:23 smithi031 ceph-mon[111289]: pgmap v342: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:23.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:23 smithi044 ceph-mon[82146]: pgmap v342: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:25.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:25 smithi031 ceph-mon[111289]: pgmap v343: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:25.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:25 smithi044 ceph-mon[82146]: pgmap v343: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:27.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:27 smithi031 ceph-mon[111289]: pgmap v344: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:27.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:27 smithi044 ceph-mon[82146]: pgmap v344: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:29.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:29 smithi031 ceph-mon[111289]: pgmap v345: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:29.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:29 smithi044 ceph-mon[82146]: pgmap v345: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:30.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:30.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:31.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:31 smithi031 ceph-mon[111289]: pgmap v346: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:31.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:31 smithi044 ceph-mon[82146]: pgmap v346: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:33.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:33 smithi031 ceph-mon[111289]: pgmap v347: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:33.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:33 smithi044 ceph-mon[82146]: pgmap v347: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:35.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:35 smithi031 ceph-mon[111289]: pgmap v348: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:35.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:35 smithi044 ceph-mon[82146]: pgmap v348: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:37.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:37 smithi031 ceph-mon[111289]: pgmap v349: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:37.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:37 smithi044 ceph-mon[82146]: pgmap v349: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:39.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:39 smithi031 ceph-mon[111289]: pgmap v350: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:39.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:39 smithi044 ceph-mon[82146]: pgmap v350: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:41.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:41 smithi031 ceph-mon[111289]: pgmap v351: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:41.709 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:41 smithi044 ceph-mon[82146]: pgmap v351: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:43.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:43 smithi031 ceph-mon[111289]: pgmap v352: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:43.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:43 smithi044 ceph-mon[82146]: pgmap v352: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:45.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:45 smithi031 ceph-mon[111289]: pgmap v353: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:45.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:45.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:45 smithi044 ceph-mon[82146]: pgmap v353: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:59:47.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:47 smithi031 ceph-mon[111289]: pgmap v354: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:47.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:47 smithi044 ceph-mon[82146]: pgmap v354: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:49.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:49 smithi031 ceph-mon[111289]: pgmap v355: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:49.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:49 smithi044 ceph-mon[82146]: pgmap v355: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:51.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:51 smithi031 ceph-mon[111289]: pgmap v356: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:51.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:51 smithi044 ceph-mon[82146]: pgmap v356: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:53.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:53 smithi031 ceph-mon[111289]: pgmap v357: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:53.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:53 smithi044 ceph-mon[82146]: pgmap v357: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:55.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:55 smithi031 ceph-mon[111289]: pgmap v358: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:55.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:55 smithi044 ceph-mon[82146]: pgmap v358: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-22T23:59:57.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:57 smithi031 ceph-mon[111289]: pgmap v359: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:57.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:57 smithi044 ceph-mon[82146]: pgmap v359: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:59.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:59 smithi031 ceph-mon[111289]: pgmap v360: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:59.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 22 23:59:59 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:59:59.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:59 smithi044 ceph-mon[82146]: pgmap v360: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-22T23:59:59.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 22 23:59:59 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:00:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:00 smithi031 ceph-mon[111289]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T00:00:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:00:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:00:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:00:00.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:00 smithi044 ceph-mon[82146]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-23T00:00:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:00:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:00:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:00:00.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:01.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:01 smithi031 ceph-mon[111289]: pgmap v361: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:01.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:01 smithi044 ceph-mon[82146]: pgmap v361: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:03.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:03 smithi031 ceph-mon[111289]: pgmap v362: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:03.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:03 smithi044 ceph-mon[82146]: pgmap v362: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:05.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:05 smithi031 ceph-mon[111289]: pgmap v363: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:05.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:05 smithi044 ceph-mon[82146]: pgmap v363: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:07.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:07 smithi031 ceph-mon[111289]: pgmap v364: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:07.708 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:07 smithi044 ceph-mon[82146]: pgmap v364: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:09.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:09 smithi031 ceph-mon[111289]: pgmap v365: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:09.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:09 smithi044 ceph-mon[82146]: pgmap v365: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:11.744 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:11 smithi031 ceph-mon[111289]: pgmap v366: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:11.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:11 smithi044 ceph-mon[82146]: pgmap v366: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:13.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:13 smithi031 ceph-mon[111289]: pgmap v367: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:13.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:13 smithi044 ceph-mon[82146]: pgmap v367: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:15.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:15 smithi031 ceph-mon[111289]: pgmap v368: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:15.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:15.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:15 smithi044 ceph-mon[82146]: pgmap v368: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:15.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:17.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:17 smithi031 ceph-mon[111289]: pgmap v369: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:17.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:17 smithi044 ceph-mon[82146]: pgmap v369: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:19.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:19 smithi031 ceph-mon[111289]: pgmap v370: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:19.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:19 smithi044 ceph-mon[82146]: pgmap v370: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:21.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:21 smithi031 ceph-mon[111289]: pgmap v371: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:21.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:21 smithi044 ceph-mon[82146]: pgmap v371: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:23.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:23 smithi031 ceph-mon[111289]: pgmap v372: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:23.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:23 smithi044 ceph-mon[82146]: pgmap v372: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:25.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:25 smithi031 ceph-mon[111289]: pgmap v373: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:25.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:25 smithi044 ceph-mon[82146]: pgmap v373: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:27.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:27 smithi031 ceph-mon[111289]: pgmap v374: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:27.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:27 smithi044 ceph-mon[82146]: pgmap v374: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:29.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:29 smithi031 ceph-mon[111289]: pgmap v375: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:29.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:29 smithi044 ceph-mon[82146]: pgmap v375: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:30.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:30.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:31.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:31 smithi031 ceph-mon[111289]: pgmap v376: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:31.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:31 smithi044 ceph-mon[82146]: pgmap v376: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:33.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:33 smithi031 ceph-mon[111289]: pgmap v377: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:33.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:33 smithi044 ceph-mon[82146]: pgmap v377: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:35.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:35 smithi031 ceph-mon[111289]: pgmap v378: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:35.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:35 smithi044 ceph-mon[82146]: pgmap v378: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:37.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:37 smithi031 ceph-mon[111289]: pgmap v379: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:37.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:37 smithi044 ceph-mon[82146]: pgmap v379: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:39.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:39 smithi031 ceph-mon[111289]: pgmap v380: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:39.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:39 smithi044 ceph-mon[82146]: pgmap v380: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:41.744 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:41 smithi031 ceph-mon[111289]: pgmap v381: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:41.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:41 smithi044 ceph-mon[82146]: pgmap v381: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:43.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:43 smithi031 ceph-mon[111289]: pgmap v382: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:43.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:43 smithi044 ceph-mon[82146]: pgmap v382: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:45.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:45 smithi031 ceph-mon[111289]: pgmap v383: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:45.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:45.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:45 smithi044 ceph-mon[82146]: pgmap v383: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:45.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:00:47.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:47 smithi031 ceph-mon[111289]: pgmap v384: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:47.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:47 smithi044 ceph-mon[82146]: pgmap v384: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:49.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:49 smithi031 ceph-mon[111289]: pgmap v385: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:49.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:49 smithi044 ceph-mon[82146]: pgmap v385: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:51.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:51 smithi031 ceph-mon[111289]: pgmap v386: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:51.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:51 smithi044 ceph-mon[82146]: pgmap v386: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:53.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:53 smithi031 ceph-mon[111289]: pgmap v387: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:53.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:53 smithi044 ceph-mon[82146]: pgmap v387: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:55.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:55 smithi031 ceph-mon[111289]: pgmap v388: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:55.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:55 smithi044 ceph-mon[82146]: pgmap v388: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:00:57.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:57 smithi031 ceph-mon[111289]: pgmap v389: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:57.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:57 smithi044 ceph-mon[82146]: pgmap v389: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:59.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:00:59 smithi031 ceph-mon[111289]: pgmap v390: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:00:59.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:00:59 smithi044 ceph-mon[82146]: pgmap v390: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:00.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:00.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:01:00.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:00.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:01:01.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:01 smithi031 ceph-mon[111289]: pgmap v391: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:01.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:01:01.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:01:01.943 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:01:01.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:01 smithi044 ceph-mon[82146]: pgmap v391: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:01.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:01:01.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:01:01.959 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:01:03.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:03 smithi031 ceph-mon[111289]: pgmap v392: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:03.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:03 smithi044 ceph-mon[82146]: pgmap v392: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:05.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:05 smithi031 ceph-mon[111289]: pgmap v393: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:05.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:05 smithi044 ceph-mon[82146]: pgmap v393: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:07.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:07 smithi031 ceph-mon[111289]: pgmap v394: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:07.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:07 smithi044 ceph-mon[82146]: pgmap v394: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:09.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:09 smithi044 ceph-mon[82146]: pgmap v395: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:10.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:09 smithi031 ceph-mon[111289]: pgmap v395: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:11.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:11 smithi031 ceph-mon[111289]: pgmap v396: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:11.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:11 smithi044 ceph-mon[82146]: pgmap v396: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:13.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:13 smithi044 ceph-mon[82146]: pgmap v397: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:14.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:13 smithi031 ceph-mon[111289]: pgmap v397: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:15.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:15 smithi044 ceph-mon[82146]: pgmap v398: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:15.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:16.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:15 smithi031 ceph-mon[111289]: pgmap v398: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:16.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:17.958 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:17 smithi044 ceph-mon[82146]: pgmap v399: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:18.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:17 smithi031 ceph-mon[111289]: pgmap v399: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:20.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:19 smithi031 ceph-mon[111289]: pgmap v400: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:20.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:19 smithi044 ceph-mon[82146]: pgmap v400: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:21.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:21 smithi031 ceph-mon[111289]: pgmap v401: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:22.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:21 smithi044 ceph-mon[82146]: pgmap v401: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:24.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:23 smithi031 ceph-mon[111289]: pgmap v402: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:24.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:23 smithi044 ceph-mon[82146]: pgmap v402: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:26.010 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:25 smithi044 ceph-mon[82146]: pgmap v403: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:26.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:25 smithi031 ceph-mon[111289]: pgmap v403: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:28.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:27 smithi031 ceph-mon[111289]: pgmap v404: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:28.192 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:27 smithi044 ceph-mon[82146]: pgmap v404: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:30.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:29 smithi031 ceph-mon[111289]: pgmap v405: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:30.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:29 smithi044 ceph-mon[82146]: pgmap v405: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:31.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:31.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:31.942 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:31 smithi031 ceph-mon[111289]: pgmap v406: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:32.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:31 smithi044 ceph-mon[82146]: pgmap v406: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:34.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:33 smithi031 ceph-mon[111289]: pgmap v407: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:34.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:33 smithi044 ceph-mon[82146]: pgmap v407: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:36.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:35 smithi031 ceph-mon[111289]: pgmap v408: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:36.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:35 smithi044 ceph-mon[82146]: pgmap v408: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:38.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:37 smithi031 ceph-mon[111289]: pgmap v409: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:38.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:37 smithi044 ceph-mon[82146]: pgmap v409: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:40.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:39 smithi031 ceph-mon[111289]: pgmap v410: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:40.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:39 smithi044 ceph-mon[82146]: pgmap v410: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:42.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:41 smithi031 ceph-mon[111289]: pgmap v411: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:42.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:41 smithi044 ceph-mon[82146]: pgmap v411: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:44.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:43 smithi031 ceph-mon[111289]: pgmap v412: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:44.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:43 smithi044 ceph-mon[82146]: pgmap v412: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:46.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:45 smithi031 ceph-mon[111289]: pgmap v413: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:46.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:46.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:45 smithi044 ceph-mon[82146]: pgmap v413: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:46.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:01:48.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:47 smithi031 ceph-mon[111289]: pgmap v414: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:48.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:47 smithi044 ceph-mon[82146]: pgmap v414: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:50.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:49 smithi031 ceph-mon[111289]: pgmap v415: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:50.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:49 smithi044 ceph-mon[82146]: pgmap v415: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:52.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:51 smithi031 ceph-mon[111289]: pgmap v416: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:52.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:51 smithi044 ceph-mon[82146]: pgmap v416: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:53.941 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:53 smithi031 ceph-mon[111289]: pgmap v417: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:54.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:53 smithi044 ceph-mon[82146]: pgmap v417: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:56.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:55 smithi031 ceph-mon[111289]: pgmap v418: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:56.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:55 smithi044 ceph-mon[82146]: pgmap v418: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:01:58.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:57 smithi031 ceph-mon[111289]: pgmap v419: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:01:58.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:57 smithi044 ceph-mon[82146]: pgmap v419: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:00.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:01:59 smithi031 ceph-mon[111289]: pgmap v420: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:00.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:01:59 smithi044 ceph-mon[82146]: pgmap v420: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:01.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:01.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:02.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:01 smithi031 ceph-mon[111289]: pgmap v421: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:02.192 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:01 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:02:02.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:01 smithi044 ceph-mon[82146]: pgmap v421: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:02.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:01 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:02:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:02:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:02:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:02:03.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:02:03.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:02:03.208 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:02:04.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:03 smithi031 ceph-mon[111289]: pgmap v422: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:04.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:03 smithi044 ceph-mon[82146]: pgmap v422: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:06.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:05 smithi031 ceph-mon[111289]: pgmap v423: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:06.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:05 smithi044 ceph-mon[82146]: pgmap v423: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:08.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:07 smithi031 ceph-mon[111289]: pgmap v424: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:08.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:07 smithi044 ceph-mon[82146]: pgmap v424: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:10.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:09 smithi031 ceph-mon[111289]: pgmap v425: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:10.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:09 smithi044 ceph-mon[82146]: pgmap v425: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:12.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:11 smithi031 ceph-mon[111289]: pgmap v426: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:12.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:11 smithi044 ceph-mon[82146]: pgmap v426: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:13.957 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:13 smithi044 ceph-mon[82146]: pgmap v427: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:14.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:13 smithi031 ceph-mon[111289]: pgmap v427: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:16.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:15 smithi031 ceph-mon[111289]: pgmap v428: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:16.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:15 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:16.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:15 smithi044 ceph-mon[82146]: pgmap v428: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:16.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:15 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:18.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:17 smithi031 ceph-mon[111289]: pgmap v429: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:18.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:17 smithi044 ceph-mon[82146]: pgmap v429: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:20.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:19 smithi031 ceph-mon[111289]: pgmap v430: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:20.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:19 smithi044 ceph-mon[82146]: pgmap v430: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:22.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:21 smithi031 ceph-mon[111289]: pgmap v431: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:22.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:21 smithi044 ceph-mon[82146]: pgmap v431: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:24.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:23 smithi031 ceph-mon[111289]: pgmap v432: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:24.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:23 smithi044 ceph-mon[82146]: pgmap v432: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:26.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:25 smithi031 ceph-mon[111289]: pgmap v433: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:26.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:25 smithi044 ceph-mon[82146]: pgmap v433: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:28.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:27 smithi031 ceph-mon[111289]: pgmap v434: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:28.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:27 smithi044 ceph-mon[82146]: pgmap v434: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:30.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:29 smithi031 ceph-mon[111289]: pgmap v435: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:30.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:29 smithi044 ceph-mon[82146]: pgmap v435: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:31.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:30 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:31.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:30 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:32.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:31 smithi031 ceph-mon[111289]: pgmap v436: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:32.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:31 smithi044 ceph-mon[82146]: pgmap v436: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:34.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:33 smithi031 ceph-mon[111289]: pgmap v437: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:34.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:33 smithi044 ceph-mon[82146]: pgmap v437: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:36.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:35 smithi031 ceph-mon[111289]: pgmap v438: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:36.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:35 smithi044 ceph-mon[82146]: pgmap v438: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:38.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:37 smithi031 ceph-mon[111289]: pgmap v439: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:38.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:37 smithi044 ceph-mon[82146]: pgmap v439: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:40.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:39 smithi031 ceph-mon[111289]: pgmap v440: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:40.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:39 smithi044 ceph-mon[82146]: pgmap v440: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:42.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:41 smithi031 ceph-mon[111289]: pgmap v441: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:42.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:41 smithi044 ceph-mon[82146]: pgmap v441: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:44.174 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:43 smithi031 ceph-mon[111289]: pgmap v442: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:44.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:43 smithi044 ceph-mon[82146]: pgmap v442: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:46.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:45 smithi031 ceph-mon[111289]: pgmap v443: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:46.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:45 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:46.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:45 smithi044 ceph-mon[82146]: pgmap v443: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:46.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:45 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:02:48.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:47 smithi031 ceph-mon[111289]: pgmap v444: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:48.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:47 smithi044 ceph-mon[82146]: pgmap v444: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:50.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:49 smithi031 ceph-mon[111289]: pgmap v445: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:50.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:49 smithi044 ceph-mon[82146]: pgmap v445: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:52.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:51 smithi031 ceph-mon[111289]: pgmap v446: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:52.210 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:51 smithi044 ceph-mon[82146]: pgmap v446: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:54.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:53 smithi031 ceph-mon[111289]: pgmap v447: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:54.206 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:53 smithi044 ceph-mon[82146]: pgmap v447: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:56.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:55 smithi031 ceph-mon[111289]: pgmap v448: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:56.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:55 smithi044 ceph-mon[82146]: pgmap v448: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:02:58.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:57 smithi031 ceph-mon[111289]: pgmap v449: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:02:58.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:57 smithi044 ceph-mon[82146]: pgmap v449: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:03:00.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:02:59 smithi044 ceph-mon[82146]: pgmap v450: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:03:00.440 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:02:59 smithi031 ceph-mon[111289]: pgmap v450: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-04-23T00:03:01.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:00 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:03:01.440 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:00 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-23T00:03:02.206 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:01 smithi044 ceph-mon[82146]: pgmap v451: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:03:02.440 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:01 smithi031 ceph-mon[111289]: pgmap v451: 65 pgs: 65 active+clean; 256 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-23T00:03:02.801 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-23T00:03:02.802 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T00:03:02.803 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T00:03:02.823 INFO:tasks.cephadm:Teardown begin 2024-04-23T00:03:02.823 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T00:03:02.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T00:03:02.852 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T00:03:02.878 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T00:03:02.879 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T00:03:02.907 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T00:03:02.935 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T00:03:02.935 INFO:tasks.cephadm.mon.smithi031:Stopping mon.smithi031... 2024-04-23T00:03:02.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031 2024-04-23T00:03:03.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:03:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:03:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:03:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:02 smithi031 ceph-mon[111289]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:03:03.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:02 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:03.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T00:03:03.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T00:03:03.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T00:03:03.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:02 smithi044 ceph-mon[82146]: from='mgr.34104 172.21.15.31:0/2399986385' entity='mgr.smithi031.fazdcv' 2024-04-23T00:03:03.691 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:03 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[111285]: 2024-04-23T00:03:03.260+0000 7f7cdc8a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:03.691 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:03 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031[111285]: 2024-04-23T00:03:03.260+0000 7f7cdc8a3700 -1 mon.smithi031@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T00:03:03.691 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:03 smithi031 podman[157121]: 2024-04-23 00:03:03.495925122 +0000 UTC m=+0.350705203 container died 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T00:03:04.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:04 smithi031 podman[157121]: 2024-04-23 00:03:04.128015433 +0000 UTC m=+0.982795526 container cleanup 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, ceph=True, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T00:03:04.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:04 smithi031 bash[157121]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031 2024-04-23T00:03:04.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service' 2024-04-23T00:03:04.580 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:04 smithi031 podman[157146]: 2024-04-23 00:03:04.329799419 +0000 UTC m=+0.826219956 container remove 82d8b2207da32fc0d9ba54d1a957b264b2fe656ff8169c46be53ce24398657c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:04.580 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:04 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-04-23T00:03:04.580 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:04 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for efa92c84-0100-11ef-bc93-c7b262605968. 2024-04-23T00:03:04.581 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 23 00:03:04 smithi031 systemd[1]: ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi031.service: Consumed 11.750s CPU time. 2024-04-23T00:03:04.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:04.608 INFO:tasks.cephadm.mon.smithi031:Stopped mon.smithi031 2024-04-23T00:03:04.608 INFO:tasks.cephadm.mon.smithi044:Stopping mon.smithi044... 2024-04-23T00:03:04.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi044 2024-04-23T00:03:04.895 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:04 smithi044 systemd[1]: Stopping Ceph mon.smithi044 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:05.206 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:04 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044[82142]: 2024-04-23T00:03:04.906+0000 7f46ad8e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi044 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:05.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:04 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044[82142]: 2024-04-23T00:03:04.906+0000 7f46ad8e5700 -1 mon.smithi044@1(peon) e3 *** Got Signal Terminated *** 2024-04-23T00:03:05.207 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:05 smithi044 podman[119478]: 2024-04-23 00:03:05.02019742 +0000 UTC m=+0.201704568 container died 7af6a8053287c6ebe12b0a0ef7308dab1216dc6bfa54637ad7531a007dc2bbfc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T00:03:05.846 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:05 smithi044 podman[119478]: 2024-04-23 00:03:05.59084485 +0000 UTC m=+0.772352037 container cleanup 7af6a8053287c6ebe12b0a0ef7308dab1216dc6bfa54637ad7531a007dc2bbfc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T00:03:05.847 INFO:journalctl@ceph.mon.smithi044.smithi044.stdout:Apr 23 00:03:05 smithi044 bash[119478]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-mon-smithi044 2024-04-23T00:03:06.006 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@mon.smithi044.service' 2024-04-23T00:03:06.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:06.053 INFO:tasks.cephadm.mon.smithi044:Stopped mon.smithi044 2024-04-23T00:03:06.053 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T00:03:06.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.0 2024-04-23T00:03:06.363 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:06 smithi031 systemd[1]: Stopping Ceph osd.0 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:06.690 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:06 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-23T00:03:06.374+0000 7f14e0d48700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:06.690 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:06 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-23T00:03:06.374+0000 7f14e0d48700 -1 osd.0 92 *** Got signal Terminated *** 2024-04-23T00:03:06.690 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:06 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0[122899]: 2024-04-23T00:03:06.374+0000 7f14e0d48700 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T00:03:11.742 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:11 smithi031 podman[157254]: 2024-04-23 00:03:11.749510832 +0000 UTC m=+5.480297884 container died 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:12.380 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 podman[157254]: 2024-04-23 00:03:12.149939568 +0000 UTC m=+5.880726625 container cleanup 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0) 2024-04-23T00:03:12.381 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 bash[157254]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0 2024-04-23T00:03:12.690 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 podman[157271]: 2024-04-23 00:03:12.393203057 +0000 UTC m=+0.644331124 container remove 95e2a9c1a50ad90f5297b4b423bc8a01271d9693b5c3df33c1bdfce82a6d1f9f (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T00:03:13.035 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 podman[157371]: 2024-04-23 00:03:12.645706641 +0000 UTC m=+0.018970613 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-23T00:03:13.035 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 podman[157371]: 2024-04-23 00:03:12.74417275 +0000 UTC m=+0.117436704 container create bf9d85d27f2565a299b717f28a6465adcc66f9ca6d1f086805ef02c024180e47 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T00:03:13.035 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 podman[157371]: 2024-04-23 00:03:12.972374973 +0000 UTC m=+0.345638935 container init bf9d85d27f2565a299b717f28a6465adcc66f9ca6d1f086805ef02c024180e47 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T00:03:13.035 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:12 smithi031 podman[157371]: 2024-04-23 00:03:12.976411968 +0000 UTC m=+0.349675922 container start bf9d85d27f2565a299b717f28a6465adcc66f9ca6d1f086805ef02c024180e47 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:13.035 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:13 smithi031 podman[157371]: 2024-04-23 00:03:13.047127174 +0000 UTC m=+0.420391137 container attach bf9d85d27f2565a299b717f28a6465adcc66f9ca6d1f086805ef02c024180e47 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T00:03:13.293 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 23 00:03:13 smithi031 podman[157371]: 2024-04-23 00:03:13.287832909 +0000 UTC m=+0.661096862 container died bf9d85d27f2565a299b717f28a6465adcc66f9ca6d1f086805ef02c024180e47 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:13.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T00:03:13.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:13.913 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T00:03:13.913 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T00:03:13.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.1 2024-04-23T00:03:14.190 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:14 smithi031 systemd[1]: Stopping Ceph osd.1 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:14.511 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:14 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-23T00:03:14.262+0000 7feaa9829700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:14.511 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:14 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-23T00:03:14.262+0000 7feaa9829700 -1 osd.1 92 *** Got signal Terminated *** 2024-04-23T00:03:14.511 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:14 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1[128471]: 2024-04-23T00:03:14.262+0000 7feaa9829700 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T00:03:19.928 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:19 smithi031 podman[157479]: 2024-04-23 00:03:19.628947449 +0000 UTC m=+5.481876208 container died fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T00:03:20.190 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157479]: 2024-04-23 00:03:20.029670329 +0000 UTC m=+5.882599086 container cleanup fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T00:03:20.191 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 bash[157479]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1 2024-04-23T00:03:20.580 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157497]: 2024-04-23 00:03:20.24788346 +0000 UTC m=+0.619857572 container remove fff5512a8954423ad34c1bc99cdab7ac89a513291a47d1565967085fc93e5f1e (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T00:03:20.897 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157589]: 2024-04-23 00:03:20.496229732 +0000 UTC m=+0.019094781 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-23T00:03:20.897 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157589]: 2024-04-23 00:03:20.598724886 +0000 UTC m=+0.121589963 container create d8b710c8b03370b4d40d774100d4938fbe9dbdf714f9abe73870e9376f35ca58 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T00:03:20.897 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157589]: 2024-04-23 00:03:20.834689747 +0000 UTC m=+0.357554762 container init d8b710c8b03370b4d40d774100d4938fbe9dbdf714f9abe73870e9376f35ca58 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T00:03:20.897 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157589]: 2024-04-23 00:03:20.838291543 +0000 UTC m=+0.361156564 container start d8b710c8b03370b4d40d774100d4938fbe9dbdf714f9abe73870e9376f35ca58 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD) 2024-04-23T00:03:20.898 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:20 smithi031 podman[157589]: 2024-04-23 00:03:20.909586515 +0000 UTC m=+0.432451533 container attach d8b710c8b03370b4d40d774100d4938fbe9dbdf714f9abe73870e9376f35ca58 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:21.191 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 23 00:03:21 smithi031 podman[157589]: 2024-04-23 00:03:21.142336869 +0000 UTC m=+0.665201900 container died d8b710c8b03370b4d40d774100d4938fbe9dbdf714f9abe73870e9376f35ca58 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:21.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T00:03:21.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:21.773 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T00:03:21.774 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T00:03:21.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.2 2024-04-23T00:03:22.105 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:21 smithi031 systemd[1]: Stopping Ceph osd.2 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:22.440 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:22 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-23T00:03:22.115+0000 7f02be8f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:22.440 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:22 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-23T00:03:22.115+0000 7f02be8f9700 -1 osd.2 92 *** Got signal Terminated *** 2024-04-23T00:03:22.440 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:22 smithi031 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2[134334]: 2024-04-23T00:03:22.115+0000 7f02be8f9700 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T00:03:27.714 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:27 smithi031 podman[157697]: 2024-04-23 00:03:27.460283756 +0000 UTC m=+5.449960555 container died 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-23T00:03:28.072 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:27 smithi031 podman[157697]: 2024-04-23 00:03:27.816655025 +0000 UTC m=+5.806331824 container cleanup 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:28.072 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:27 smithi031 bash[157697]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2 2024-04-23T00:03:28.441 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:28 smithi031 podman[157714]: 2024-04-23 00:03:28.084726806 +0000 UTC m=+0.618927302 container remove 2cb66af4feded053db0dbcdab9137f2b78e187743837cca6e1f2f989179f73c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-23T00:03:28.701 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:28 smithi031 podman[157808]: 2024-04-23 00:03:28.360163559 +0000 UTC m=+0.019102413 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-23T00:03:28.701 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:28 smithi031 podman[157808]: 2024-04-23 00:03:28.494328325 +0000 UTC m=+0.153267195 container create ffeae901d5f2a76c71e10596d227fbd987cfaefa58c6a5f8492d69cc2c09fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS) 2024-04-23T00:03:28.701 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:28 smithi031 podman[157808]: 2024-04-23 00:03:28.713736678 +0000 UTC m=+0.372675512 container init ffeae901d5f2a76c71e10596d227fbd987cfaefa58c6a5f8492d69cc2c09fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T00:03:29.008 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:28 smithi031 podman[157808]: 2024-04-23 00:03:28.717609029 +0000 UTC m=+0.376547862 container start ffeae901d5f2a76c71e10596d227fbd987cfaefa58c6a5f8492d69cc2c09fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T00:03:29.008 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:28 smithi031 podman[157808]: 2024-04-23 00:03:28.82187 +0000 UTC m=+0.480808847 container attach ffeae901d5f2a76c71e10596d227fbd987cfaefa58c6a5f8492d69cc2c09fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T00:03:29.303 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 23 00:03:29 smithi031 podman[157808]: 2024-04-23 00:03:29.021203238 +0000 UTC m=+0.680142081 container died ffeae901d5f2a76c71e10596d227fbd987cfaefa58c6a5f8492d69cc2c09fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T00:03:29.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T00:03:29.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:29.627 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T00:03:29.627 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T00:03:29.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.3 2024-04-23T00:03:29.950 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:29 smithi044 systemd[1]: Stopping Ceph osd.3 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:30.206 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:29 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-23T00:03:29.971+0000 7f0bf3e96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:30.206 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:29 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-23T00:03:29.971+0000 7f0bf3e96700 -1 osd.3 92 *** Got signal Terminated *** 2024-04-23T00:03:30.207 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:29 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-23T00:03:29.972+0000 7f0bf3e96700 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T00:03:35.206 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:34 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3[91918]: 2024-04-23T00:03:34.967+0000 7f0bf048a700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.692301+0000 front 2024-04-23T00:03:10.692370+0000 (oldest deadline 2024-04-23T00:03:34.792079+0000) 2024-04-23T00:03:35.706 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:35 smithi044 podman[119601]: 2024-04-23 00:03:35.507863734 +0000 UTC m=+5.660084400 container died bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T00:03:35.707 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:35 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:35.333+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:35.707 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:35 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:35.542+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:36.136 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:35 smithi044 podman[119601]: 2024-04-23 00:03:35.89685729 +0000 UTC m=+6.049077954 container cleanup bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-23T00:03:36.136 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:35 smithi044 bash[119601]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3 2024-04-23T00:03:36.456 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:36 smithi044 podman[119630]: 2024-04-23 00:03:36.148571316 +0000 UTC m=+0.638857336 container remove bec8ae29bf316dc616d1a51f137c28c357425a3c663344efc4477a4a410e81ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T00:03:36.457 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:36 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:36.320+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:36.846 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:36 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:36.525+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:36.847 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:36 smithi044 podman[119722]: 2024-04-23 00:03:36.491798004 +0000 UTC m=+0.018239529 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-23T00:03:36.847 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:36 smithi044 podman[119722]: 2024-04-23 00:03:36.607859258 +0000 UTC m=+0.134300806 container create 2d51abe0543b9315dcb02d8c83ef6bbadad1157e091537f2e6abe4334b2fdeff (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True) 2024-04-23T00:03:37.206 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:36 smithi044 podman[119722]: 2024-04-23 00:03:36.986521281 +0000 UTC m=+0.512962798 container init 2d51abe0543b9315dcb02d8c83ef6bbadad1157e091537f2e6abe4334b2fdeff (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T00:03:37.207 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:36 smithi044 podman[119722]: 2024-04-23 00:03:36.989999514 +0000 UTC m=+0.516441027 container start 2d51abe0543b9315dcb02d8c83ef6bbadad1157e091537f2e6abe4334b2fdeff (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T00:03:37.207 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:37 smithi044 podman[119722]: 2024-04-23 00:03:37.060961032 +0000 UTC m=+0.587402560 container attach 2d51abe0543b9315dcb02d8c83ef6bbadad1157e091537f2e6abe4334b2fdeff (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T00:03:37.510 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:37 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:37.296+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:37.511 INFO:journalctl@ceph.osd.3.smithi044.stdout:Apr 23 00:03:37 smithi044 podman[119722]: 2024-04-23 00:03:37.297900478 +0000 UTC m=+0.824341987 container died 2d51abe0543b9315dcb02d8c83ef6bbadad1157e091537f2e6abe4334b2fdeff (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T00:03:37.869 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:37 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:37.522+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:37.895 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T00:03:37.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:37.941 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T00:03:37.941 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T00:03:37.941 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.4 2024-04-23T00:03:38.206 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:38 smithi044 systemd[1]: Stopping Ceph osd.4 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:38.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:38 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:38.552+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:38.706 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:38 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:38.333+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:38.707 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:38 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:38.392+0000 7faf601c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:38.707 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:38 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:38.392+0000 7faf601c1700 -1 osd.4 92 *** Got signal Terminated *** 2024-04-23T00:03:38.707 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:38 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:38.392+0000 7faf601c1700 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T00:03:39.706 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:39 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:39.376+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:39.707 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:39 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:39.536+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:40.706 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:40 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:40.351+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:40.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:40 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:40.570+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:41.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:41 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:41.547+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:41.707 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:41 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:41.375+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:41.707 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:41 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:41.375+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:15.124151+0000 front 2024-04-23T00:03:15.124039+0000 (oldest deadline 2024-04-23T00:03:41.023939+0000) 2024-04-23T00:03:42.706 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:42 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:42.353+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:42.706 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:42 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:42.353+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:15.124151+0000 front 2024-04-23T00:03:15.124039+0000 (oldest deadline 2024-04-23T00:03:41.023939+0000) 2024-04-23T00:03:42.707 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:42 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:42.573+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:42.707 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:42 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:42.573+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:43.577 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:43 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:43.337+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:11.123175+0000 front 2024-04-23T00:03:11.123225+0000 (oldest deadline 2024-04-23T00:03:35.123609+0000) 2024-04-23T00:03:43.577 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:43 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4[96840]: 2024-04-23T00:03:43.337+0000 7faf5c7b5700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:15.124151+0000 front 2024-04-23T00:03:15.124039+0000 (oldest deadline 2024-04-23T00:03:41.023939+0000) 2024-04-23T00:03:43.956 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:43 smithi044 podman[119829]: 2024-04-23 00:03:43.812534622 +0000 UTC m=+5.663129305 container died 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T00:03:43.957 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:43 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:43.589+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:43.957 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:43 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:43.589+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:44.348 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:44 smithi044 podman[119829]: 2024-04-23 00:03:44.151884882 +0000 UTC m=+6.002479555 container cleanup 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-23T00:03:44.349 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:44 smithi044 bash[119829]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4 2024-04-23T00:03:44.706 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:44 smithi044 podman[119846]: 2024-04-23 00:03:44.361545905 +0000 UTC m=+0.549026669 container remove 26f6a1ebe5e508dc3b5288c4a5814f49f325b5d10304229e55f6be8bd51bcb19 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-23T00:03:44.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:44 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:44.562+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:44.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:44 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:44.562+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:45.102 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:44 smithi044 podman[119933]: 2024-04-23 00:03:44.689078374 +0000 UTC m=+0.018211688 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-23T00:03:45.102 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:44 smithi044 podman[119933]: 2024-04-23 00:03:44.804136098 +0000 UTC m=+0.133269393 container create 83f27f27652064fc1ca9a51a9878c7a4b4aa78743e3a46206461a04f51042496 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T00:03:45.102 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:45 smithi044 podman[119933]: 2024-04-23 00:03:45.014978343 +0000 UTC m=+0.344111638 container init 83f27f27652064fc1ca9a51a9878c7a4b4aa78743e3a46206461a04f51042496 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0) 2024-04-23T00:03:45.102 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:45 smithi044 podman[119933]: 2024-04-23 00:03:45.018301172 +0000 UTC m=+0.347434467 container start 83f27f27652064fc1ca9a51a9878c7a4b4aa78743e3a46206461a04f51042496 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T00:03:45.102 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:45 smithi044 podman[119933]: 2024-04-23 00:03:45.114928771 +0000 UTC m=+0.444062066 container attach 83f27f27652064fc1ca9a51a9878c7a4b4aa78743e3a46206461a04f51042496 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-23T00:03:45.456 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 23 00:03:45 smithi044 podman[119933]: 2024-04-23 00:03:45.322947599 +0000 UTC m=+0.652081004 container died 83f27f27652064fc1ca9a51a9878c7a4b4aa78743e3a46206461a04f51042496 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T00:03:45.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:45 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:45.526+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:45.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:45 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:45.526+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:45.986 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T00:03:46.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:46.030 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T00:03:46.030 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T00:03:46.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.5 2024-04-23T00:03:46.351 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:46 smithi044 systemd[1]: Stopping Ceph osd.5 for efa92c84-0100-11ef-bc93-c7b262605968... 2024-04-23T00:03:46.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:46.363+0000 7f15b78f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T00:03:46.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:46.363+0000 7f15b78f0700 -1 osd.5 92 *** Got signal Terminated *** 2024-04-23T00:03:46.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:46.363+0000 7f15b78f0700 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T00:03:46.707 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:46.517+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:46.707 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:46 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:46.517+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:47.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:47 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:47.559+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:47.706 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:47 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:47.559+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:48.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:48 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:48.552+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:48.957 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:48 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:48.552+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:49.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:49 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:49.509+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:49.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:49 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:49.509+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:50.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:50 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:50.494+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-23T00:03:10.095597+0000 front 2024-04-23T00:03:10.095687+0000 (oldest deadline 2024-04-23T00:03:34.795473+0000) 2024-04-23T00:03:50.956 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:50 smithi044 ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5[102147]: 2024-04-23T00:03:50.494+0000 7f15b3ee4700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-23T00:03:14.796131+0000 front 2024-04-23T00:03:14.796079+0000 (oldest deadline 2024-04-23T00:03:41.796351+0000) 2024-04-23T00:03:51.935 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:51 smithi044 podman[120043]: 2024-04-23 00:03:51.681425621 +0000 UTC m=+5.408576104 container died 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T00:03:52.202 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120043]: 2024-04-23 00:03:52.022134051 +0000 UTC m=+5.749284525 container cleanup 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T00:03:52.202 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 bash[120043]: ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5 2024-04-23T00:03:52.456 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120061]: 2024-04-23 00:03:52.214994609 +0000 UTC m=+0.524290580 container remove 38dba54af5f6695eb00a1af7e7ab8346c94970f962285b1ce6ca2467022e30e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T00:03:52.880 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120152]: 2024-04-23 00:03:52.62403112 +0000 UTC m=+0.111178168 container create 00ba47f5e449ce53228f3a57a1c6428b1fc8837d424ef960078960291d1e66e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416) 2024-04-23T00:03:52.880 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120152]: 2024-04-23 00:03:52.530912611 +0000 UTC m=+0.018059671 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-23T00:03:52.881 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120152]: 2024-04-23 00:03:52.81888185 +0000 UTC m=+0.306028905 container init 00ba47f5e449ce53228f3a57a1c6428b1fc8837d424ef960078960291d1e66e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0) 2024-04-23T00:03:52.881 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120152]: 2024-04-23 00:03:52.822468704 +0000 UTC m=+0.309615752 container start 00ba47f5e449ce53228f3a57a1c6428b1fc8837d424ef960078960291d1e66e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T00:03:52.881 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:52 smithi044 podman[120152]: 2024-04-23 00:03:52.893410732 +0000 UTC m=+0.380557785 container attach 00ba47f5e449ce53228f3a57a1c6428b1fc8837d424ef960078960291d1e66e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-23T00:03:53.206 INFO:journalctl@ceph.osd.5.smithi044.stdout:Apr 23 00:03:53 smithi044 podman[120152]: 2024-04-23 00:03:53.125575847 +0000 UTC m=+0.612722893 container died 00ba47f5e449ce53228f3a57a1c6428b1fc8837d424ef960078960291d1e66e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-efa92c84-0100-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T00:03:53.814 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-efa92c84-0100-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T00:03:53.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T00:03:53.855 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T00:03:53.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid efa92c84-0100-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T00:03:54.019 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: efa92c84-0100-11ef-bc93-c7b262605968 2024-04-23T00:03:58.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[75066]: fuse finished with error 0 and tester_r 0 2024-04-23T00:04:16.187 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid efa92c84-0100-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T00:04:16.351 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: efa92c84-0100-11ef-bc93-c7b262605968 2024-04-23T00:04:31.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T00:04:31.741 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T00:04:31.767 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T00:04:31.768 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/remote/smithi031/crash 2024-04-23T00:04:31.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/crash -- . 2024-04-23T00:04:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T00:04:31.810 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-04-23T00:04:31.812 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/remote/smithi044/crash 2024-04-23T00:04:31.812 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/crash -- . 2024-04-23T00:04:31.836 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/efa92c84-0100-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T00:04:31.836 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-04-23T00:04:31.837 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T00:04:31.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T00:04:31.885 INFO:tasks.cephadm:Compressing logs... 2024-04-23T00:04:31.886 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T00:04:31.929 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T00:04:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log: No such file or directory 2024-04-23T00:04:31.953 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-04-23T00:04:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T00:04:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mon.smithi031.log 2024-04-23T00:04:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T00:04:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mon.smithi031.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mgr.smithi031.fazdcv.log 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.1.log/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.audit.log: 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mgr.smithi031.fazdcv.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.ezuwxk.log 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.cephadm.log: 88.5% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr: 86.0%/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.ezuwxk.log: -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T00:04:31.962 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi031.log 2024-04-23T00:04:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-volume.log: 91.5% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mon.smithi044.log 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi044.log 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mon.smithi044.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T00:04:31.965 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi044.log: /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mgr.smithi044.iglcti.log 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi044.onzoro.log 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mgr.smithi044.iglcti.log: 88.6% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi044.onzoro.log: 93.0% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi044.log.gz 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi044.ibkztg.log 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr: 91.7%/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.cephadm.log: -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr: 85.9% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T00:04:31.966 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi044.ibkztg.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-23T00:04:31.970 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T00:04:31.971 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.usiyoa.log 2024-04-23T00:04:31.973 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi031.log: 94.3% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi031.log.gz 2024-04-23T00:04:31.977 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.1.log: 91.0% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T00:04:31.978 INFO:teuthology.orchestra.run.smithi031.stderr: 91.0% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T00:04:31.979 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-23T00:04:31.979 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.usiyoa.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T00:04:31.994 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T00:04:32.010 INFO:teuthology.orchestra.run.smithi044.stderr: 89.3% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mgr.smithi044.iglcti.log.gz 2024-04-23T00:04:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T00:04:32.405 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mgr.smithi031.fazdcv.log.gz 2024-04-23T00:04:32.443 INFO:teuthology.orchestra.run.smithi044.stderr: 92.5% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mon.smithi044.log.gz 2024-04-23T00:04:33.296 INFO:teuthology.orchestra.run.smithi031.stderr: 90.5% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mon.smithi031.log.gz 2024-04-23T00:04:36.089 INFO:teuthology.orchestra.run.smithi044.stderr: 93.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T00:04:36.634 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T00:04:37.184 INFO:teuthology.orchestra.run.smithi044.stderr: 93.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T00:04:37.498 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T00:04:37.737 INFO:teuthology.orchestra.run.smithi031.stderr: 93.7% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T00:04:38.669 INFO:teuthology.orchestra.run.smithi044.stderr: 93.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T00:04:39.736 INFO:teuthology.orchestra.run.smithi031.stderr: 94.7% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.ezuwxk.log.gz 2024-04-23T00:04:39.965 INFO:teuthology.orchestra.run.smithi044.stderr: 94.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi044.onzoro.log.gz 2024-04-23T00:04:45.706 INFO:teuthology.orchestra.run.smithi031.stderr: 94.8% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.usiyoa.log.gz 2024-04-23T00:04:48.276 INFO:teuthology.orchestra.run.smithi044.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-23T00:04:48.276 INFO:teuthology.orchestra.run.smithi044.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-23T00:04:48.482 INFO:teuthology.orchestra.run.smithi031.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-23T00:04:48.484 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-04-23T00:04:48.484 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m16.542s 2024-04-23T00:04:48.484 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m54.075s 2024-04-23T00:04:48.484 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m1.683s 2024-04-23T00:06:03.830 INFO:teuthology.orchestra.run.smithi044.stderr: 92.7% -- replaced with /var/log/ceph/efa92c84-0100-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi044.ibkztg.log.gz 2024-04-23T00:06:03.832 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-23T00:06:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:real 1m31.889s 2024-04-23T00:06:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:user 2m9.063s 2024-04-23T00:06:03.832 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m3.538s 2024-04-23T00:06:03.832 INFO:tasks.cephadm:Archiving logs... 2024-04-23T00:06:03.833 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/remote/smithi031/log 2024-04-23T00:06:03.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T00:06:10.692 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/remote/smithi044/log 2024-04-23T00:06:10.693 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T00:06:29.271 INFO:tasks.cephadm:Removing cluster... 2024-04-23T00:06:29.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid efa92c84-0100-11ef-bc93-c7b262605968 --force 2024-04-23T00:06:29.432 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: efa92c84-0100-11ef-bc93-c7b262605968 2024-04-23T00:06:29.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid efa92c84-0100-11ef-bc93-c7b262605968 --force 2024-04-23T00:06:29.912 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: efa92c84-0100-11ef-bc93-c7b262605968 2024-04-23T00:06:30.304 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T00:06:30.304 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T00:06:30.321 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T00:06:30.336 INFO:tasks.cephadm:Teardown complete 2024-04-23T00:06:30.337 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T00:06:30.349 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T00:06:30.350 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T00:06:30.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T00:06:30.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T00:06:30.472 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T00:06:30.483 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T00:06:30.483 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T00:06:30.486 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T00:06:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-23T00:06:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-23T00:06:30.504 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T00:06:30.504 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-23T00:06:30.504 INFO:teuthology.orchestra.run.smithi031.stdout:^- hv01.front.sepia.ceph.com 2 6 377 7 +560us[ +566us] +/- 576ms 2024-04-23T00:06:30.504 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -170us[ -163us] +/- 59ms 2024-04-23T00:06:30.504 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +236us[ +236us] +/- 82ms 2024-04-23T00:06:30.504 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T00:06:30.506 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T00:06:30.506 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-23T00:06:30.506 INFO:teuthology.orchestra.run.smithi044.stdout:^- hv01.front.sepia.ceph.com 2 6 377 5 +541us[ +544us] +/- 576ms 2024-04-23T00:06:30.506 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -173us[ -170us] +/- 59ms 2024-04-23T00:06:30.506 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +226us[ +229us] +/- 82ms 2024-04-23T00:06:30.506 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T00:06:30.506 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T00:06:30.518 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T00:06:30.519 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T00:06:30.530 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T00:06:30.630 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T00:06:30.705 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T00:06:30.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T00:06:30.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T00:06:30.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T00:06:30.761 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T00:06:30.794 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T00:06:30.804 INFO:teuthology.task.internal:Duration was 2235.793718 seconds 2024-04-23T00:06:30.804 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T00:06:30.814 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T00:06:30.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T00:06:30.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T00:06:30.852 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T00:06:30.855 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T00:06:31.231 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T00:06:31.232 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-04-23T00:06:31.232 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T00:06:31.292 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-23T00:06:31.293 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T00:06:31.318 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T00:06:31.318 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T00:06:31.335 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T00:06:31.384 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T00:06:31.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T00:06:31.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T00:06:32.059 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T00:06:32.166 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T00:06:32.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T00:06:32.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T00:06:32.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T00:06:32.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T00:06:32.268 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T00:06:32.288 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-04-23T00:06:32.291 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-23T00:06:32.317 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T00:06:32.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T00:06:32.364 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T00:06:32.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T00:06:32.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T00:06:32.392 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T00:06:32.392 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/remote/smithi031 2024-04-23T00:06:32.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T00:06:32.476 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236/remote/smithi044 2024-04-23T00:06:32.476 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T00:06:32.518 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T00:06:32.519 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T00:06:32.521 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T00:06:32.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T00:06:32.593 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T00:06:32.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T00:06:32.604 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T00:06:32.604 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T00:06:32.607 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T00:06:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 23 00:06 /home/ubuntu/cephtest 2024-04-23T00:06:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: 548396 4 d--------- 2 ubuntu ubuntu 4096 Apr 22 23:42 /home/ubuntu/cephtest/mnt.0 2024-04-23T00:06:32.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T00:06:32.621 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-23T00:06:32.621 INFO:teuthology.orchestra.run.smithi031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-23T00:06:32.621 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-23T00:06:32.622 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T00:06:32.635 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T00:06:32.649 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-23T00:06:32.649 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T00:06:32.679 DEBUG:teuthology.nuke:shortname: smithi031 2024-04-23T00:06:32.679 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T00:06:32.706 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:23:57.527950', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBESVGLMkNtRRe7tETX/sK+TUY0nJ8AYlK9F1xxqjyxN75ulpFLBtUjlxBtE2ciBRMxb5W1aN3R5fVL9RrqD7pWQ='} 2024-04-23T00:06:32.718 DEBUG:teuthology.nuke:shortname: smithi044 2024-04-23T00:06:32.718 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T00:06:32.757 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669236', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:23:57.529779', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCmB9spPn3CIDgj/WILKVNRGeaMpgJey6jQF8fstR2T8K+07kmqyozznVXOu6KkIm6Jw91VigVeqSmLM5K/qMp4='} 2024-04-23T00:06:32.791 INFO:teuthology.orchestra.console.smithi031:Power off 2024-04-23T00:06:32.791 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T00:06:32.838 INFO:teuthology.orchestra.console.smithi044:Power off 2024-04-23T00:06:32.838 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T00:06:32.848 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2024-04-23T00:06:32.848 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:32.887 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-04-23T00:06:32.888 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:32.897 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-23T00:06:32.922 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-23T00:06:36.898 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:36.923 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:37.011 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-23T00:06:37.033 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-23T00:06:41.014 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:41.034 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:41.124 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-04-23T00:06:41.142 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-23T00:06:41.225 INFO:teuthology.orchestra.console.smithi031:Power off completed 2024-04-23T00:06:45.146 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T00:06:45.255 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-23T00:06:45.356 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-04-23T00:06:45.460 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2235.7937183380127 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_adking@teuthology status: fail success: false 2024-04-23T00:06:45.460 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T00:06:45.559 INFO:teuthology.run:FAIL